Background
The current implementation of Update() uses the spi_flash_* api to write and read from flash. These functions ignore the partition->encrypted flag and always write raw data to flash even if the partition is marked as encrypted.
Changes in this PR
Update() now uses the esp_partition_* api.
Wrapper functions for esp_partition_* added to ESP.cpp. This was done to maintain a consistent approach to the way the spi_flash_* functions were used. I note though that not all of the esp-idf functions are used are wrapped, for example esp_ota_get_next_update_partition() so it may be that these should not be added?
The current implementation of Update() changes the first (magic) byte of firmware to 0xFF on write, and then when the firmware is completely written changes it back to ESP_IMAGE_HEADER_MAGIC. This works without erasing the sector because flash bits can be changed from 1->0 (but not 0->1). If the flash is encrypted then the actual data written to flash will not be all ones, so this approach will not work. In addition, encrypted flash must be written in 16 byte blocks. So, instead of changing the first byte the changed code stashes the first 16 bytes, and starts writing at the 17th byte, leaving the first 16 bytes as 0xFF. Then, in _enablePartition() the stashed bytes can be successfully written.
Benefits
Whilst it's not possible to use encrypted flash directly from either the Arduino IDE or PIO it's reasonably straightforward to compile and flash a bootloader with the necessary support from a simple esp-idf project and then use ArduinoOTA for subsequent updates. This PR enables the use of this workflow until such time as encrypted flash is supported, and is a first (small) step toward adding support.
Regardless of the above, the esp_partition_* api is recommended over the api_flash_* api.
Application code should mostly use these esp_partition_* API functions instead of lower level spi_flash_* API functions. Partition table API functions do bounds checking and calculate correct offsets in flash, based on data stored in a partition table.
* ESP.getChipModel() returns model of the chip
* ESP.getChipCores() returns the core count.
* Example gives chip model, revision and core count.
* Read efuse for chipmodel
Co-authored-by: Martijn Scheepers <ms@SDNengineering.nl>
* Minimize HardwareSerial Receive and Transmit delays
* Remove uartRxFifoToQueue from esp-hal-uart.h
Co-authored-by: Me No Dev <me-no-dev@users.noreply.github.com>
* fix removeApbChangeCallback() error in spiStopBus()
spiStartBus() was using spiStopBus() to init the hardware, one of spiStopBus() functions is to unregister the runtime CPU clock speed change callback. But, spiStartBus() only wanted to init the hardware. This patch separates the hardware init into a standalone function spiInitBus() that both spiStartBus() and spiStopBus() call.
* Update esp32-hal-spi.c
capitalization problem
* API compatibility to AVR, ESP8266, et al
* Add non-blocking HardwareSerial::read(buffer, size) extension (ESP8266 portability)
* Refactor for fewer indirect calls.
With the >= used let's say you have four RMT inputs, each using 2 channels wide for their receive buffer. This is 4*2 = 8 buffers which is the number of hardware buffers (MAX_CHANNELS). But, for the fourth input the starting buffer will be 6 (this is correct, the buffers used for each input are 0-1, 2-3, 4-5, 6-7). But, 6+2 = 8 which is MAX_CHANNELS. This is valid but the >= would match against it and abort. It is correct to only abort if the value i+j is only greater than MAX_CHANNELS. Thus, a simple one character fix. Delete the equals sign.
* `ledcWriteTone()` added a `apbcallback()` evertime the tone value was non zero.
* `addApbChangeCallback()` did not detect duplicate callbacks.
* changed the apbcallback list to a double link to support roll forward, roll back execution. This made the sequences of clock change callback start with the newest registered -> to oldest on the `before` then oldest -> newest after the clock change. This made the UART debug log output have minimal gibberish during the clock change.
* change how the UART callback handled the MUTEX because if any `apbchangeCallback()` executed a `log_x()` a deadlock would occur.
This fixes#3555
* Update esp32-hal-bt.c
BluetoothSerial crash when restart: this is because the BT controller remains in state ESP_BT_CONTROLLER_STATUS_INITED instead of state ESP_BT_CONTROLLER_STATUS_IDLE after the end() method.
in file esp_bt.h it is specified
> @brief Enable BT controller.
> Due to a known issue, you cannot call esp_bt_controller_enable() a second time
> to change the controller mode dynamically. To change controller mode, call
> esp_bt_controller_disable() and then call esp_bt_controller_enable() with the new mode.
after **esp_bt_controller_disable()** the controller remains in state INITED so we do call the **esp_bt_controller_deinit()** function to put the controller into state IDLE.
i have modified the **esp32-hal-bt.c** file
line 57 and next
(i have insert the esp_bt_controller_deinit() function so the controller go into Idle state)
```c++
bool btStop(){
if(esp_bt_controller_get_status() == ESP_BT_CONTROLLER_STATUS_IDLE){
log_i("bt stopped");
return true;
}
if(esp_bt_controller_get_status() == ESP_BT_CONTROLLER_STATUS_ENABLED){
log_i("bt enabled");
if (esp_bt_controller_disable()) {
log_e("BT Disable failed");
return false;
}
while(esp_bt_controller_get_status() == ESP_BT_CONTROLLER_STATUS_ENABLED);
}
if(esp_bt_controller_get_status() == ESP_BT_CONTROLLER_STATUS_INITED){
log_i("inited");
if (esp_bt_controller_deinit()) {
log_e("BT deint failed");
return false;
}
while (esp_bt_controller_get_status() == ESP_BT_CONTROLLER_STATUS_INITED);
return true;
}
log_e("BT Stop failed");
return false;
}
```
* Update esp32-hal-bt.c
remove while to avoid infinite loop
* SPI: Fix discarded-qalifiers warning when compiling with all warnings
This fixes an error introduced with changeset b847f41 which
tightened the use of const for read-only data. The helper
funtion __transferBytes also requires the const qualifier on
outgoing data. Without this change a warning is displayed
when compiling with the Arduino IDE set to display "All"
compiler warnings.
Tests:
- Build an ESP32 SPI sketch that uses static const data to send
to an SPI device using the SPI.transferBytes() API
* SPI:Ensure all local functions are marked static
This audits all functions in the esp32-hal-xpi.c module and
ensures that any functions entirely local to the module are
marked as static.
Tests:
- Build with Arduino set to show all warnings and ensure none
are displayed
* SPI: Remove unused local __spiTranslate24 function
This removes the __spiTranslate24() function which is unused.
* add option to Flush() to only clear txQueue
Add the option to cause Flush() to just wait for tx data to clear the tx fifo and uart, leave the rx queue and rx fifo as is.
* support tx only flush()
* support tx only Flush()
* support txOnly for Flush()
* compatibility to Stream()
* compatibility for Stream()
* default value error
* default value error
* Update esp32-hal-uart.h
* Update esp32-hal-uart.c
* Update HardwareSerial.cpp
* sp
* correctly implement flushTxOnly()
This changes all SPI functions that take data pointers which are
not modified so that the declaration is const. This allows them
to be used with const data (i.e. held in flash). No functional
changes are required.
The defnitions of spiWrite() and spiTransferBytes() in
esp-hal-spi.h/c have been updated to be consistent.
Tests:
- Build a simple sketch using SPI.writePattern() and
SPI.transferBytes() which uses const data and verify that the
attached device functions as expected.
* Expose uartStartDetectBaudrate(uart_t *) in esp32-hal-uart.h and call it from HardwareSerial::begin() if baudrate detection is requested (by passing a baudrate of 0) to solve baudrate detection problems
* Avoid a division by zero error in uartGetBaudRate()
* Use loc_buf for small strings, check for error return from vsnprintf
* cleanup arg when bailing out of new
* Use malloc/free instead of new/delete in printf
* Return actual bytes written in printf
* FIX: write before free
* Other Arduino cores uses a macro to redefine libc abs() to take any
type, meaning abs(-3.3) == 3.3 not the normal libc result of 3.
* 1e4bf14a3 (#1783) replaced similar min, max macros with c++ stdlib. However
this change includes <algorithm> after the line which defines the abs() macro.
<algorithm> includes <cstdlib> which undefines abs() and re-defines it.
* This means abs() becomes the plain libc version again which only takes
integers, so abs(-3.3) == 3. As reported here:
https://github.com/espressif/esp-idf/issues/3405
This fix tries to keep in the spirit of #1783 by using libstdc++. The other
option would be to include <cstdlib> before defining the abs() macro, so it
doesn't get undef-ed again later on.
As found by @mongozmaki in https://github.com/esp8266/Arduino/pull/6035
With SSO implementation in String, StreamString::write generates wrong
strings under some circumstances. Reason is that String::len() returns
strlen(sso_buf) if SSO=true but with newly written data
(in StreamString::write) the null-termination missing at the time len()
is called.
Furthermore, len() is called twice which is inefficient if SSO=true.
I redid the ESP8266 WString library to enable small string optimization
(SSO) a while back, and think it would be helpful even on the ESP32 with
its higher memory complement.
SSO avoids lots of tiny mallocs() on the heap which cause fragmentation
by using the memory in the class object itself to store the actual
string and only mallocing() for buffers that are larger than what can
fit in thie class object. Modern C++ std::string implementations have
this optimization as well, but since we're using Arduino strings we had
to roll our own.
* add opportunity for more than one retry to _uploadReadByte
* an alternative timeout-based method to making _uploadReadByte more resilient
* move timing variables in the correct scope
* implement and use client.getTimeout instead of hard-coded timeout in _uploadReadByte
* add missing return
* some refactoring to address respecting the timeout in a potentially deadlocked connection
* fix spelling in comment
* address review comments; move impl to cpp file for getTimeout, and remove local variable for currentMillis
* remove redundant cast
* need to check for timeout outside the inner while as well
* update WebUpdate example to print something in unexpected callback condition
* update log_e messages per review comments
