Background
The current implementation of Update() uses the spi_flash_* api to write and read from flash. These functions ignore the partition->encrypted flag and always write raw data to flash even if the partition is marked as encrypted.
Changes in this PR
Update() now uses the esp_partition_* api.
Wrapper functions for esp_partition_* added to ESP.cpp. This was done to maintain a consistent approach to the way the spi_flash_* functions were used. I note though that not all of the esp-idf functions are used are wrapped, for example esp_ota_get_next_update_partition() so it may be that these should not be added?
The current implementation of Update() changes the first (magic) byte of firmware to 0xFF on write, and then when the firmware is completely written changes it back to ESP_IMAGE_HEADER_MAGIC. This works without erasing the sector because flash bits can be changed from 1->0 (but not 0->1). If the flash is encrypted then the actual data written to flash will not be all ones, so this approach will not work. In addition, encrypted flash must be written in 16 byte blocks. So, instead of changing the first byte the changed code stashes the first 16 bytes, and starts writing at the 17th byte, leaving the first 16 bytes as 0xFF. Then, in _enablePartition() the stashed bytes can be successfully written.
Benefits
Whilst it's not possible to use encrypted flash directly from either the Arduino IDE or PIO it's reasonably straightforward to compile and flash a bootloader with the necessary support from a simple esp-idf project and then use ArduinoOTA for subsequent updates. This PR enables the use of this workflow until such time as encrypted flash is supported, and is a first (small) step toward adding support.
Regardless of the above, the esp_partition_* api is recommended over the api_flash_* api.
Application code should mostly use these esp_partition_* API functions instead of lower level spi_flash_* API functions. Partition table API functions do bounds checking and calculate correct offsets in flash, based on data stored in a partition table.
* ESP.getChipModel() returns model of the chip
* ESP.getChipCores() returns the core count.
* Example gives chip model, revision and core count.
* Read efuse for chipmodel
Co-authored-by: Martijn Scheepers <ms@SDNengineering.nl>
* Minimize HardwareSerial Receive and Transmit delays
* Remove uartRxFifoToQueue from esp-hal-uart.h
Co-authored-by: Me No Dev <me-no-dev@users.noreply.github.com>
* fix removeApbChangeCallback() error in spiStopBus()
spiStartBus() was using spiStopBus() to init the hardware, one of spiStopBus() functions is to unregister the runtime CPU clock speed change callback. But, spiStartBus() only wanted to init the hardware. This patch separates the hardware init into a standalone function spiInitBus() that both spiStartBus() and spiStopBus() call.
* Update esp32-hal-spi.c
capitalization problem
* API compatibility to AVR, ESP8266, et al
* Add non-blocking HardwareSerial::read(buffer, size) extension (ESP8266 portability)
* Refactor for fewer indirect calls.
With the >= used let's say you have four RMT inputs, each using 2 channels wide for their receive buffer. This is 4*2 = 8 buffers which is the number of hardware buffers (MAX_CHANNELS). But, for the fourth input the starting buffer will be 6 (this is correct, the buffers used for each input are 0-1, 2-3, 4-5, 6-7). But, 6+2 = 8 which is MAX_CHANNELS. This is valid but the >= would match against it and abort. It is correct to only abort if the value i+j is only greater than MAX_CHANNELS. Thus, a simple one character fix. Delete the equals sign.
* `ledcWriteTone()` added a `apbcallback()` evertime the tone value was non zero.
* `addApbChangeCallback()` did not detect duplicate callbacks.
* changed the apbcallback list to a double link to support roll forward, roll back execution. This made the sequences of clock change callback start with the newest registered -> to oldest on the `before` then oldest -> newest after the clock change. This made the UART debug log output have minimal gibberish during the clock change.
* change how the UART callback handled the MUTEX because if any `apbchangeCallback()` executed a `log_x()` a deadlock would occur.
This fixes#3555
* Update esp32-hal-bt.c
BluetoothSerial crash when restart: this is because the BT controller remains in state ESP_BT_CONTROLLER_STATUS_INITED instead of state ESP_BT_CONTROLLER_STATUS_IDLE after the end() method.
in file esp_bt.h it is specified
> @brief Enable BT controller.
> Due to a known issue, you cannot call esp_bt_controller_enable() a second time
> to change the controller mode dynamically. To change controller mode, call
> esp_bt_controller_disable() and then call esp_bt_controller_enable() with the new mode.
after **esp_bt_controller_disable()** the controller remains in state INITED so we do call the **esp_bt_controller_deinit()** function to put the controller into state IDLE.
i have modified the **esp32-hal-bt.c** file
line 57 and next
(i have insert the esp_bt_controller_deinit() function so the controller go into Idle state)
```c++
bool btStop(){
if(esp_bt_controller_get_status() == ESP_BT_CONTROLLER_STATUS_IDLE){
log_i("bt stopped");
return true;
}
if(esp_bt_controller_get_status() == ESP_BT_CONTROLLER_STATUS_ENABLED){
log_i("bt enabled");
if (esp_bt_controller_disable()) {
log_e("BT Disable failed");
return false;
}
while(esp_bt_controller_get_status() == ESP_BT_CONTROLLER_STATUS_ENABLED);
}
if(esp_bt_controller_get_status() == ESP_BT_CONTROLLER_STATUS_INITED){
log_i("inited");
if (esp_bt_controller_deinit()) {
log_e("BT deint failed");
return false;
}
while (esp_bt_controller_get_status() == ESP_BT_CONTROLLER_STATUS_INITED);
return true;
}
log_e("BT Stop failed");
return false;
}
```
* Update esp32-hal-bt.c
remove while to avoid infinite loop
* SPI: Fix discarded-qalifiers warning when compiling with all warnings
This fixes an error introduced with changeset b847f41 which
tightened the use of const for read-only data. The helper
funtion __transferBytes also requires the const qualifier on
outgoing data. Without this change a warning is displayed
when compiling with the Arduino IDE set to display "All"
compiler warnings.
Tests:
- Build an ESP32 SPI sketch that uses static const data to send
to an SPI device using the SPI.transferBytes() API
* SPI:Ensure all local functions are marked static
This audits all functions in the esp32-hal-xpi.c module and
ensures that any functions entirely local to the module are
marked as static.
Tests:
- Build with Arduino set to show all warnings and ensure none
are displayed
* SPI: Remove unused local __spiTranslate24 function
This removes the __spiTranslate24() function which is unused.
* add option to Flush() to only clear txQueue
Add the option to cause Flush() to just wait for tx data to clear the tx fifo and uart, leave the rx queue and rx fifo as is.
* support tx only flush()
* support tx only Flush()
* support txOnly for Flush()
* compatibility to Stream()
* compatibility for Stream()
* default value error
* default value error
* Update esp32-hal-uart.h
* Update esp32-hal-uart.c
* Update HardwareSerial.cpp
* sp
* correctly implement flushTxOnly()
This changes all SPI functions that take data pointers which are
not modified so that the declaration is const. This allows them
to be used with const data (i.e. held in flash). No functional
changes are required.
The defnitions of spiWrite() and spiTransferBytes() in
esp-hal-spi.h/c have been updated to be consistent.
Tests:
- Build a simple sketch using SPI.writePattern() and
SPI.transferBytes() which uses const data and verify that the
attached device functions as expected.
* Expose uartStartDetectBaudrate(uart_t *) in esp32-hal-uart.h and call it from HardwareSerial::begin() if baudrate detection is requested (by passing a baudrate of 0) to solve baudrate detection problems
* Avoid a division by zero error in uartGetBaudRate()
* Use loc_buf for small strings, check for error return from vsnprintf
* cleanup arg when bailing out of new
* Use malloc/free instead of new/delete in printf
* Return actual bytes written in printf
* FIX: write before free
* Other Arduino cores uses a macro to redefine libc abs() to take any
type, meaning abs(-3.3) == 3.3 not the normal libc result of 3.
* 1e4bf14a3 (#1783) replaced similar min, max macros with c++ stdlib. However
this change includes <algorithm> after the line which defines the abs() macro.
<algorithm> includes <cstdlib> which undefines abs() and re-defines it.
* This means abs() becomes the plain libc version again which only takes
integers, so abs(-3.3) == 3. As reported here:
https://github.com/espressif/esp-idf/issues/3405
This fix tries to keep in the spirit of #1783 by using libstdc++. The other
option would be to include <cstdlib> before defining the abs() macro, so it
doesn't get undef-ed again later on.
As found by @mongozmaki in https://github.com/esp8266/Arduino/pull/6035
With SSO implementation in String, StreamString::write generates wrong
strings under some circumstances. Reason is that String::len() returns
strlen(sso_buf) if SSO=true but with newly written data
(in StreamString::write) the null-termination missing at the time len()
is called.
Furthermore, len() is called twice which is inefficient if SSO=true.
I redid the ESP8266 WString library to enable small string optimization
(SSO) a while back, and think it would be helpful even on the ESP32 with
its higher memory complement.
SSO avoids lots of tiny mallocs() on the heap which cause fragmentation
by using the memory in the class object itself to store the actual
string and only mallocing() for buffers that are larger than what can
fit in thie class object. Modern C++ std::string implementations have
this optimization as well, but since we're using Arduino strings we had
to roll our own.
* add opportunity for more than one retry to _uploadReadByte
* an alternative timeout-based method to making _uploadReadByte more resilient
* move timing variables in the correct scope
* implement and use client.getTimeout instead of hard-coded timeout in _uploadReadByte
* add missing return
* some refactoring to address respecting the timeout in a potentially deadlocked connection
* fix spelling in comment
* address review comments; move impl to cpp file for getTimeout, and remove local variable for currentMillis
* remove redundant cast
* need to check for timeout outside the inner while as well
* update WebUpdate example to print something in unexpected callback condition
* update log_e messages per review comments
The i2c peripheral will hang if a READ request is issued with a zero data length. The peripheral
drops into a continuous timeout interrupt response. The STOP command can not be set out to the I2C
bus. The SLAVE device correctly ACK'd the address byte, with READ bit set, it has control of the SDA
pin. The ESP32 send out the next SCL HIGH pulse but, since the SLAVE is in READ Mode, and the First
bit it is sending happened to be a ZERO, the ESP32 cannot send the STOP. When it releases SDA during
the SCL HIGH, the pin does not change state. The pin stays low because the SLAVE is outputing a LOW!
The ESP32 drops into a perminent WAIT state waiting for SDA to go HIGH (the STOP).
**esp32-hal-i2c.c**
* add databuff length checks to `i2cRead()` and `i2cWrite()`
* Add APB change callbacks and move cpu code to own file
* Properly set esp_timer and FreeRTOS tick dividers
* Improve updated devisors
* No need to update REF_TICK yet
* Add initial handling for UART baud change
* fix uartWriteBuf and uartDetectBaudrate
* trigger callbacks even when APB did not change
* toggle UART ISR on CPU change
* add XTAL freq getter and add cpu freq validation
* Support CPU frequency changes in I2C (#2287)
**esp32-hal-i2c.c**
* add callback for cpu frequency changes
* adjust fifo thresholds based on cpu frequency and i2c bus frequency
* reduce i2c bus frequency if differential is too small
**Wire.h**
* version to 1.1.0
* Implement clock change for the other peripherals
* remove bad CPU clock values from the menu
* Add note to CPU freqs that support WiFi and BT
The I2C hardware has limitations on min and max bus frequency directly related to CPU frequency, bus speed cannot be greater than 1/100 CPU clock, nor less than CPU clock / 8192.
* Add support to HAL for APB frequencies different than 80MHz
* Add support for CPU frequencies in the IDE board menu
* Switch to fast set_config
* Add method to uart so debug can be reassigned after apb frequency switch
* Return real APB frequency
* Fix error in PR #2048: if ::available() is called before ::connect() _rxBuffer is not initialised
* Fixed flash size check and added SPIFFS size check
* Rewriting ESP.getFreeSketchSpace(), moving code from HTTPUpdate.cpp
* Clean warnings when all warning enabled
Not used variables / functions due to debug log
Dual define with different values :
cores\esp32/binary.h
#define B110 6
#define B1000000 64
tools/sdk/include/newlib/sys/termios.h
#define B110 3
#define B1000000 23
Local variable returned in WiFiclient Secure
* change due to deprecated function
* Update with proper variable and label
* Update esp32-hal-i2c.c
* Apply changes requested
* Fix warnings due to #define conflict thanks @atanisoft
* Added baudrate detection to esp32-hal-uart and HardwareSerial
* Solved compiler warning for uartResizeRxBuffer()
* Add unit to header variable name (timeout_ms)
* Reverting accidentally changed files to master
* Add small delay after baudrate detection
#1869 exposed a resource exhaustion issue. The current HAL layer for I2C support is designed to use a shared interrupt, But, during debugging to solve the interrupt overloading condition identified in #1588, and the generation of pr #1717, the interrupt allocation parameters were changed. This change was unnecessary, the code will work successfully with shared interrupts. So, there is no need to assign a private interrupt for each I2C peripheral.
* TX Flow Control and Code cleanup
* Use semaphore instead of delay
TX functionality is done.
* Use single buffer and empty queue on exit
* Fix compile issues because of LwIP code relocation
* Add temporary header to fix Azure not compiling
* Fix AsyncUDP early init
* AsyncUDP Multicast fixes
* Add source mac address and rework multicast
* Allow redefinition of default pins for Serials 1 and 2
* Update IDF to 3276a13
* Update esptool.py to 2.5.0
* Fix sketches
* Fix log level in BluetoothSetial
* rmt driver initial version
* supporting conti mode plus interrupts
* using conitnous mode for sending more data
* working continous mode
* rmt driver cleanup after conti mode
* initial version of rmt driver
* adding a simple example
* adding channel and block locks
* modified of rmt interface for simpler/easier usage
* adding header sentinels, split interface to common and additional settings
* Fixes per code review + support for rx callback mode
* renamed internal structures and enums, fixed formatting
* cmake support for rmt
* refactored tx-conti interrupts to function to make it more readable
* added Tx and Rx examples
* added license headers
* minor updates per review
* used struct access, renamed defines, corrected diagram
If the esp32 is reset during a i2c read cycle the slave device may be in control of the SDA line.
If the SDA line is held low, the esp32 cannot issue a START or STOP to recover the bus.
The previous code did not correctly configure the SCL output pin, and it cycled SCL 9 times with SDA Low. Since the slave device was in a READ cycle, it just continued outputting the bits of the current byte. When the ACK/NAK bit space occurred, The low output value of SDA was interpreted as ACK so the slave device continued with the next byte. It never terminated the READ cycle.
This new code will correctly recover from an interrupted READ
* ReSTART fix, Sequencing fix
pr #1665 introduce a problem with ReSTART, when solving this problem I found an interaction between the TxFifo refill, RxFifo empty and CMD[] fill. during certain sequences a dataqueue command would be skipped, this skipping resulted in a mismatch between the contents of the TxFifo and the i2c command sequence. The problem manifested as an ACK error.
In addition to this required bug fix I propose:
* `Wire.begin()` be changed from a `void` to a `bool` this will allow the reset functionality of `Wire.begin()` to be reported. Currently `Wire.begin()` attempts to reset the i2c Peripheral, but cannot report success/failure.
* `Wire.busy()` be added. this `bool` function returns the hardware status of the bus. This status can be use in multi-master environments for application level interleaving of commands, also in single master environment, it can be used to detect a 'hung' bus. With the functional change to `Wire.begin()` this allows app level recover of a hung bus.
* `Wire.lastError()` value updated for all errors, previously when interleaving `Wire.endTransmission(false)` and `Wire.readTransmission(false)`, the 128 byte `Wire.write()` buffer was exhausted without generating and error(very exotic). I discovered this error when I created a sequence of directed reads to a EEPROM. Each directed read used 2 bytes of the 128 byte `write()` buffer, so after 64 consecutive ReSTART writes with ReSTART reads, `Wire()` had no room to record the directed address bytes. It generated just a NAK check without setting the EEPROMs internal register address. The succeeding ReSTART read succeeded at incorrect address.
* Changes to the HAL layer:
** added `i2cGetStatus()` which returns the i2c peripheral status word, used to detect bus_busy currently
** added `i2cDebug()` programmatic control of debug buffer output
** changed `i2cAddQueue()` to allow data_only queue element this will allow a i2c transaction to use multiple data pointers.
** removed direct access to DumpInts(), DumpI2c() from app, use i2cDebug() to set trigger points
*
* Update esp32-hal-i2c.c
* Update Wire.cpp
* ReSTART, Sequencing
pr #1665 introduce a problem with ReSTART, when solving this problem I found an interaction between the TxFifo refill, RxFifo empty and CMD[] fill. during certain sequences a dataqueue command would be skipped, this skipping resulted in a mismatch between the contents of the TxFifo and the i2c command sequence. The problem manifested as an ACK error.
In addition to this required bug fix I propose:
* `Wire.begin()` be changed from a `void` to a `bool` this will allow the reset functionality of `Wire.begin()` to be reported. Currently `Wire.begin()` attempts to reset the i2c Peripheral, but cannot report success/failure.
* `Wire.busy()` be added. this `bool` function returns the hardware status of the bus. This status can be use in multi-master environments for application level interleaving of commands, also in single master environment, it can be used to detect a 'hung' bus. With the functional change to `Wire.begin()` this allows app level recover of a hung bus.
* `Wire.lastError()` value updated for all errors, previously when interleaving `Wire.endTransmission(false)` and `Wire.readTransmission(false)`, the 128 byte `Wire.write()` buffer was exhausted without generating and error(very exotic). I discovered this error when I created a sequence of directed reads to a EEPROM. Each directed read used 2 bytes of the 128 byte `write()` buffer, so after 64 consecutive ReSTART writes with ReSTART reads, `Wire()` had no room to record the directed address bytes. It generated just a NAK check without setting the EEPROMs internal register address. The succeeding ReSTART read succeeded at incorrect address.
* Changes to the HAL layer:
** added `i2cGetStatus()` which returns the i2c peripheral status word, used to detect bus_busy currently
** added `i2cDebug()` programmatic control of debug buffer output
** changed `i2cAddQueue()` to allow data_only queue element this will allow a i2c transaction to use multiple data pointers.
** removed direct access to DumpInts(), DumpI2c() from app, use i2cDebug() to set trigger points
*
* Forgot DebugFlags Return
@andriyadi found this, total brain fade on my part.
