* Speed up upload by a factor of 17
Uploads are very slow because of an unnecessary "client.connected()" check in _uploadReadByte().
Here is what happens:
client.connected() is called for every byte read. WiFiClient::connected() calls recv(fd(), &dummy, 0, MSG_DONTWAIT); which takes a relatively long time, so the optimized path of returning a buffered byte via client.read() is effectively nullified.
Removing the one line changed the upload speed for a 2 MB file (discarding the received data) from 22 KB/sec (before) to 367 KB/sec (after).
The change is safe in the face of disconnects because client.read(), when it no longer has buffered data, calls (WiFiClient) fillBuffer(), which calls recv(), so the disconnection will be detected in due course.
* Move disconnect check into the timeout loop
Compiler complains on unused parameter pCharacteristicMap:
framework-arduinoespressif32\libraries\BLE\src\BLERemoteService.cpp: In member function 'void BLERemoteService::getCharacteristics(std::map<short unsigned int, BLERemoteCharacteristic*>*)':
\framework-arduinoespressif32\libraries\BLE\src\BLERemoteService.cpp:246:89: warning: parameter 'pCharacteristicMap' set but not used [-Wunused-but-set-parameter]
void BLERemoteService::getCharacteristics(std::map<uint16_t, BLERemoteCharacteristic*>* pCharacteristicMap) {
See #3960 for more details of the problem and the solution. I only implemented what was proposed in this ticket, as it solves my problem, which was the same as in this ticket. Credits for the code going to @etrinh ;-)
This also is a more consistence behaviour compared to esp8266, where it also is possible to restart the wifiserver immediately on the same port.
* fix typo in WiFiMulti
* clean up trailing spaces
* clean up script file used in cleaning
Co-authored-by: Me No Dev <me-no-dev@users.noreply.github.com>
* improve & fix BLEScan when too many BLE devices
- when wantDuplicates, no need to check duplicate and no more insert into vector
- delete advertisedDevice when not insert into vector, fix memory leak
- add showParse when you just want raw advertised data
Remove device from Peer list if connection fails.
Only call onConnect callback if connection was successful.
Only call onDisconnect callback if the connection was previously connected (ESP_GATTC_DISCONNECT_EVT is fired on a unsuccessful connection attempt also).
Resolves a number of issues with phantom events and callbacks being fired.
::addPeerDevice and ::removePeerDevice are called with m_appId, so should ::updatePeerDevice as all use the same parameter for the underlying map's key.
* Fix for issue #4158: Crash with stack trace originating in Bluedroid
Improved configuration of scan response data in 'BLEAdvertising' avoids the crash:
- Added member variable 'm_scanRespData' to configure scan response differently from advertising data
- Initialization of 'm_scanRespData' in BLEAdvertising constructor
- Use of 'm_scanRespData' within BLEAdvertising::start() to configure the scan response
- 'Flags' and 'Appearance' are cleared in the scan response data
- With this fix, device names of up to 29 characters can be used without causing a crash.
Fixes: #3460
This code has been run in production for 1 month and it looks stable, no data dropped and it definitely fixes the issue described. I think that this can be merged to avoid using custom package referencing in PlatformIO that has been used in quite a few projects for now.
Co-authored-by: Ivan Golubic <ivan@mvt-solutions.com>
The UpdateClass in the Updater component has the ability to update data to
a SPIFFS partition. It selects the first available partition using the
ESP-IDF esp_partition_find_first() function.
That behaviour is problematic if one has multiple SPIFFS partitions.
This change allows a user to pass the label argument (defaults to NULL)
to UpdateClass::begin() so a specific SPIFFS partition can be updated.
Additionally, ArduinoOTA can set this partition label using the
new method ArduinoOTAClass::setPartitionLabel which is optional.
This change does not break compatibility.
* Add partition label argument to SPIFFSSPIFFS currently assumes there is only ever one partition.This change allows a user to pass the label argument (defaults to NULL)to SPIFFS::begin() so a specific SPIFFS partition can be referenced.This change does not break compatibility.
* BLERemoteChar: fix descriptor 2902 write for characteristic notifications
When registering a notification on a characteristic, the 2902 descriptor
(CCCD) value is set to 1 (or 2 for indication).
According to the BLUETOOTH CORE SPECIFICATION Version 5.2, Revision Date
2019-12-31, section 4.12.3 "Write Characteristic Descriptors" (page 1588),
the characteristic descriptor write must expect a response.
Currently, the descriptor write is performed without expecting a reponse,
which prevents the notification to be functional with some BLE stacks.
This commit modify the write to expect the response.
Signed-off-by: Jimmy Durand Wesolowski <jimmy.durand.wesolowski@commsolid.com>
* BLERemoteChar: forward GATT client event to characteristic descriptors
This commits prevents a permanent wait when calling BLERemoteDescriptor
readValue function, on the m_semaphoreReadDescrEvt semaphore.
ESP32 BLE stack calls to remote characteristic
- notification,
- value read
- value write
and remote characteristic descriptor
- value read
are asynchronous.
When such a call is performed by this library, a semaphore is taken prior
to the BLE stack read or write operation, and waited on after it.
Releasing the semaphore is done by the characteristic event handling
function (gattClientEventHandler), when the appropriate event is received.
However, the characteristic descriptor events are discarded, and the
value read semaphore is never released.
This commits forwards the GATT client events from the remote
characteristic down to their remote characteristic descriptor, and
implements their event handling.
Adding a semaphore for the remote characteristic descriptor value write
will be done in a separate commit.
Signed-off-by: Jimmy Durand Wesolowski <jimmy.durand.wesolowski@commsolid.com>
* BLERemoteDescriptor: add semaphore to characteristic descriptor write
This adds a semaphore to characteristic descriptor value write, to mimic
the value read function, and to ensure completion of the operation before
we carry on.
Signed-off-by: Jimmy Durand Wesolowski <jimmy.durand.wesolowski@commsolid.com>
Co-authored-by: Jimmy Durand Wesolowski <jimmy.durand.wesolowski@commsolid.com>
* Add an aditional (void *) arg to the RMT callback - to allow more flexible handling of the callback (e.g. by passing a private struct or a class pointer). Same pattern as used by the Ticker() and many others. Example updated & new example with a trapoline added.
* Fix example for new API
* Fix lint warnings
* Add a second missed example.
* Correct timeout & improve socket error handling.
I met problem while I was working with the WiFiClientSecure.
I tried to found the source of the problem, and I found it in the sll_client.cpp.
Please check my contribution.
I've open this problem in #4335 but received no response.
I spent quite a while today figuring out how to get an OTA update over HTTPS on a custom port working. A part of my problem was not putting a slash before the .bin filename, since it wasn't there in the example. This produced invalid HTTP requests. Adding the slash would make it clear that it needs to be there.
Given that the URL in line 53 contains the same words "server" and "file.bin", one might assume that in line 55, the slash after the port number would get added automatically, however I have found out that without a slash you get an invalid request. Adding the slash removes any doubt.
I was playing with the mDNS service and noticed the method MDNSResponder::addService could return a Boolean with the way it is implemented just like other functions in this library.
This would be handy to know at a higher level weather or not the service was added correctly to the mDNS server of the ESP32.
This is a fix for missing scan responses after a first successfull scan.
While running the BLE_scan.ino sketch with wantDuplicates=false, i got
only one result with correct advertising and scan response length (31,26):
pBLEScan->setAdvertisedDeviceCallbacks(new MyAdvertisedDeviceCallbacks(), false);
pBLEScan->start(scanTime, false);
...
[W][BLEScan.cpp:109] handleGAPEvent(): bytes length: 31 + 26, addr type: 1
All following calls to start() just returned the advertising data without
scan response data:
pBLEScan->start(scanTime, false);
[W][BLEScan.cpp:109] handleGAPEvent(): bytes length: 31 + 0, addr type: 1
With "wantDuplicates=true" i got:
pBLEScan->setAdvertisedDeviceCallbacks(new MyAdvertisedDeviceCallbacks(), true);
pBLEScan->start(scanTime, false);
[W][BLEScan.cpp:109] handleGAPEvent(): bytes length: 31 + 26, addr type: 1
[W][BLEScan.cpp:109] handleGAPEvent(): bytes length: 31 + 26, addr type: 1
[W][BLEScan.cpp:73] handleGAPEvent(): ESP_GAP_SEARCH_INQ_CMPL_EVT
Devices found: 1
Scan done!
pBLEScan->start(scanTime, false);
[W][BLEScan.cpp:109] handleGAPEvent(): bytes length: 31 + 0, addr type: 1
[W][BLEScan.cpp:109] handleGAPEvent(): bytes length: 0 + 26, addr type: 1
[W][BLEScan.cpp:109] handleGAPEvent(): bytes length: 31 + 0, addr type: 1
[W][BLEScan.cpp:109] handleGAPEvent(): bytes length: 0 + 26, addr type: 1
Explicitly initializing m_scan_params.scan_duplicate of BLEScan solves
this issue (In my case the un-initialized value was
m_scan_params.scan_duplicate == 1073599044).
Co-authored-by: Me No Dev <me-no-dev@users.noreply.github.com>
* If WebServer.handleClient is run in a tight loop, it will starve other processes. So, if there is no connection, throw in a delay(1). Fixes#4348
* Made a variable to control the delay behavior
* Update Parsing.cpp
When uploading TLS cert files the end of file "-----END CERTIFICATE-----" (or any kind of file with the sequence "CRLF--") is taken as posible end boundary. Then it is compared to the start boundary string. As it is expected, comparison turns to be false, and the whole end boundary string is put to _currentUpload->buf through _uploadWriteByte(). Here you have the problem: if you read boundary.length() bytes from HTTP request and you have some of the actual end boundary bytes in it, when you put all those bytes into _currentUpload->buf you are making a mistake. You will miss the actual end boundary string because some of those bytes were put in _currentUpload->buf.
* Update Parsing.cpp
SPIFFS causes crashes if you attempt to rmdir. Since there are no true directories in spiffs, this ought to be a noop. It looks like @me-no-dev worked around this by using unlink instead of rmdir, which works in fatfs and doesn't panic spiffs. This behavior is not universal. In order to get littlefs working, it would be good to get this back to conformity. Rather than digging deep into the upstream spiffs, I just check the mountpoint and noop if it is "/spiffs". So, if the user has changed the mountpoint, this will not work, but I think it's a pretty good tradeoff.
Successful exploitation could lead to arbitrary code execution.
The bug can be reproduced by running the following in a browser:
```
const formData = new FormData();
for (let i = 0;i < 33;++i) { formData.append("foo", i.toString()); }
await fetch("http://esp.local", { method: 'POST', body: formData });
```
Informed by the discussion in the bug and the code in 'that other branch'
the fix was clear. Just set a flag if we start handling a write, and
use that flag to guard the long write complete call.
Background
The current implementation of Update() uses the spi_flash_* api to write and read from flash. These functions ignore the partition->encrypted flag and always write raw data to flash even if the partition is marked as encrypted.
Changes in this PR
Update() now uses the esp_partition_* api.
Wrapper functions for esp_partition_* added to ESP.cpp. This was done to maintain a consistent approach to the way the spi_flash_* functions were used. I note though that not all of the esp-idf functions are used are wrapped, for example esp_ota_get_next_update_partition() so it may be that these should not be added?
The current implementation of Update() changes the first (magic) byte of firmware to 0xFF on write, and then when the firmware is completely written changes it back to ESP_IMAGE_HEADER_MAGIC. This works without erasing the sector because flash bits can be changed from 1->0 (but not 0->1). If the flash is encrypted then the actual data written to flash will not be all ones, so this approach will not work. In addition, encrypted flash must be written in 16 byte blocks. So, instead of changing the first byte the changed code stashes the first 16 bytes, and starts writing at the 17th byte, leaving the first 16 bytes as 0xFF. Then, in _enablePartition() the stashed bytes can be successfully written.
Benefits
Whilst it's not possible to use encrypted flash directly from either the Arduino IDE or PIO it's reasonably straightforward to compile and flash a bootloader with the necessary support from a simple esp-idf project and then use ArduinoOTA for subsequent updates. This PR enables the use of this workflow until such time as encrypted flash is supported, and is a first (small) step toward adding support.
Regardless of the above, the esp_partition_* api is recommended over the api_flash_* api.
Application code should mostly use these esp_partition_* API functions instead of lower level spi_flash_* API functions. Partition table API functions do bounds checking and calculate correct offsets in flash, based on data stored in a partition table.
By default the disconnect is broadcasted to every clients. So if you call disconnect on one connected client, they'll all be disconnected if we don't filter the event by conn_id.
* ESP.getChipModel() returns model of the chip
* ESP.getChipCores() returns the core count.
* Example gives chip model, revision and core count.
* Read efuse for chipmodel
Co-authored-by: Martijn Scheepers <ms@SDNengineering.nl>
* Fix issue #3833, data parsing of Eddystone TLM data frame
Add Beacon scanner example to show usage of BLEEddystoneTLM class and BLEEddystoneURL class
Add EddystoneTLM beacon example
Add EddystoneURL beacon example
* Fix buffer size for .toString()
When connecting using transportTraits, the CA and client certificates are ignored after the initial _client->connect() is called. This is because on connect, WiFiClientSecure will call start_ssl_client with _CA_Cert and other cert options set to null unless setCACert, setCertificate etc. are called before connect. Running _transportTraits->verify after connect therefore does exactly nothing. It's easy to verify that this is the case by passing a CACert to HTTPClient with verbose logging enabled - the logs will say "WARNING: Use certificates for a more secure communication!" which is only present when both PSK and CA are null. This change fixes the issue.
Real DNS resolving timeout used by lwip library is 14[s] (7[s] for DNS1 + 7[s] for DNS2). Function WiFiGenericClass::hostByName() has timeout set to lower value (only 4[s]), so callback function may be called after this low timeout and it may overlappe stack memory used now by other function.
Fixes#3722
They are all broadly similar, all being based on the ESP32-D0WD core, but have
some subtle differences between variants around PSRAM inclusion and camera pin
mappings.
The ESP32CAM "unit" product (SKU: U007) has no PSRAM and maps the
"Pixel Data 0" (Y2 GPIO/OV2640 D2) camera pin to IO17 rather than IO32 as is
the case with most other M5Stack ESP32 camera devices.
The ESP32CAM also requires the "hmirror" and "vflip" camera configuration
bits setting in order to capture a true image.
The relevant M5Stack pin configuration documentation can be found here:
https://docs.m5stack.com/#/en/unit/esp32cam?id=pinmap
