The smallest esp32 module (so far…)

Some months ago, Espressif announced the production of a new chip, named ESP32-PICO-D4.

esp-pico-001

It’s a complete SiP (System in Package), that is a chip which integrates the esp32 microcontroller, a 4Mbit flash memory, a crystal oscillator, filter capacitors and RF matching links. The chip datasheet is available on the official website.

Using this chip, it’s possible to create very small modules. I recently received one of those from Aliexpress:

esp-pico-002

esp-pico-003

To better understand how small it is, let’s compare its size with a “classic” ESP-WROOM-32 module and with a 1 euro coin:

esp-pico-004

The module includes a chip antenna; it’s also possible to connect an external antenna thanks to the presence of an I-PEX connector.

In conclusion, the availability of the ESP32-PICO-D4 SiP makes it possible to use the esp32 chip in applications where the available space is very small…

ESP32, PlatformIO

PlatformIO is an opensource ecosystem (as it’s defined in the homepage of the project) to develop IoT projects.

The heart of the platform is a software component named PlatformIO Core. This component includes:

  • a cross-platform compiler
  • a libraries and dependences manager
  • a serial monitor

PlatformIO Core is developed in Python and therefore it can run on different operating systems (Windows, Linux, MacOS).

Although you can use the Core component directly, PlatformIO’s strength lies in its IDE, which allows the development of multi-platform projects and integrates with the Core itself.

In this article I’m going to show you how to use PlatformIO to develop projects running on the esp32 chip.

Installation

PlatformIO IDE is provided as a plugin for two different development tools: Atom and VisualStudio Code. I tried both solutions and I preferred VSCode: both the installation and the use are simpler and more immediate.

Install VSCode after having downloaded the package from Microsoft’s website (the installer is available for Windows, Linux and MacOS).

Open the Package Manager:

pio-001

search the PlatformIO IDE package, then click on Install:

pio-002

wait until the installation is complete:

pio-003

Hello world

Now it’s time to develop your first program, which traditionally will display the sentence Hello world! on the terminal.

If it doesn’t show up automatically, open the PlatformIO’s homepage:

pio-004

then click on New Project:

pio-005

give a name to the project and choose a devboard based on the esp32 chip (in this example I’ll use a Lolin32 board by Wemos). PlatformIO supports both the esp-idf framework and the arduino-esp32 one. All my tutorials are based on the first one:

pio-006

PlatformIO automatically creates some folders for your project. Choose the src folder (it stands for source, that is the folder which will contain the source code) and create a new file:

pio-007

name the file main.c and type the simple program as it follows:

pio-008

run the compiler by clicking on the corresponding button in the bottom bar:

pio-009

the editor displays an error… indeed your code is using the printf() function without having included the library:

pio-010

add the missing line, now you should be able to complile the code without errors:

pio-011

PlatformIO can also upload the compiled program to your board. Thanks to its auto-detect feature, you usually don’t need to specify the serial port the board is connected to:

pio-012

Serial monitor

PlatformIO also includes a serial monitor you can use to test your program. By default, this monitor connects to the serial port with a speed of 9600 baud. The esp32 chip instead has a default speed of 115200 baud; you have therefore to change the platformio.ini file included in your folder as it follows:

pio-013

Now open the serial monitor; you should see the correct output of your program:

pio-014

Conclusions

I found the use of PlatformIO really immediate: after a few minutes I was able to develop, compile, load and test a program. Try it and leave a comment with your impressions!

 

My new third hand

I recently brought from Banggood a support for having a third hand during soldering operations…

hand-001 hand-002

the support is made by an aluminum base and three arms which end with a crocodile clamp with insulating sheath:

hand-003 hand-004

The assembly is very simple, the arms must be screwed on the base:

hand-005

In general this support is very usefulthe arms are very well articulated and their length allows to “grasp” even medium-sized cards:

hand-007 hand-006

The only defect is the position of the feet, which makes the base not too stable:

hand-008

 

JQ6500, “flashdata error”

After having published my previous posts about the JQ6500 module, some readers wrote me about an error the MusicDownload.exe application shows when they click on the upload button to store the mp3 files in the flash memory.

The error is

flashdata error, have not define program mode in flashdata.ini!!

jq6500-err-001

It seems that the application is not fully compatible with Windows 8 or 10.

To solve the problem, you only have to enable the Windows 7 compatibility mode:

jq6500-err-002

With this setting, the upload process runs fine:

jq6500-err-003

In the following video, you can see all the required steps:

JQ6500 module recovery

In a previous post I presented the JQ6500 audio module. When you connect it to your computer, it’s detected as a CDROM drive.

I brought some modules from different webstores. A couple of them gave me the following error when I tried to browse the content of the CD:

Unable to access the disk. The disk may be damaged.

jq6500-rec-000

It seems that some manufacturers do not program the flash memory of the chip and therefore it doesn’t contain the MusicDownload.exe application.

Luckily, thanks to the work by Reinhard Max and Nikolai Radke, it’s now possible to recover those modules. From Nikolai’s Github repository it’s possible to download an ISO image with a minimal Linux distribution and the recovery toolkit:

jq6500-rec-100

After having downloaded the image, you can burn it to a CD (or use software like UNetbootin to transfer it to a bootable USB drive) and restart your PC with that image.

However, there is an alternative way: the use of VMware Workstation Player, a free virtualization software (for non-commercial use) by VMware. Using this application it’s possible to start a virtual machine on your PC that can run the recovery ISO image.

At the end of this post you can find a video that shows all the following steps…

Let’s see how to do it. After having installed the application, create a new virtual machine:

jq6500-rec-008

Choose I will install the OS later:

jq6500-rec-009

Select Linux as operating system:

jq6500-rec-010

Give the VM a name and select the folder where it will be saved:

jq6500-rec-011

You don’t really need an hard disk, so you can specify a small size for it (for example 1Gb):

jq6500-rec-012

Now you have to configure VMware Player to start the VM using the ISO image. Click on Customize Hardware:

jq6500-rec-013

Select the CD/DVD drive and connect it to the ISO image downloaded from Github:

jq6500-rec-014

Before starting the VM, verify that your module is not connected to the computer:

jq6500-rec-015

Wait for the boot process to complete. The toolkit (started automatically) will warn you that it wasn’t able to identify any modules:

jq6500-rec-001

Open the Removable Devices menu and note down all the listed devices:

jq6500-rec-002

Now connect the module to the USB port. If you open the menu again, you should be able to identify which device corresponds to your module:

jq6500-rec-003

Connect it to the virtual machine:

jq6500-rec-004

Perform a new scan typing the command. If everything is ok, now the toolkit should be able to identify the module:

jq6500-rec-005

Depending on the size of the flash memory of your module, choose the correct menu item. Most modules are sold with a 16Mbit or 2Mbyte memory chip:

jq6500-rec-006

At the end of the recovery process, you can shut down the VM and verify that your computer is now able to display the disk content:

jq6500-rec-007

Video

Here’s a video that shows the recovery process:

First look at JQ6500 modules

For some time, on different chinese webstores (for example Banggood) there is a module called JQ6500 for sale:

jq6500-001 jq6500-002

it’s often described as a voice sound module or as an MP3 player sound module.

Actually JQ6500 is the name of the main chip hosted on the module:

jq6500-003

The chip is manufactured by a Chinese company named JQ. A datasheet for the chip is also available, unfortunately only in Chinese (but Google Translate can help to understand what it contains).

On the other side of the PCB, the module houses two additional integrated circuits:

  • a 16Mbit flash memory (25L1606E)
  • a 3W audio amplifier (HXJ 8002)

When you connect the module to your computer via USB, it is detected as a CDROM drive. If you browse the content of the CD, you can find the MusicDownload.exe application that allows to upload audio files in the flash memory:

jq6500-rec-007

The software is in Chinese but its use is very simple: by moving to the second tab you can select the MP3 files to be uploaded. If you now move back to the initial tab, you can start the upload process clicking on the only available button. In the video at the end of this post you can see how it works…

You can control the JQ6500 chip in different ways. The easiest one is using external buttons connected to pins K1-2-3-4-5:

jq6500-004

When you press a button, the chip plays the corresponding audio file. For example if you press the button connected to pin K1, the chip plays the audio file named 001.mp3.

The onboard amplifier (HXJ 8002) is a mono IC and its output is connected to pin SPK+ and SPK-. You can therefore connect to those pins a small speaker. If you want a stereo audio, you can instead use pins ADL_L (left channel) and ADC_R (right channel) and connect them to an external amplifier.

Conclusions

This module is an excellent and inexpensive solution to add audio to your projects. The use of an internal flash memory has the advantage of not requiring SD cards or other media to store your audio files; in contrast its capacity (16Mbit = 2MByte) makes it more suitable to reproduce sound effects / guide voices than to make a music player.

In the next articles I will show you how to interface the module with Arduino … meanwhile here is a video showing my first tests:

 

 

 

ESP32 (35) – BLE, scan response

In the previous posts I explained how to receive and send advertising packets based on the Bluetooth LE standard.

The payload (that is the amount of “useful” data) of those packets is at most 31 bytes. It isn’t much: if – for example – you want to include the device name, little place remains for other data.

The BLE standard allows peripherals to send additional data using the scan request – scan response process.

When a device receives an advertising packet, it can contact the transmitter by sending a scan request packet to request further information. When receiving a scan request package, the peripheral can respond with a scan response packet:

scan-response-001

Advertising and scan request packets have the same format; it’s therefore possible to transfer, using scan response, additional 31 bytes of data.

esp32

The esp framework offers two modes for configuring the content of a scan response packet: using the esp_ble_adv_data_t struct or creating a byte array (raw mode). These modes are similar to the ones used to configure advertising packets you learned in previous articles (struct and raw mode).

In the first case, you have to declare a second struct, in addition to the one related to the advertising packet, to define the content of the scan response packet:

static uint8_t manufacturer_data[6] = {0xE5,0x02,0x01,0x01,0x01,0x01};
static esp_ble_adv_data_t scan_rsp_data = {
  .set_scan_rsp = true,
  .manufacturer_len = 6,
  .p_manufacturer_data = manufacturer_data,
};

Very important is set to true the set_scan_rsp parameter. It’s indeed this parameter what tells the driver that this struct is related to the scan response packet.

You can then pass the new struct to the driver, with the same function used previously:

esp_ble_gap_config_adv_data(&scan_rsp_data);

The driver will call the callback function twice: one to indicate the successful configuration of the advertising packet and one for the configuration of the scan response one. The two events are different:

case ESP_GAP_BLE_ADV_DATA_SET_COMPLETE_EVT:
  [...]
case ESP_GAP_BLE_SCAN_RSP_DATA_SET_COMPLETE_EVT:
  [...]

You have to wait until both the events have triggered before starting the advertising process. In my example program (you can download the source code from my Github repository) I use two boolean variables:

bool adv_data_set = false;
bool scan_rsp_data_set = false;
[...]
case ESP_GAP_BLE_ADV_DATA_SET_COMPLETE_EVT:
  adv_data_set = true;
  if(scan_rsp_data_set) esp_ble_gap_start_advertising(&ble_adv_params); break;
 
case ESP_GAP_BLE_SCAN_RSP_DATA_SET_COMPLETE_EVT:
  scan_rsp_data_set = true;
  if(adv_data_set) esp_ble_gap_start_advertising(&ble_adv_params); break;

If you want to use the raw mode instead, you have to declare a byte array and fill it with the content of the payload of the packet. Then you can use a specific function of the framework to pass the array to the driver:

static uint8_t scan_rsp_raw_data[8] = {0x07,0xFF,0xE5,0x02,0x01,0x01,0x01,0x01};
[...]
esp_ble_gap_config_scan_rsp_data_raw(scan_rsp_raw_data, 8);

did you notice that the content of the scan response packet is the same in the two examples?

The driver will confirm the configuration of the packet with a dedicated event. Also in this case you have to wait for the end of both configurations (advertising and scan response):

case ESP_GAP_BLE_SCAN_RSP_DATA_RAW_SET_COMPLETE_EVT:
  scan_rsp_data_set = true;
  if(adv_data_set) esp_ble_gap_start_advertising(&ble_adv_params); break;
You can also mix the two modes in your program. For example you can configure the advertising packet using the struct and configure the scan response one using the raw mode.

Now with the nRF Connect app you can verify that your scan response packet is correctly received by your smartphone:

scan-response-002

In the following video I explain how I built the payload of the packet and how the program works:

ESP32 (34) – BLE, raw advertising

In the previous post, you learned how to send BLE advertising packets with the esp32 chip.

To define the content of the packet, you used a struct, of the esp_ble_adv_data_t type:

raw-adv-001

The struct’s definition is included in the esp_gap_ble_api.h file:

raw-adv-002

Although there are many fields available, sometimes it is necessary to be able to define the content of the advertising packet arbitrarily. For this reason, the esp-idf framework provides a raw mode.

Instead of defining a struct, you create a byte array and fill it with the entire contents of the packet’s payload:

static uint8_t adv_raw_data[10] = 
  {0x09,0x09,0x4c,0x75,0x6b,0x45,0x53,0x50,0x33,0x32};

then you can use the esp_ble_gap_config_scan_rsp_data_raw() function to pass the array to the driver. You have to specify both the array and its size as parameters:

esp_ble_gap_config_scan_rsp_data_raw(scan_rsp_raw_data, 8);

When using this new function, it also changes the event that the driver passes to your callback function when the configuration is complete. The new event is ESP_GAP_BLE_ADV_DATA_RAW_SET_COMPLETE_EVT. As in the previous example, when this event is triggered you can start the advertising process:

case ESP_GAP_BLE_ADV_DATA_RAW_SET_COMPLETE_EVT: 
  esp_ble_gap_start_advertising(&ble_adv_params);
  break;

Raw data

For the advertising process to work, the data contained in the array must correspond to a valid payload.

In the blog post about the iBeacons, I’ve already shown you its structure. Let’s briefly review it:

ibeacon-002

The payload contains one or more AD (advertising data) structures. Each structure is made by 3 fields:

  • an initial byte that represents the length (in bytes) of the structure, excluding itself
  • a byte that represents the type of the data contained in the structure
  • a variable number of bytes which are the actual data

The codes that can be used to define the type of data can be found in the Bluetooth specifications. Depending on the type of data, it is then necessary to apply a particular format to the data that follows. The necessary information is found in the Core Specification Supplement document (available on the Bluetooth.com website).

Let’s see a simple example: the ADType 0x09 represents the complete local name, which is the name of the device. This name must be specified in AD data with simply a sequence of the ASCII codes that correspond to the different letters.

You can use a website to do the conversion:

raw-adv-003

The payload to transmit this name is therefore:

adv_raw_data[7] = {0x06,0x09,0x4d,0x79,0x42,0x4c,0x45};

The first byte has value 0x06 that is the sum of the name length (5 bytes) and 1 byte for the data type (0x09).

Demo

In the following video you can see how I use the raw advertising feature to simulate the advertising packet of my iBeacon and therefore I’m able to activate the relay as in the previous example.

The source code of the program is available in my Github repository.

ESP32 (33) – BLE, advertising

In the previous posts you learned how to use the esp32 chip to receive and parse the advertising packets transmitted by BLE peripherals. As a practical example, I developed a program to detect the presence of a particular iBeacon and activate an output accordingly.

In today’s tutorial, you’ll learn how to transmit advertising packets instead.

Advertising process

You’ve already discovered that the Bluetooth driver included in the esp-idf stack is executed in a dedicated thread. Whenever the driver needs to send a notification to your program, it calls a callback function indicating which event has triggered.

The advertising process is very simple:

  • the program configures the data to be transmitted with esp_ble_gap_config_adv_data()
  • the driver reports that it has finished the configuration with the event ESP_GAP_BLE_ADV_DATA_SET_COMPLETE_EVT
  • the program can now start the advertising process with esp_ble_gap_start_advertising()
  • the driver reports that the process has started with the event ESP_GAP_BLE_ADV_START_COMPLETE_EVT

esp32-adv-001

Advertising DATA

It’s possible to tell the driver which data to include in the advertising packet with the following command:

esp_err_t esp_ble_gap_config_adv_data(esp_ble_adv_data_t *adv_data);

The command accepts as parameter a pointer to an esp_ble_adv_data_t struct:

esp32-adv-002

The meaning of the different fields is explained in the Supplement to the Bluetooth Core Specification document.

First let’s find out how to transmit the device name. You have to use the esp_ble_gap_set_device_name() function to pass the name to the driver and set the field include_name to true in the struct:

static esp_ble_adv_data_t adv_data = {
  .include_name = true,
};
[...]
ESP_ERROR_CHECK(esp_ble_gap_set_device_name("ESP32_BLE"));
ESP_ERROR_CHECK(esp_ble_gap_config_adv_data(&adv_data));

Using the flags, you can publish some features of your device. The available constants are:

esp32-adv-003

you can combine them with the OR operator. If, for example, you want to tell the world that your device is limited discoverable (i.e. it sends the advertising packets only for a limited time, usually 30 seconds) and that it doesn’t support classic Bluetooth (BR/EDR, Basic Rate/Enhanced Data Rate) you’ll write:

static esp_ble_adv_data_t adv_data = {
  .flag = ESP_BLE_ADV_FLAG_LIMIT_DISC | ESP_BLE_ADV_FLAG_BREDR_NOT_SPT,
};

Advertising PARAMETERS

After configuring the content of the advertising packet, you have also to tell the driver how to send the packet.

The command:

esp_err_t esp_ble_gap_start_advertising(esp_ble_adv_params_t *adv_params);

accepts as parameter an esp_ble_adv_params_t struct:

esp32-adv-004

You can configure the minimum and maximum transmission interval of the packet. The two parameters can assume a value from 0x20 to 0x4000. To calculate the interval in milliseconds, the value specified must be multiplied for 0.625. This means that the minimum value (0x20) corresponds to an interval of 12.5ms.

The esp_gap_ble_api.h file lists the constants that can be used for the other parameters (esp_ble_adv_type_t, esp_ble_addr_type_t …).

For example, let’s configure the advertising process as it follows:

  • minimum transmission interval: 0x20, maximum: 0x40
  • non connectable device (it doesn’t accept incoming connections and only sends data in broadcast)
  • public MAC address
  • transmission on all the 3 channels dedicated to advertising packets
  • no filter on devices who can perform a scan or connect
static esp_ble_adv_params_t ble_adv_params = {
  .adv_int_min = 0x20,
  .adv_int_max = 0x40,
  .adv_type = ADV_TYPE_NONCONN_IND,
  .own_addr_type  = BLE_ADDR_TYPE_PUBLIC,
  .channel_map = ADV_CHNL_ALL,
  .adv_filter_policy  = ADV_FILTER_ALLOW_SCAN_ANY_CON_ANY,
};
[...]
esp_ble_gap_start_advertising(&ble_adv_params);

Demo

I prepared a program that includes what explained above. The source code is available in my Github repository.

Here’s how it works:

Arduino bootloader and ISP

After having developed a sketch using the Arduino IDE, you can compile and load it on the Arduino board connected to your PC with just a click on the upload button:

isp-002

The program is stored in the flash memory of the microcontroller (on Arduino Uno boards this is the ATmega328p).

You can upload your program on the microcontroller without the need of a dedicated programmer because of the microcontroller itself runs a small program named bootloader.

The bootloader is indeed a program, already flashed in every microcontroller of the Arduino boards, that is executed everytime the microcontroller is reset. The first operation the bootloader does is to check if on the serial/USB connection there’s a request to flash a new program. If so, the bootloader talks with the Arduino IDE, receives the new program and stores it in the flash memory:

isp-001

The bootloader used as of now on the Arduino Uno boards is named optiboot.

If you buy an ATmega328p chip and you want to upload your sketches on it using the Arduino IDE you have first to flash the bootloader in the chip. You usually need a dedicated programmer to program the chip. ATmega microcontrollers support a programming method named In System Programming (ISP), designed to program the chips directly on the board where theyt are located.

This method requires to connect 6 pins of the chip to the programmer:

  • 5V and ground
  • reset
  • MISO, MOSI and SCK

For example every Arduino Uno has a dedicated connector which allows to re-program the ATmega328p chip without removing it from the board:

isp-003

On the Internet you can find several ATmega programmers (for example this by Adafruit); very interesting is the possibility to use an Arduino as a programmer, thanks to the work of Randall Bohn.

First you have to upload on your Arduino the ArduinoISP sketch which is included in the Examples shipped with the IDE:

isp-004

then you have to create – for example on a breadboard – a minimal circuit with the chip to be programmed, a 16MHz crystal and two 22pF capacitors:

isp-013

using some jumpers, connect your Arduino Uno to this circuit as it follows:

  • 5V -> pin 7 and 20
  • Ground -> pin 8, 22 and the two capacitors
  • pin 10 of Arduino or RESET of the ICSP connector -> pin 1
  • pin 11 of Arduino or MOSI of the ICSP connector -> pin 17
  • pin 12 of Arduino or MISO of the ICSP connector -> pin 18
  • pin 13 of Arduino or CLK of the ICSP connector -> pin 19

isp-014 isp-015

Configure the IDE to use Arduino as programmer:

isp-005

then burn the bootloader:

isp-006

after few seconds, the IDE should confirm that the bootloader was programmed on the chip:

isp-011

Shield

On some webstores I found an Arduino shield which makes it simple to program ATmega chips. It’s named AVR ISP Shield and manufactured by a company called “OPEN-SMART”:

isp-012

This shield has a ZIF (Zero Insertion Force) socket where the chip goes, some pins to connect it to external boards and a buzzer that is used to confirm the burn process with two beeps.

On the Internet I found an archive with the documentation about this shield and the “official” sketch. You can also use the sketch shipped with the IDE, it won’t only activate the buzzer when the burning process ends successfully.

If you need to program several ATmega328p chips (for example if you’re building your Arduino-compatible boards) the use of this shield makes the programming process much easier and faster!