ESP32, the new Wemos Lolin32 LITE

On my blog I’ve already posted about the development board Wemos designed for the esp32 chip and that, on many webstores, you can find boards that “copy” the Lolin32 name but are instead clones of other opensource projects.

Few weeks ago Wemos published on its official site and make available on its store on Aliexpress a new revision of the board, now named Lolin32 LITE. Thanks to my friend Mauro Alfieri I was able to test a copy:

lolin-lite-001 lolin-lite-002

The main features of the board are the same as the old one:

  • esp32 chip
  • PH-2 (2mm) connector for a LiPo 1S battery
  • charging circuit via USB (max 500mA)

At a visual comparison it is immediately noticeable that the LITE is slightly smaller than the previous, being 5mm long compared to the 5.8mm of the Lolin32:

lolin-lite-003

The smaller length forced Wemos to reduce the number of available pins (19 instead of 26) and to abandon the use of the ESP-WROOM-32 module soldering the esp32 chip and flash memory directly onto the PCB, which also accomodates the antenna.

Speaking of components, for the LITE version Wemos decided to change the chip used to perform the USB-serial conversion and adopted the CH340G IC (which I also used in this project). Finally, the card in my possession uses the latest revision of the esp32 chip (labeled 302017):

lolin-lite-004

 

esp32, chip revisions

In the same way that software programs have bugs, even hardware chips may have problems due to errors in their design. Producers resolve these errors by producing a new chip revision and keeping updated a document with the known bugs for the different revisions and the possible workarounds.

Espressif manufactured two different revisions of the esp32 chip to date:

lolin-lite-005

If you want to know the revision of a chip, you can use the espefuse.py utility included in the esptool package:

lolin-lite-008

specify the serial port (-p) your board is connected to and the summary command.

Observing the values of CHIP_VERSION and CHIP_PACKAGE you can determine the revision of the chip, for example my Lolin32 board has a chip with revision 0:

lolin-lite-006

while the new Lolin32 LITE a chip with revision 1:

lolin-lite-007

ESP32 (27) – GPS

In today’s post I’ll show you how to interface the esp32 chip to a GPS receiver to receive the actual position, speed and more…

GPS receiver

On the Internet and in electronic stores you can find different GPS receivers… thanks to the spread of navigators, smartphones and multicopters you can now buy one for few euros. For this tutorial, I used a GPS receiver sold by Banggood and based on the u-blox NEO-M8 chip:

gps-010 gps-011

Almost all the GPS receivers offer a serial interface, you can therefore connect them to an esp32 chip using one of its UART controllers, as I explained in a previous article. Sometimes it’s not easy to identify the various pins and it may be necessary – as in my case – to open the plastic case that contains the chip to read the PCB silk screen:

gps-001 gps-002

From the photos above you can understand the pinout of my receiver:

  • GND (ground) -> black cable
  • VCC (power supply) -> red cable
  • TXD (transmit) -> green  cable
  • RXD (receive) -> yellow cable

I decided to use the UART1 controller of my esp32 chip, with pins 4 (TX) and 16 (RX). I therefore connected all the cables, paying attention to connect the TX pin of the receiver to the RX pin of the chip and viceversa:

gps-003

NMEA

GPS receivers send data following the NMEA 0183 standard (often called just NMEA).

NMEA is based on strings (sentences) composed by a maximum of 80 characters with an ending CRLF. Each string has the following format:

$PREFIX,data1,data2 ... dataN-1,datoN*CHECKSUM

The prefix (5 chars) defines the type of device (for GPS receiver is GP) and the type of sentence (the following 3 chars). Each sentence includes a checksum (XOR) that allows the receiver to validate the data received.

The NMEA standard defines many types of sentences and each manufacturer can add proprietary types to communicate specific data. A complete list of NMEA sentences is available at this website. Let’s analyze one GGA (Global Positioning System Fix Data) sentence:

$GPGGA,183619,3877.038,N,07702.110,W,1,08,0.9,545.4,M,46.9,M,,*47

The receiver is sending the actual position (latitude 38°77.038′ NORTH and longitude 77°02.110′ WEST), obtained at 18:36:19 thanks to 8 satellites. The fix (position) quality is GPS Fix (1).

To be able to get data from the GPS receiver your program must therefore “understand” the NMEA sentences it receives. I found a very good library, minmea, developed in C and able to parse the most important NMEA sentences.

You only need to copy the two source files (minmea.c e minmea.h) in a subfolder of the components folder of your project and to create a component.mk file as it follows:

gps-001

I had to add the highlighted flag because of the esp framework does not include the timegm() function and therefore, as explained also in the README of the library, you need to tell the compiler to use the more common mktime() instead.

Program

In my Github repository you can find the program I developed for this tutorial. Let’s analyse the most important pieces of code:

To use the minmea library, include its header file:

#include "minmea.h"

Read a line from the UART1 controller and pass it to the library to be parsed:

char *line = read_line(UART_NUM_1);
switch (minmea_sentence_id(line, false)) {
  case MINMEA_SENTENCE_RMC:
  [...]
  case MINMEA_SENTENCE_GGA:
  [...]
}

Finally verify if the new parameters are different from the ones already saved and, if so, print them on the screen:

float new_latitude = minmea_tocoord(&frame.latitude);
if((new_latitude != NAN) && (abs(new_latitude - latitude) > 0.001)) {
  latitude = new_latitude;
  printf("New latitude: %f\n", latitude);
}

I added a threshold value (0.001) to avoid printing a lot of small updates, due to disturbances or oscillations on received data.

Here’s the program running on my laptop:

gps-010

ESP32 (26) – UART

UART (Universal Asynchronous Receiver-Transmitter) is an hardware peripheral which allows serial, asynchronous communication with configurable data format and speed. The UART interface usually works at logic level: the electric signals are generated by an external circuit, following the standards of the communication bus you chose.

For example the classical “serial port” of many personal computers is based on the EIA RS-232 standard, which defines how – at the physical layer – signals are generated on the communication medium. There are dedicated chips (the most famous of which is surely the MAX232 by Maxim Integrated) to convert the logic levels of a UART peripheral to the physical signals of the EIA RS232 standard:

uart-001

The esp32 chip offers 3 UART controllers. These controllers are connected to the GPIO matrix; this allows to assign them most of the digital pins of the chip:

uart-002

The esp-idf framework includes a driver (uart.c) to simplify the configuration and the use of the controllers; to use it, include its header file in your program:

#include "driver/uart.h"

For the driver, the 3 controllers are named as follows:

uart-003

In this first post, I’ll explain the basic use of a controller; events and interrupts will be covered in a future article.

The first thing to do is to configure the controller using the uart_config_t struct:

uart_config_t uart_config = {
  .baud_rate = 115200,
  .data_bits = UART_DATA_8_BITS,
  .parity = UART_PARITY_DISABLE,
  .stop_bits = UART_STOP_BITS_1,
  .flow_ctrl = UART_HW_FLOWCTRL_DISABLE
};
  • baud_rate is the transmission speed
  • data_bits, and stop_bits are the number of bits for each “word” and the number of stop bits
  • parity defines if the controller must send the parity bit or not
  • flow_ctrl is the type of flow control (hardwaresoftware or disabled)

There are two additional parameters (rx_flow_ctrl_thresh and use_ref_tick) you can use to set the threshold for the RTS signal if hardware flow control is selected and to enable the REF_TICK signal as clock for the UART controller.

The constant values for data_bitsstop_bits… are declared in the uart.h file.

Often the parameters required to communicate with a serial device are expressed in a “condensed” format, for example if you read 9600,8N1 it means:

  • speed 9600 baud
  • “word” of 8 bits
  • No parity
  • 1 stop bit

Configure the controller with the method:

uart_param_config(uart_port_t uart_num, const uart_config_t *uart_config);

passing as parameters the number of the controller (uart_num) and the struct with the configuration previously defined (uart_config).

Conclude the configuration setting the pins the controller have to use for the different signals:

uart_set_pin(uart_port_t uart_num, int tx_io_num, int rx_io_num, 
  int rts_io_num, int cts_io_num);

You can use the UART_PIN_NO_CHANGE constant if that specific signal is not used or if you want to keep the default pin.

For example to map the controller to pins 4 and 5 without using the RTS and CTS signals:

uart_set_pin(UART_NUM_0, 4, 5, UART_PIN_NO_CHANGE, UART_PIN_NO_CHANGE);

Now you can install the driver with:

uart_driver_install(uart_port_t uart_num, int rx_buffer_size, int tx_buffer_size, 
  int queue_size, QueueHandle_t* uart_queue, int intr_alloc_flags);

In addition to the controller number, you have to pass the size for the receive and transmit buffers. The parameters about queue and interrupts will be discussed in a future article.

The two buffers must have a size bigger than the corresponding hardware buffers (128). Only for the transmit buffer you can specify size = 0; in this case, the driver will be blocking, that is the execution of the task will be halted until the transmission ends.

Let’s now learn how to send data. To better understand the differences between the available commands, you have to understand that there are two buffers: one hardware, included in the UART controller, and one software, implemented in the driver:

uart-004

The first command to send data – to be used only if the software trasmit buffer is disabled – is uart_tx_chars():

int uart_tx_chars(uart_port_t uart_num, const char* buffer, uint32_t len);

This command sends len bytes from buffer. As you’re not using a software buffer, it may happens that the command cannot send all the bytes because the hardware buffer is full; the uart_tx_chars method therefore returns the number of bytes actually sent.

To use the software buffer, the uart_write_bytes() is available:

int uart_write_bytes(uart_port_t uart_num, const char* src, size_t size);

This command copies size bytes from the src array to the driver’s buffer: the driver will take care of filling the hardware buffer of the controller until all the data is transmitted. The uart_write_bytes() also retuns the number of bytes actually copied in the tx buffer.

When receiving, you can use the uart_read_bytes() command:

int uart_read_bytes(uart_port_t uart_num, uint8_t* buf, uint32_t length, 
  TickType_t ticks_to_wait);

The command reads a maximum of length bytes from the receive buffer and copies them in the buf array. The command waits for data the number of specified ticks, then returns the number of bytes actually read.

You can know the number of bytes at each moment available in the receive buffer with:

uart_get_buffered_data_len(uart_port_t uart_num, size_t* size);

UART and stdio

The framework allows to use a UART controller as a peripheral for standard I/O. Unix streams stdin, stdout and stderr are indeed linked to RX and TX operations on that controller: you can therefore use standard C functions like printf()scanf()… to write and read from the UART controller.

Via menuconfig you can specify which controller to use, the parameters for the controller and you can also disable this feature at all:

uart-005

Demo

In the following video I’ll show how to work with UART controllers connecting one of them to a USB->serial converter. Both the converter and the esp32 devboard are connected to my laptop: in this way I can send data from the laptop to UART1 via the converter, read it and send it back to my laptop using the UART0 controller.

ESP32 (25) – Oled display with U8G2

If you read my post ESP32, Wemos or not to Wemos you know that I brought a development board, that happened to be a D-duino-32 clone, with an ESP-WROOM-32 module and a 0.96″ oled display.

This display, available also as a standalone module on several websites (for example on Banggood), has the following features:

  • size: 0.96 inches
  • resolution: 128×64 pixels
  • controller: SSD1306 with I2C interface
  • power supply: 3.3V – 6V

u8g2-00a u8g2-00b

It’s very easy to interface it with the esp32 chip thanks to the work of olikraus and Neil Kolban. The first wrote the u8g2 library, while the second implemented the specific functions of the u8g2’s hardware abstraction layer (HAL) for the esp32.

u8g2, installation

u8g2 is a fantastic library for monochrome displays: it supports several types of displays and controllers, it’s easy to port to new platforms and offers many methods to draw geometric figures, display images and write strings using different fonts.

Let’s learn how to use it in your project. First download the archive with the content of the library’s Github repository:

u8g2-01

If it doesn’t exist, create the components folder in the main folder of your project. Unzip the archive into that folder and rename the subfolder u8g2-master in u8g2:

u8g2-02

In the u8g2 folder create a file named component.mk with the following content:

u8g2-03

Now download the files u8g2_esp32_hal.c and u8g2_esp32_hal.h from nkolban’s repository:

u8g2-04

You can copy the two files in the main folder of your project:

u8g2-05

u8g2, configuration

To use the u8g2 library in your program, first include the header file:

#include "u8g2_esp32_hal.h"

The HAL configuration is stored in the u8g2_esp32_hal_t struct:

typedef struct {
  gpio_num_t clk;
  gpio_num_t mosi;
  gpio_num_t sda;
  gpio_num_t scl;
  gpio_num_t cs;
  gpio_num_t reset;
  gpio_num_t dc;
} u8g2_esp32_hal_t;

The library supports both I2C and SPI displays: this is the reason why you see in the struct fields related to the two buses.

You can define and initialize the struct with default values with:

u8g2_esp32_hal_t u8g2_esp32_hal = U8G2_ESP32_HAL_DEFAULT;

The oled display of the D-duino-32 board has an I2C interface connected to esp32’s pin 4 (SCL) and 5 (SDA):

u8g2_esp32_hal.sda = 5;
u8g2_esp32_hal.scl = 4;

When the configuration of the struct is complete, you can use the u8g2_esp32_hal_init() method:

u8g2_esp32_hal_init(u8g2_esp32_hal);

Now move to the configuration of the u8g2 library. Define an u8g2_t variable:

u8g2_t u8g2;

Based on the display you’re using, you have to choose the correct setup function. The parameters for the function are:

  • the pointer to the u8g2_t variable you defined before
  • a constant value that specifies the display rotation
  • two HAL functions to send data on the bus and to delay

The constants for the display roation are:

u8g2-06

and the two HAL functions developed by Kolban are:

  • u8g2_esp32_msg_i2c_cb
  • u8g2_esp32_msg_i2c_and_delay_cb

Here I’m using the setup function for the ssd1306 “noname” controller with the _f prefix that means full framebuffer:

u8g2_Setup_ssd1306_128x64_noname_f(
  &u8g2, U8G2_R0,
  u8g2_esp32_msg_i2c_cb,
  u8g2_esp32_msg_i2c_and_delay_cb);

framebuffer

full framebuffer means that all the data to send to the display are stored in the RAM memory of the microcontroller. This makes easier and faster to work with the display but consumes more RAM. The u8g2 library also supports a page buffer mode to save some memory. A comparison between the different modes, with pros and cons, is available on the library’s wiki.

Finally – if you’re using an I2C display – you have to specify its address (here 0x78):

u8x8_SetI2CAddress(&u8g2.u8x8,0x78);

u8g2, use

After the setup, initialize the display with:

u8g2_InitDisplay(&u8g2);

The display is now in power save mode, to “turn it on” you have to disable this mode with:

u8g2_SetPowerSave(&u8g2, 0);

Now you can call the different methods available to display text, images, shapes. If you’re using the full framebuffer mode, first prepare the buffer in memory and then send it to the display.

Prepare an empty buffer with ClearBuffer():

u8g2_ClearBuffer(&u8g2);

Now for example use the SetFont() and DrawStr() methods to write a string into the buffer with a specific font:

u8g2_SetFont(&u8g2, u8g2_font_timR14_tf);
u8g2_DrawStr(&u8g2, 2,17,"Hello World!");

and finally display the buffer’s content on the display with:

u8g2_SendBuffer(&u8g2);

u8g2-00c

Demo

In the following video I quickly explain how to install the library, how to prepare an image to be displayed and in the end you can see the execution, on my D-duino-32, of the example program available on Github.

ESP32 (24) – I2C a pratical example with HTU21D sensor

In my previous tutorial I explained how to use the I2C driver included in the esp-idf framework to communicate with I2C devices. Today I’ll show you a pratical example: the use of a temperature/humidity sensor.

The sensor

For this tutorial I chose the HTU21D sensor by Te Connectivity. This sensor offers a good level of accuracy and it’s available, already soldered on a breakboard, for few euros:

htu21d-00a htu21d-00b

htu21d-00c

Sparkfun was the first manufacturer to sell a board based on this sensor… even if now this board is retired, other websites (for example Banggood) sell similar products.

To connect the sensor to your esp32 development board, first you have to choose which pins will be used for SDA and SCL signals. Besides these, you have also to connect VDD (3.3V) and GND:

htu21d-00d htu21d-00e

htu21d-00f htu21d-00g

Datasheet and commands

After having completed the physical connection between the devboard and the sensor, you have to understand how to write a program that communicates with it. The first thing to do is certainly to read the datasheet of the sensor (here in PDF format). Do not be scared if you realize that it contains 21 pages of technical details, in the following paragraphs I’ll explain what are the information you absolutely have to know!

On page 10, it starts the chapter about the communication protocol (COMMUNICATION PROTOCOL WITH HTU21D(F) SENSOR). Immediately you read that the sensor offers an I2C slave interface with address 0x40; just below you can also find the table with the supported commands:

htu21d-001

The sensor offers two different modes when measuring temperature and humidity:

  • hold master
  • no hold master

In the first mode, the sensor blocks the clock signal (SCK) during measurement process: this means that the master can send the read command only when the measure is completed. In the second mode instead the master can operate on the bus (for example send commands to other devices) during the measurement process.

In no hold mode the master, after having sent the “trigger maesurement” command, must wait for the sensor to complete the measurement before reading the value. You can verify if the measuring process is complete sending to the sensor a read command and wait for the ACK: if you receive it, it means that the value is ready.

The sensor returns a raw value of 16bits (= 2 bytes). In addition to the value, it also returns a checksum byte; this byte allows the master to verify that no errors occurred when data was transmitted on the bus.

In the previous post you’ve already learned how to query a slave device; therefore the instructions to read the temperature value from a HTU21D sensor in no hold mode are:

// constants
#define HTU21D_ADDR			0x40
#define TRIGGER_TEMP_MEASURE_NOHOLD  	0xF3
 
// send the command
i2c_cmd_handle_t cmd = i2c_cmd_link_create();
i2c_master_start(cmd);
i2c_master_write_byte(cmd, (HTU21D_ADDR << 1) | I2C_MASTER_WRITE, true);
i2c_master_write_byte(cmd, TRIGGER_TEMP_MEASURE_NOHOLD, true);
i2c_master_stop(cmd);
ret = i2c_master_cmd_begin(_port, cmd, 1000 / portTICK_RATE_MS);
i2c_cmd_link_delete(cmd);
 
// wait for the sensor (50ms)
vTaskDelay(50 / portTICK_RATE_MS);
 
// receive the answer
uint8_t msb, lsb, crc;
cmd = i2c_cmd_link_create();
i2c_master_start(cmd);
i2c_master_write_byte(cmd, (HTU21D_ADDR << 1) | I2C_MASTER_READ, true);
i2c_master_read_byte(cmd, &msb, 0x00);
i2c_master_read_byte(cmd, &lsb, 0x00);
i2c_master_read_byte(cmd, &crc, 0x01);
i2c_master_stop(cmd);
ret = i2c_master_cmd_begin(_port, cmd, 1000 / portTICK_RATE_MS);
i2c_cmd_link_delete(cmd);

The code above waits 50ms before fetching the data; from the datasheet you can indeed read that the maximum measuring time is exactly 50ms for the temperature value at the maximum resolution:

htu21d-002

To convert the raw value into the “real” temperature (in °C) or humidity (in %) you can use the formulas in the datasheet:

uint16_t raw_value = ((uint16_t) msb << 8) | (uint16_t) lsb;
float temperature = (raw_value * 175.72 / 65536.0) - 46.85;
float humidity = (raw_value * 125.0 / 65536.0) - 6.0;

Resolution

The sensor offers 4 different resolution combinations for temperature and humidity:

  • humidity 12bit, temperature 14bit
  • humidity 8bit, temperature 12bit
  • humidity 10bit, temperature 13bit
  • humidity 11bit, temperature 11bit

You can select the resolution changing the value of a configuration register with the command write user register. Specifically, the registry bits to be modified are bit 0 and bit 7:

htu21d-003

CRC

The algorithm for verifying the CRC value is explained in the datasheet.

The polynomial used is x^8 + x^5 + x^4 + 1, in binary:

htu21d-004

First add to the polynomial as many zeroes as the bit length of the CRC:

htu21d-005

The resulting value, in hexadecimal, is 0x98800.

Add, at the right of the value you want to verify, the CRC value received from the sensor as third byte:

uint32_t row = (uint32_t)value << 8;
row |= crc;

Then check the input bit above the leftmost divisor bit: if the bit is 1, the input value is XORed to the divisor. Finally, the divisor is shifted one bit right:

for (int i = 0 ; i < 16 ; i++) {
 if (row & (uint32_t)1 << (23 - i)) row ^= divisor;  divisor >>= 1;
}

If, at the end of the process, the row value equals zero, the CRC is verified.

Component

I developed a component for the esp-idf framework that implements what explained in this post. The component is available on Github and the documentation about its use is published in a dedicated page.

Here’s a video about it:

ESP32 (23) – I2C basic

In today’s tutorial I’ll explain you how to interface the esp32 chip to external devices (sensors, displays…) using a very widespread bus: the I2C bus.

I2C

I2C (pronounced i-squared-c) is a serial communication bus – invented by Philips in 1982 – that allows two or more devices to communicate. The devices connected to the bus can act as masters (devices that “control” the bus) or slaves. Usually, a bus has only one master and one or more slaves, but the standard allows also more complex topologies. Each slave device must have an unique address.

Two different transmission speeds are available: standard (100Kbit/s) and fast (400Kbit/s).

The I2C bus requires only two communication lines that connect the devices:

  • SDA, Serial DAta – where data transit
  • SCLSerial CLock – where the master generates the clock signal

The two lines must be connected to a reference voltage (Vdd) via pull-up resistors:

i2c-01

If you want to learn more about the I2C bus, I strongly suggest the website www.i2c-bus.org.

esp32

The esp32 chip offers two I2C controllers, that can act both as master and slave and communicate in standard and fast speed.

The I2C controllers are internally connected to the IO_MUX matrix, so – as I explained in a previous post – you can assign different pins (with some exceptions) to them in your program.

the esp-idf framework includes a driver which makes it easy to work with those controllers, without worring about how the different registers have to be configured. To use the driver in your program, you only need to include its header file:

#include "driver/i2c.h"

First you have to configure the controller (port) you want to use. The configuration is performed with the i2c_param_config() method, which accepts as parameter an i2c_config_t struct that contains the settings for the controller:

esp_err_t i2c_param_config(i2c_port_t i2c_num, const i2c_config_t* i2c_conf);

The two available ports are listed in an enum variable in the i2c.h file:

i2c-02

The i2c_config_t struct is also defined in the same header file:

typedef struct{
  i2c_mode_t mode
  gpio_num_t sda_io_num;
  gpio_pullup_t sda_pullup_en;
  gpio_num_t scl_io_num;
  gpio_pullup_t scl_pullup_en;
  union {
    struct {
      uint32_t clk_speed;
    } master;
    struct {
      uint8_t addr_10bit_en;
      uint16_t slave_addr;
    } slave;
  };
} i2c_config_t;

Here’s the meaning of the different parameters:

  • mode is the working mode of the controller (it can be I2C_MODE_SLAVE or I2C_MODE_MASTER)
  • sda_io_num and scl_io_num configure the pins connected to SDA and SCL signals
  • sda_pullup_en and scl_pullup_en allow to enable or disable the internal pull-up resistors (possible values: GPIO_PULLUP_DISABLE or GPIO_PULLUP_ENABLE)
  • master.clk_speed is the speed in hertz of the clock if in master mode (100000 for standard and 400000 for fast speed)
  • slave.slave_addr is the device address if in slave mode
  • slave.addr_10bit_en tells the controller to use an extended (10 instead of 7bits) address (if value is 0, the 10bit address mode is disabled)

pin choice

The esp32 chip allows, using the IO_MUX matrix, to assign “almost” all the pins to the two I2C controllers. The I2C driver is able to verify if the pins specified in the configuration can be used; if not, it stops the program with an error.

For example, if you want to configure the first I2C controller in master mode, with standard speed and use pins 18 and 19 without enabling the internal pull-up resistors, this is your code:

i2c_config_t conf;
conf.mode = I2C_MODE_MASTER;
conf.sda_io_num = 18;
conf.scl_io_num = 19;
conf.sda_pullup_en = GPIO_PULLUP_DISABLE;
conf.scl_pullup_en = GPIO_PULLUP_DISABLE;
conf.master.clk_speed = 100000;
i2c_param_config(I2C_NUM_0, &conf);

Once configured the controller, you can install the driver with i2c_driver_install():

esp_err_t i2c_driver_install(i2c_port_t i2c_num, i2c_mode_t mode, 
 size_t slv_rx_buf_len, size_t slv_tx_buf_len, int intr_alloc_flags)

Besides the controller number and its mode, you have to specify the size for the transmitting and receiving buffers (only in slave mode) and additional flags used to allocate the interrupt (usually this parameter is 0).

For the controller configured in the example above, the driver is installed as it follows:

i2c_driver_install(I2C_NUM_0, I2C_MODE_MASTER, 0, 0, 0)

Master

Let’s now learn how to use the controller in master mode, to send commands and receive data from a slave.

First, you have to create a command link, that is a “logical” element that will contain the list of actions to be performed to interact with the slave device. The command link is created by the i2c_cmd_link_create() method which returns a pointer to the command link handler:

i2c_cmd_handle_t cmd = i2c_cmd_link_create();

You can now use some methods to add actions to the command link:

  • i2c_master_start and i2c_master_stop
  • i2c_master_write and i2c_master_write_byte
  • i2c_master_read and i2c_master_read_byte

To understand their meaning, first you have to learn how a master device communicate with slaves. First, the master sends on the bus the START signal, followed by the address (7bits) of the slave device and a bit that specifies the requested operation (0 for WRITE, 1 for READ). After every byte sent by the master (including the one that contains the address and the operation bit) the slave answers with an ACK bit:

i2c-04

In your program this translates into:

i2c_cmd_handle_t cmd = i2c_cmd_link_create();
i2c_master_start(cmd);
i2c_master_write_byte(cmd, (slave_addr << 1) | I2C_MASTER_WRITE, true);

The i2c_master_start() method adds to the cmd handler the action to send the START signal, while the i2c_master_write_byte() method sends one byte on the bus. This byte is composed of 7bits for the slave address (slave_addr) left-shifted to 1 bit (<< 1) and of the bit 0 (= IC2_MASTER_WRITE). If you want to perform a READ, you can instead use the I2C_MASTER_READ constant.

The last parameter, set to true, tells the master to wait for the slave to send the ACK bit.

If the requested operation is write, now the master can send n bytes to the slave. At the end, the master sends the STOP signal:

i2c_master_write(cmd, data_array, data_size, true);
i2c_master_stop(cmd);

I used the i2c_master_write method that allows to send an array (uint8_t*) of data. The data_size parameter is the size of the array. Alternatively, I could have used more times the i2c_master_write_byte used previously.

To “execute” the command link, you have to call the i2c_master_cmd_begin() method:

i2c_master_cmd_begin(I2C_NUM_0, cmd, 1000 / portTICK_RATE_MS);

its parameters are the number of the I2C controller, the command link handler and the maximum number of ticks it can wait (this method is indeed blocking; in the example below it waits for a maximum of 1 second).

Finally you can free the resources used by the command link with i2c_cmd_link_delete(cmd).

The read operation is slightly more complex. First you have to send to the slave device the command for the data you want to read. In the following tutorial I’ll show you a real application, for now let’s assume that the slave device is a temperature sensor with address 0x40 and that the command measure the actual temperature corresponds to byte 0xF3.

You’ve already learned how to send the command:

i2c_cmd_handle_t cmd = i2c_cmd_link_create();
i2c_master_start(cmd);
i2c_master_write_byte(cmd, (0x40 << 1) | I2C_MASTER_WRITE, true);
i2c_master_write_byte(cmd, (0xF3, true);
i2c_master_stop(cmd);
i2c_master_cmd_begin(I2C_NUM_0, cmd, 1000 / portTICK_RATE_MS);
i2c_cmd_link_delete(cmd);

After having successfully sent the command (and eventually after having waited for the sensor to execute it) you can read the output of the sensor creating a new command link with the sensor address (but this time with the READ mode) and adding one or mor read actions (based on the number of bytes the sensor sends):

uint8_t first_byte, second_byte;
cmd = i2c_cmd_link_create();
i2c_master_start(cmd);
i2c_master_write_byte(cmd, (0x40 << 1) | I2C_MASTER_READ, true);
i2c_master_read_byte(cmd, &first_byte, ACK_VAL);
i2c_master_read_byte(cmd, &second_byte, NACK_VAL);
i2c_master_stop(cmd);
i2c_master_cmd_begin(I2C_NUM_0, cmd, 1000 / portTICK_RATE_MS);
i2c_cmd_link_delete(cmd);

The main difference is that, after having read the last byte, the master sends the NACK signal. With this signal, it tells that it cannot receive more bytes and therefore the slave must stop transmitting.

ACK and NACK constants are defined as follows:

#define ACK_VAL    0x0
#define NACK_VAL   0x1

Demo

At the end of this post, I want to show you a classic example that uses the master mode: an I2C scanner. Goal of the program is to analyze the bus looking for any slave devices and print their address on screen.

It’s a very simple program, up to you to understand its source code in my Github repository. In the following post you’ll learn how to interface with a real sensor to get its data.

 

ESP32 (22) – SPIFFS

In some of the previous tutorials, I explained how to include “external” items (images, SSL certificates…) in your program thanks to the embedding binary data feature of the esp framework. Today I’ll show you how to use part of the flash memory to store data, like it was an hard drive or a memory card.

Partitions and filesystem

You’ve already learned in my fourth tutorial that the flash memory connected to the chip is divided into partitions to store different elements (the bootloader, the main program…) and that you can – via the menuconfig – configure the layout of the partitions using standard templates (for example “single factory app, no OTA”) or o in a totally custom way.

The list of the partitions, their position in the flash memory and their size are defined in the partition table, this too stored in the flash memory at address 0x8000.

We can define a custom layout writing a csv file with the following format:

spiffs-001

Each line represents a single partition: you have to specify its type, a subtype, the offset (i.e. the starting address of the partition) and the size. The csv file in the screenshot above corresponds to the following layout:

spiffs-002

To verify that the partitions are aligned (= there’s no empty space between them) you have to sum to the offset the size of the partition to find the offset of the next one… for example 0x9000 + 0x6000 = 0xf000.

A partition is just an “area” inside the memory: to be able to store data, you have to “organize” them. For example, I’ve already blogged about the NVS component that store data in the form “key-value” in a dedicated partition. An operating system organizes data on mass storage devices (memory cards, hard drives…) using a filesystem. During the years, many filesystems have been developed, depending on specific requirements or dedicated to specific devices. If you have a Windows PC, your harddrive is probably using the NTFS filesystem, while memory cards usually work with the FAT filesystem. A flash memory has unique features and requirements; for this reason Peter Andersson developed a filesystem optimized for them, named SPIFFS.

Component

To be able to use the SPIFFS filesystem in your program, you have to add to your project’s folder the component developed by Boris Lovosevic (loboris) which acts as a wrapper between the SPIFFS library by Peter Andersson (pellepl) and the Virtual File System (VFS) of the esp-idf framework.

First download from loboris’ Github repository the archive containing all the files:

spiffs-01

Now copy the spiffs folder from the zip to the components folder of your project:

spiffs-02

The zip contains also the tool (mkspiffs) you can use to “transform” a folder of your PC to a file that represents a SPIFFS partition, ready to be uploaded to the flash memory. You can copy the mkspiffs folder for example in your home directory:

spiffs-04

The spiffs component requires some configuration variables, which define the geometry of the partition, i.e. how it is “built”.

You have to add the following parameters to the Kconfig.projbuild file of your project (here I explained the purpose of this file):

  • SPIFFS_BASE_ADDR, address of the flash memory the SPIFFS partition starts from
  • SPIFFS_SIZE, size – in bytes – of the partition
  • SPIFFS_LOG_BLOCK_SIZE, size – in bytes – for each block of memory
  • SPIFFS_LOG_PAGE_SIZE, size – in bytes – for each page of memory

If you want to understand how the SPIFFS partition is divided into blocks and pages, you can read this document.

Base address and size must be the ones you configured in the csv file that defines the layout of the flash memory, while “good” values for block and page size are 8192 e 256.

When you prepare the csv file, you can specify the partition size using for example 1M for 1MB (1 Mega Bytes). This size equals to 1.048.576 bytes (1024 * 1024 bytes) and not – how you could think – to 1.000.000 bytes.

Prepare and upload the image

If you want to store in the SPIFFS partition some data before the program is executed (for example static images, audio… which will be then used by the program) you can use the mkspiffs tool to prepare a SPIFFS image with the content of a folder of your PC.

First you have to compile mkspiffs:

cd mkspiffs/src
make

spiffs-05

Now create on your PC a folder and copy in it all the files you want in your SPIFFS partition. You can also create subfolders… for example I created the spiffs_image folder in my home with the following content:

spiffs-06

Run the following command to create an .img file which represents an SPIFFS partition with the files above:

./mkspiffs.exe -c /home/luca/spiffs_image/ -b 8192 -p 256 
 -s 1048576 /home/luca/spiffs_image.img

The -b parameter is the value of LOG_BLOCK_SIZE, -p the value of LOG_PAGE_SIZE and -s the size of the partition. The mkspiffs command shows the content of the image just created:

spiffs-07

Now with esptool you can transfer the image into the flash memory:

python $IDF_PATH/components/esptool_py/esptool/esptool.py --chip esp32 
--port COM15 --baud 115200 write_flash --flash_size detect 0x180000 /home/luca/spiffs_image.img

In the command above replace the COM port with the one your devboard is connected to. Pay special attention to the address (0x180000) from which esptool writes the data: it must be the same you specified in the csv with the partition layout.

Use SPIFFS in your program

First include in your program the headers of the SPIFFS wrapper and the VFS component:

// VFS and SPIFFS includes
#include "esp_vfs.h"
#include "spiffs_vfs.h"

At the beginning of the program, register the SPIFFS filesystem:

vfs_spiffs_register();

This method informs the VFS component about the SPIFFS partition and which functions to call for the different operations (read, write…) on the filesystem. The method also mounts the partition (and formats it if necessary) to make it visible starting from the path /spiffs.

You can verify if the partition was correctly mounted with:

if(spiffs_is_mounted) {...}

Now, thanks to the VFS, you can use the standard C functions to work with files and folder. For example you can open for reading the “readme.txt” file in the root of the partition with:

FILE *file;
file = fopen("/spiffs/readme.txt", "r");

and read its content with:

int filechar;
while((filechar = fgetc(file)) != EOF)
[...]

In my Github repository you can download a program that allows the user to navigate in a SPIFFS partition with unix-like commands (cdls) and to display the content of text files (cat):

Conclusions

The ability to use part of the flash memory to store files and to access them using standard functions makes it easy to develop programs that require external elements (html pages, images, sounds…) without adding memory devices to the esp32 chip.

DCC shield for Arduino

In addition to the love for electronics, in my (little) spare time I’m working with my friend Davide on a model railway in H0 scale; in particular my main task is to take care of the digital control of locos, turnouts and accessories.

Davide’s main job is to create architectural and railway models. You can see his projects and follow the updates to our model railway on the dedicated Facebook page and website.

The de facto standard to digitally control a model railway is named DCC (Digital Command Control); it is a communication protocol defined by the american National Model Railroad Association. The specs of the protocol are available on NMRA’s website.

Using DCC, you control a loco sending commands through the rails to modules (decoders) installed within the loco itself: based on the commands it receives, the decoder controls motor, lights and in some cases also sounds and smoke generators…

dcc-01

DCC decoder installed in a locomotive

Specific decoders are also available to control turnouts and accessories:

dcc-02

turnout DCC decoder DCC by Esu

Thanks to Alex Shepherd and other developers, an Arduino library is available to receive and parse DCC commands.

You can install the library using the IDE’s Library Manager:

dcc-03

I designed in Eagle a simple shield to have an optocoupled connection between Arduino and the DCC bus:

dcc-04

The NmraDcc library requires that you connect the DCC signal to an Arduino pin which supports interrupts. Because of Arduino Uno has pin 2 and 3 with that feature, I added a jumper to the shield you can use to choose the pin:

dcc-05

The Eagle files for the shield are available on Github; in the same repository you can also find an example program, which changes color and brightness of an led based on speed and direction of the loco with id 10:

The use of the library is quite straightforward. First specify the pin for the DCC signal:

Dcc.pin(0, 2, 1);

The first parameter is the interrupt number, the second is the pin number and the third is the status of the internal pullup resistor (1 means enabled). You can find the link between pin and interrupt number in the official documentation:

dcc-06

Then initialize the library:

Dcc.init(MAN_ID_DIY, 10, 0, 0);

The first two parameters are the manufacturer and the version of the decoder. NMRA keeps a list of manufacturers and reserved the value 0x0D (constant MAN_ID_DIY) for opensource and DIY decoders:

dcc-07

With the third parameter you can change the default behaviour of the library:

dcc-08

while the last one is the EEPROM address from which the library stores its values.

To let the library process incoming packets, you have to call the process() method as frequently as possible in the loop:

Dcc.process();

If the library receives a command to change the speed/direction of a loco, it executes the callback function notifyDccSpeed:

void notifyDccSpeed( uint16_t Addr, DCC_ADDR_TYPE AddrType, uint8_t Speed, 
 DCC_DIRECTION Dir, DCC_SPEED_STEPS SpeedSteps ) {

in this function I implemented the code that controls the led.

dccshield-02 dccshield-01

RFID and Arduino (1)

In this tutorial, divided in two posts, I’ll show you how to use RFID (Radio-frequency Identification) tags with Arduino.

In the first part you’ll learn how to connect the reader to your Arduino and how to write a simple sketch to display the tag’s ID, while in the second part you’ll learn how to build a complete access control system based on RFID tags.

PN532

I chose as RFID reader a board based on the PN532 chip by NXP. This is a very versatile chip: it can work as a tag reader/writer but it can also act as a RFID tag; moreover it supports both I2C and SPI communication buses.

Adafruit created a breakout board for the PN532 chip and the Arduino libraries we’re going to use. Alternatively you can find on several webstores the following board, that I’m also going to use for this tutorial:

rfid-03 rfid-04

Connections

As I wrote before, the PN532 chip supports both I2C and SPI. For simplicity, I’ll use the first one, connecting the SDA and SCL pins of the board to the corresponding pins of Arduino. You have also to connect the IRQ pin to a digital pin of your Arduino (I chose pin 2); thanks to this connection the PN532 pin “warns” Arduino if a new tag is being read:

rfid-06 rfid-07

To select the I2C bus, you have to set the board’s dip switches as explained on the silk screen:

rfid-05

Finally, power the board connecting the VCC and GND pins to pins 5V and GND of Arduino.

For this project, I used the beta version of a new LCD shield by Lemontech. The main feature of this shield is that the LCD is connected to Arduino via an I2C expander; moreover all the buttons are connected to only one pin, the analog pin A0. This means that almost all the Arduino pins are still available for connecing other devices. The display’s default address – but you can change it – is 0x27 while the PN532 chip has address 0x24 so there’s no conflict.

rfid-10 rfid-11jpg

Having two devices connected to the I2C bus, I had to add two 10Kohm pull-up resistors for SCL and SDA as explained in the following schematics:

rfid-13

To keep things simple, I place them on a small breadboard:

rfid-08

I also added a small speaker (connected to pin 8 and GND) to play a sound everytime Arduino reads a tag:

rfid-09

Libraries

To be able to compile the sketch of this tutorial you have to install the following libraries in your IDE:

  • Adafruit PN532 di Adafruit
  • hd44780 di Bill Perry

Both the libraries are available in the Library Manager:

rfid-01

Sketch

The complete sketch is available in my Github repository.

To use the LCD, first you have to define its size (rows and columns), the address on the I2C bus and the pins it’s connected to. You can then initialize the library in the setup():

#define LCD_COLS      16
#define LCD_ROWS      2
hd44780_I2Cexp lcd(0x27, I2Cexp_PCF8574, 0,1,2,4,5,6,7,3,HIGH);
[...]
if(lcd.begin(LCD_COLS, LCD_ROWS) != 0) {
  Serial.println("- Unable to initialize LCD!");
  while(1);
}

The same for the PN532 chip: you have to declare the pin connected to the IRQ signal (RESET is optional) and then initialize it. Using the getFirmwareVersion method you can get the chip version and therefore verify that it’s working correctly:

#define PN532_IRQ     2
#define PN532_RESET   3
Adafruit_PN532 nfc(PN532_IRQ, PN532_RESET);
[...]
nfc.begin();
uint32_t versiondata = nfc.getFirmwareVersion();
if(!versiondata) {
  Serial.println("- Unable to find a PN532 board");
  while(1);
}
Serial.print("- found chip PN5"); 
Serial.println((versiondata>>24) & 0xFF, HEX);

Lastly, call the SAMConfig() method to configure the chip in normal mode and to enable the IRQ pin:

rfid-02

It’s very easy to read a tag. The readPassiveTargetID method returns true if a tag is near the reader:

success = nfc.readPassiveTargetID(PN532_MIFARE_ISO14443A, uid, &uidLength);
if (success) {

In this case, you can play a sound and display the tag’s ID on the display:

tone(SPEAKER_PIN, TONE_FREQ, TONE_TIME);
lcd.clear();
lcd.print("Found RFID tag!");
lcd.setCursor(1,2);
lcd.print("ID: 0x");
for(int i = 0; i < uidLength; i++) {
  if(uid[i] <= 0xF) lcd.print("0");
  lcd.print(uid[i] & 0xFF, HEX);    
}

Demo

ESP32 (21) – Mutual authentication

After having published my post about how to implement a webserver on the esp32 chip, some readers correctly warned me that everyone, once connected to the wifi network, could control the relay and asked me how to limit access to the webpage.

A classic solution (that I also used in this previous tutorial) is to ask the user to enter a password. It’s very easy to implement, but has a negative aspect, that is you have to type the password everytime you need to switch the relay.

You can avoid asking the password everytime the user access the page if you implement a login page and keep the user logged in using a session cookie as I’ll explain in a future tutorial!

Today I’ll show you how to protect your website using a feature of the SSL/TLS protocol: the mutual authentication.

Mutual authentication

Everytime you connect to a website using the https protocol, the server which hosts the site sends to your browser the SSL certificate of the site. Thanks to this certificate, you can verify the identity of the server and establish a protected connection to it:

ssl-02

The TLS protocol also offers the possibility for the server to require a certificate from the client: this means a two-way authentication, that is server and client authenticates each other using SSL certificates:

ssl-01

With this authentication method, a user has only to install on his device the client certificate (and the corresponding private key) and choose which certificate to send to the server if more than one is present:

ssl-03

The authentication with SSL certificates is therefore based on the principle of something I own (for example, a key) and not of something I know (for example, a password).

Certificates

Generally speaking, SSL certificates can be:

  • self-signed
  • signed by a Certification Authority

The first ones are perfect for internal use or for testing, while the second ones, normally signed by trusted CAs, are widely used in production environments or on the Internet. Sometimes you probably received this warning message when surfing on the Internet:

ssl-04

that means that your browser doesn’t trust the certificate sent by the server because it’s self-signed or signed by a non trusted CA.

For this tutorial, I’m going to use OpenSSL as a certification authority to create the certificates we need. If you’re using the toolchain provided by Espressif, you don’t need to install anything else because OpenSSL is included.

First create a folder that will contains all the files required by OpenSSL to operate as a CA in your home directory:

cd 
mkdir myCA

Now move into that folder and create some empty files:

cd myCA
mkdir csr certs crl newcerts private
touch index.txt
echo 1000 > serial

sslca-01

Copy the certification authority‘s configuration file (openssl.cnf) from my Github repository into the myCA folder. Open the file and change the main path accordingly to your system:

sslca-02

Now you have to generate the private key and the certificate for the CA. Let’s start with the key (its length will be 2048bit, enough to ensure good security):

winpty openssl genrsa -aes256 -out private/ca.key 2048

(all the commands must start with winpty only if you’re using Windows!)

You’re prompted for a password; it’s very important to remember it because you’ll have to type it everytime you use the CA:

sslca-03

Now generate the CA certificate (with a duration of 3650 days, that is 10 years):

winpty openssl req -config openssl.cnf -key private/ca.key -new -x509 -days 3650 -sha256 -extensions ca_cert -out certs/ca.cer

You have to enter some informations… the only mandatory is the common name which is the name of the CA, that will be included in all the certificates the CA will sign. At the end of the process you can open the file (ca.cer) and verify name and duration:

sslca-04

Let’s now generate the server and client certificates. The steps are the same for both:

  • generate a new private key – openssl genrsa
  • generate the CSR (Certificate Signing Request) file – openssl req
  • sign the CSR file with the CA to obtain the final certificate – openssl ca

Server certificate:

winpty openssl genrsa -out private/esp-server.key 2048
winpty openssl req -config openssl.cnf -key private/esp-server.key -new -sha256 -out csr/esp-server.csr
winpty openssl ca -config openssl.cnf -extensions server_cert -days 365 -notext -md sha256 -in csr/esp-server.csr -out certs/esp-server.cer

Client certificate:

winpty openssl genrsa -out private/esp-user.key 2048
winpty openssl req -config openssl.cnf -key private/esp-user.key -new -sha256 -out csr/esp-user.csr
winpty openssl ca -config openssl.cnf -extensions usr_cert -days 365 -notext -md sha256 -in csr/esp-user.csr -out certs/esp-user.cer

As said before, the client device must own both the certificate and the private key. You can merge certificate and key in a single p12 (or pfx) file with the following command:

winpty openssl pkcs12 -export -out esp-user.pfx -inkey private/esp-user.key -in certs/esp-user.cer

The way you install the P12 file on your device depends on the specific operating system it runs: on Windows you only need to double-click the file and follow the import wizard.

SSL webserver

The source code for the program running on the esp32 devboard is available in my Github repository.

First you have to copy in the prioject folder the certificates (CA and server ones) and the private key for the server certificate and include them – as text data – in the program (I’ve already explained how to include binary files in my previous post):

esp-ssl01

To implement the SSL protocol I’m using the mbedTLS library, included in the esp framework.

First include all the required header files:

#include "mbedtls/platform.h"
#include "mbedtls/net.h"
#include "mbedtls/ssl.h"
#include "mbedtls/entropy.h"
#include "mbedtls/ctr_drbg.h"
#include "mbedtls/debug.h"
#include "mbedtls/error.h"

Before being able to accept SSL connections, you have to define and initialize all the variables you’re going to use:

// mbed TLS variables
mbedtls_ssl_config conf;
mbedtls_ssl_context ssl;
mbedtls_net_context listen_fd, client_fd;
mbedtls_entropy_context entropy;
mbedtls_ctr_drbg_context ctr_drbg;
mbedtls_x509_crt srvcert;
mbedtls_x509_crt cachain;
mbedtls_pk_context pkey;
 
[...]
 
// initialize mbedTLS components
mbedtls_net_init(&listen_fd);
mbedtls_net_init(&client_fd);
mbedtls_ssl_config_init(&conf);
mbedtls_ctr_drbg_init(&ctr_drbg);
mbedtls_entropy_init(&entropy);	
mbedtls_x509_crt_init(&srvcert);
mbedtls_x509_crt_init(&cachain);
mbedtls_pk_init(&pkey);

Now parse the certificates and the private key and store them in the variables:

mbedtls_x509_crt_parse(&cachain, ca_cer_start, ca_cer_end - ca_cer_start);
mbedtls_x509_crt_parse(&srvcert, espserver_cer_start, espserver_cer_end - espserver_cer_start);
mbedtls_pk_parse_key(&pkey, espserver_key_start, espserver_key_end - espserver_key_start, NULL, 0);

Tell the library which variables to use for CA chain and own certificate and require the mutual authentication (MBEDTLS_SSL_VERIFY_REQUIRED):

mbedtls_ssl_conf_ca_chain(&conf, &cachain, NULL);
mbedtls_ssl_conf_own_cert(&conf, &srvcert, &pkey);
mbedtls_ssl_conf_authmode(&conf, MBEDTLS_SSL_VERIFY_REQUIRED);

After having configured other elements of the library (the random number generator and the debug function) you can bind it to the standard TCP port for the https protocol (443) and accept new incoming connections:

mbedtls_net_bind(&listen_fd, NULL, "443", MBEDTLS_NET_PROTO_TCP);
mbedtls_net_accept(&listen_fd, &client_fd, NULL, 0, NULL);

After having accepted a connection, you only have to call the function:

mbedtls_ssl_handshake(&ssl);

during the handshake process the library sends to the client its own certificate and require the one from the client, verifying that this is signed by the specified CA: the mutual authentication is implemented!

After the handshake, the program is similar to the previous example, you only have to use mbedTLS specific send/receive functions (mbedtls_ssl_read e mbedtls_ssl_write).

Test