Irrighino and Yun Rev.2

Irrighino has been one of my most appreciated projects: it’s a automatic irrigation system – based on Arduino Yun – with a web-based interface, optimized for smartphones.

After having published the project on this blog, it was reviewed by Open Electronics and a dedicated shield was also produced. The shield is still sold on Futurashop:

irrighino-shield-001

When it was launched the new Arduino Yun (named Rev.2) and the Yun Shield, the Linux distribution running on the board, OpenWrt, was updated as well. In particular, a key component for Irrighino has been replaced with its newer version, the php engine.

If you own a “first generation” Arduino Yun and you’d like to test the new functionalities shipped with Rev.2, in this post I explain how you can update the operating system of your board!

The commands you have to issue on your Yun to install all the required components for Irrighino now are:

install php with cgi and cli

opkg install php7 php7-cgi php7-cli

install curl, json and sqlite modules

opkg install php7-mod-curl php7-mod-json php7-mod-pdo php7-mod-pdo-sqlite

install zoneinfo

opkg install zoneinfo-core zoneinfo-europe

(replace zoneinfo-europe with the one related to your continent)

Final step is to configure the uhttpd webserver to run the php engine.

Edit the /etc/config/uhttpd file adding the following line (pay attention to its position!):

irrighino-php-002

then restart the webserver with:

irrighino-php-003

Now the Yun is ready to run Irrighino!

ESP32 (38) – Factory reset

In the previous two posts of this tutorial, I explained how to perform an over-the-air update of the firmware running on the esp32 chip.

Sometimes you may need to revert to the factory firmware, that is the firmware stored in the flash memory when the chip was programmed. Many consumer devices have a button or a pin that, if you press it for some seconds, triggers a reset function:

A10317_image1

In this post I’ll show you how to add this functionality to your project.

Partitions

As explained in a previous post, the flash memory connected to the esp32 chip is divided into some partitions, based on a layout configured when you program the chip.

Partitions that can store firmwares are of the app type. The partition that contains the firmware programmed via USB, has the factory subtype.

The esp-idf framework includes a method to search partitions in the flash memory:

#include "esp_partition.h"
[...]
esp_partition_iterator_t esp_partition_find(
  esp_partition_type_t type, esp_partition_subtype_t subtype, const char* label);

You can specify some filters to narrow down the results (they are not mandatory, use NULL if a filter is not needed): the type of the partition, the subtype and also a specific label.

If you want to look for the partition that contains the factory firmware, you can therefore write:

esp_partition_iterator_t pi = esp_partition_find(
  ESP_PARTITION_TYPE_APP, ESP_PARTITION_SUBTYPE_APP_FACTORY, NULL);

The method returns a partition iterator, that is an object that allows to scroll through the partitions found.

If the search was successful, this object is not NULL and you can get a pointer to the partition with the method:

const esp_partition_t* esp_partition_get(esp_partition_iterator_t iterator);

After the use, it’s important to release the iterator object with:

void esp_partition_iterator_release(esp_partition_iterator_t iterator);

After having obtained the correct partition, that contains the factory firmware, you only have to flag it as the boot partition and restart the chip:

if(pi != NULL) {
  const esp_partition_t* factory = esp_partition_get(pi);
  esp_partition_iterator_release(pi);
  if(esp_ota_set_boot_partition(factory) == ESP_OK) esp_restart();	
}

Demo

In the following video you can see how to perform a factory reset. In the video you can also learn how to “count” the number of seconds a button is pressed to trigger the reset function only after a fixed threshold (3 seconds in my example). Enjoy!

DCC, configure the decoder with CVs

In my previous tutorial I explained how to build a simple DCC accessory decoder with Arduino.

To keep the sketch simple, all the configuration parameters for the decoder (in particular its address) were defined as constants:

dcc-cv-001Often it can be useful to change the configuration of the decoder without re-programming its firmware (= the sketch that runs on the ATMega microcontroller). The DCC standard includes a document (S-9.2.2 DCC Configuration Variables) that introduces – for the decoders – the idea of Configuration Variables (CV).

CVs are parameters, stored in a non-volatile way (it means that the values are kept also when the power is off) in the decoders, which can define the behavior of the decoder itself.

NMRA defines, in the same S-9.2.2 document, a list of CVs – numbered from 1 to 1024 – and a meaning for each of them. For example, for an accessory decoder it is established that its address (9 bits) is contained in CV513 and 521:

dcc-cv-002

When you buy a commercial decoder, in the manual it is always given a table that indicates, for each supported CV, the meaning. For example, the manual of the ESU SwitchPilot Servo decoder, a decoder able to command servos to activate turnouts or other movements, lists the CVs that control position and speed of the 4 servos:

dcc-cv-003

Many digital stations support the service mode (document S-9.2.3), that is, they are able to program the values of the CVs of a decoder connected to a particular output, called the programming track. The model maker can therefore adjust the behavior of the decoders installed in the locomotives or that control accessories simply positioning the locomotives (or connecting the accessory decoders) on the programming track and programming the correct values in the CVs.

CV1 vs CV513

The NMRA standard provides that the CVs that can be used by accessory decoders are those from 513 to 1024. In fact, many manufacturers, in order not to create confusion or differences between the different types of decoders, have decided to use “low” CV numbers for accessory decoders. This is the reason why many accessory decoders use CVs 1 and 9 to configure their address.

NmraDcc

Once you have learned the meaning of the CVs, let’s see how to manage their programming in our sketch.

We have to store the CV values in a memory that is not erased every time we turn off the decoder. The ATMega328 processor used by the Arduino Uno has a non-volatile memory of 1024 bytes and, thanks to the Arduino’s EEPROM library, we can store and read values from this memory.

The NmraDcc library can independently manage the CVs or – alternatively – leave it completely to the external program.

For example, in the case of writing a CV (the command station that writes the value of a CV in the decoder), the NmraDcc library offers two callback functions:

  • extern uint8_t notifyCVWrite( uint16_t CV, uint8_t Value);
  • extern void notifyCVChange( uint16_t CV, uint8_t Value);

The first substitutes the standard CV handling of the library, while the second one is called to notify that a CV changed its value. If you inspect the source code of the library, the difference is clear:

dcc-cv-004

The library also defines the numbers of some standard CVs, including those related to the decoder address:

dcc-cv-005

LED decoder

Let’s go back to the sketch of the LED decoder of the previous tutorial and add the CV management.

Define two CVs:

  • the working mode (CV10)
  • the blink frequency (CV11)

Thanks to CV10 it will be possible to choose whether the activated LED will be stready on (CV value = 0) or blinking (CV value = 1). In this second case, the blink frequency will be configured via the CV11 (from value 1 = 5 seconds to value 100 = 50ms).

First, let’s define a constant for each CV and its number:

#define CV_ACCESSORY_DECODER_MODE        10
#define CV_ACCESSORY_DECODER_BLINK_FREQ  11

You also need some variables that will contain the runtime value for those CVs to avoid continuously reading the EEPROM (which, according to the datasheet, has a life cycle of about 100,000 reads/writes):

int decoderMode;
int blinkFrequency;

In the setup() method, you have to read the values of the CVs from the EEPROM and store them in the variables:

decoderMode = Dcc.getCV(CV_ACCESSORY_DECODER_MODE);
blinkFrequency = Dcc.getCV(CV_ACCESSORY_DECODER_BLINK_FREQ);

Let the library handle the CVs. To be informed about when a value changes and update the variables accordingly, implement the notifyCVChange() method:

void notifyCVChange(uint16_t CV, uint8_t Value) {
 
  if(CV == CV_ACCESSORY_DECODER_MODE) decoderMode = Value;
  else if(CV == CV_ACCESSORY_DECODER_BLINK_FREQ) blinkFrequency = Value;
}

Blink

To make the led blink, you can use the Timer1 library which allows to trigger a callback function periodically:

This callback function simply changes the led status:

void blinkLED() {
  digitalWrite(ledActive, !digitalRead(ledActive));
}

If the decoder is in blink mode, when the output status is changed, the active led is identified and the callback function is scheduled according to the frequency configured in the appropriate CV. Otherwise the led is simply switched on:

if(outputInPair == 0) {      
  digitalWrite(GREEN_LED_PIN, LOW);
  ledActive = RED_LED_PIN;
} else {
  digitalWrite(RED_LED_PIN, LOW);
  ledActive = GREEN_LED_PIN;    
}
if(decoderMode == 1) {
  Timer1.setPeriod(5000000 / blinkFrequency);
  Timer1.attachInterrupt(blinkLED);
} 
else digitalWrite(ledActive, HIGH);
}

lettura CV

The DCC standard also includes the possibility – for a command station – to read the value of a CV of a decoder connected to the programming track. In a next article you’ll learn how to implement this feature!

Demo

Here’s a video that shows how it is possible to modify the behavior of the led decoder programming its CVs using a laptop that runs the Rocrail application connected to a SPROG command station:

Arduino Yun, how to update to rev.2

This year in March, Massimo Banzi announced on the Arduino blog the production of a new revision (Rev.2) of the Arduino Yun board.

yun-rev2-001

In addition to some improvements on the hardware design, the new revision offers a complete update of the software side of the Yun, that is the Linux distribution (OpenWrt) running on the board. The original Yun was indeed stuck to a distribution released in 2014 (still available in the download section of Arduino website) and that distribution contained outdated software packages which also presented some security vulnerabilites.

Who now buy a new Yun board, has the new OS version pre-installed. If you own a “Rev.1″ board, in this article you’ll learn how to update it…

Updater

In the Arduino forum, Martino Facchin (one of the contributors of the OpenWrt porting to Yun) published a post with links to an automatic updater for the different platforms (Windows, Linux, Mac). The application updates both the bootloader and the operating system (rootfs). You have to also update the bootloader because of the new OS requires a bigger partition than the original one.

Before running the updater process, there are 3 prerequisites:

  • the Yun board must be connected both to your computer via USB and to your home router/switch using an ethernet cable (the update process cannot happen using wifi)
  • your computer, which runs the updater application, must be connected to the same home network the Yun is connected to
  • your computer must not run softwares (like firewalls) that could block incoming connections

The third point in particular is the one that can give more problems… for example I use the Eset Internet Security suite and I had to temporary disable the protection:

yun-rev2-002

An additional suggestion is to disable other network cards in your computer, keeping active only the one connected to the home network:

yun-rev2-003

If you’re using Windows 10, it may happen that the yun-go-updater.exe program is blocked… in this case you have to cancel the block from the file properties:

yun-rev2-004

If you run the program, the update process should go automatically. When complete, if you connect to the Yun board you should see the new version:

yun-rev2-005

DHCP

During the update process, the Yun board requests a valid IP address for your network using the DHCP protocol. Sometimes this request could fail; in this case you can assign a static IP to the board answering “n” to the first question:

yun-rev2-006

ESP32 (37) – https OTA

In the previous post of this tutorial, I explained how it is possible to update your board Over-The-Air thanks to a feature of the Freshen IoT dashboard.

Today I’ll show you how to update the firmware running on an esp32 chip using only components included in the esp-idf framework, without the need of any external tools or platforms.

OTA API

The esp-idf framework offers a set of native functions to implement, in your program, the ability to be updated over the air.

Those functions are grouped in the app_update component and to use them in your program you have to include the corresponding header file:

#include "esp_ota_ops.h"

Altough the use of the native functions is not very difficult (on Github you can find an example program), Espressif developers have added a component to the framework that makes it even easier the over the air update if the new firmware is located on a web site.

The component is named esp_https_ota.

esp_https_ota

The esp_https_ota component uses the OTA API to update the firmware of your board, downloading the binary file that contains the new firmware from a web site. As the name suggests, the only requirement (for security reason) is that the web site supports the secure version of the protocol (HTTPS).

The component is able to automatically identify an OTA partition in the flash memory that is not in use and to save the new firmware in that partition. It then configures the chip to boot from that partition:

https_ota_001

The use is very simple. First create an esp_http_client_config_t struct to configure the URL of the file with the new firmware and the SSL certificate of the server (or the certificate of the CA that signed it):

esp_http_client_config_t ota_client_config = {
  .url = "https://mysite.com/newfirmware.bin",
  .cert_pem = server_cert_pem_start,
};

You have to provide the certificate in PEM format. To store the certificate in your program, you can leverage the embedding binary files functionality of the firmware, as I already explained in a previous tutorial.

Then you only have to call the function:

esp_err_t ret = esp_https_ota(&ota_client_config);

to start the update process. If – when the process is complete – the ret variable contains a positive result (ESP_OK), you can reboot the chip to run the new firmware:

esp_restart();

A real application would probably need to periodically check if a new firmware is available and, only in that case, to start the update process. How can it be done?

In the program I wrote for this post and that is explained in the video below, I’m going to show a way widely used also in commercial products… enjoy the show ;)

as usual, the source code of the program is available in my Github repository

ESP32 (36) – OTA with Freshen

In one of the first posts of this tutorial, I wrote about the bootloader and about how the flash memory is organized. That article says:

This allows to implement an over-the-air (OTA) application update process: you send the new version of your application to the esp32 chip; the version is stored in a new app partition

There are different ways to implement OTA updates for your application… today I’ll show you how to do it in an easy way, thanks to a cloud service named Freshen.

Freshen

Freshen is an IoT backend, that is a cloud dashboard (published on the Internet) to manage IoT devices.

After having connected your devices to Frashen (I’ll show you later how to do it) you can:

  • display the list of the devices and their status
  • send a command to a device
  • manage the files stored in a device
  • update the firmware over-the-air (OTA)

Freshen is developed by Cesanta, the company that also develops the Mongoose library and the MongooseOS, very used in the embedded world and compatible with the esp32 chip. The service has different fees, including a free one:

freshen-001s

To use Freshen, first you have to register, using your Github or Google account:

freshen-002

Client library

Cesanta offers a client library you can use to connect your IoT project based on the esp32 chip to Freshen. The library consists only in a header file (freshen.h), and is available on the official site.

The library is fully compatible with the esp-idf framework and supports all the functionalities of the dashboard:

freshen-003

It’s also very easy to use. First copy the freshen.h file in the main folder of your project:

freshen-009

Then include the file in your program:

#include "freshen.h"

To let the library communicate with the Freshen platform in the cloud, in your program you have to periodically call the freshen_loop() function:

You can do it in a dedicated task:

void freshen_task(void *pvParameter) {
  while(1) {	
    freshen_loop(FIRMWARE_VERSION, ACCESS_TOKEN);
    vTaskDelay(2000 / portTICK_RATE_MS);
  }
}

This task calls the function every 2 seconds. Create the task in your app_main() with:

// start the freshen update task
xTaskCreate(&freshen_task, "freshen_task", 10000, NULL, 5, NULL);

The freshen_loop() function requires two parameters: the firmware version (it’s a text string at your choice, for example “v1.0″) and the access token, a code that is generated by the platform when you register a device.

Connect to the dashboard and click on Add new device:

freshen-010

A new device will be added (My Device #n). You can click on its name to display the details.

The access token is hidden… Click on click to copy to save it in the clipboard on your computer; you can then paste it in your program:

freshen-011

The device details page also allows to change the name of your device.

If you run the example program (you can download it from my Github repository) you’ll notice that, after a couple of seconds, the status of your device changes to online, to indicate that it’s correctly sending data to the dashboard:

freshen-012

In the details page, you can now list the functions that you can remotely call:

freshen-013

For example, if you call the Sys.GetInfo function you can retrieve the information about firmware version, architecture, compile date…:

freshen-014

OTA

To be able to update the firmware of your device over the air, you have to select an appropriate partition layout. For example you can choose the “OTA” layout that is already included in the framework:

freshen-006

Let’s now modify the firmware version, to be “1.1”:

freshen-015

then compile the program but don’t use the flash command. Note the path of the binary (compiled) file:

freshen-016

Select your device in the dashboard and click on OTA update selected:

freshen-017

 

Choose the .bin file that contains the new version of your firmware.

After a few moments, you’ll probably see the device going offline and return online. If the update process was successful, call the Sys.GetInfo function again and you’ll see the new version:

freshen-018

 

MIUI and tethering

My smartphone is a Xiaomi Redmi Note 3 Pro, running the “Global” MIUI ROM. Some days ago I brought a SIM card from a new italian operator, ho-mobile, which allows to use the tethering feature of the phone to connect external devices to the Internet.

I faced some problems before being able to use that feature… this blog post is to describe how I solved them.

Tethering was not working due to the incorrect configuration of a property of the Android operating system, called

tether_dun_required

As explained in Android source code, this property tells the O.S. if it’s necessary to use a DUN (dial-up networkAPN for the tethering funcion. Actually, as well explained on Taming the Droid:

This refers to an outdated method for using your phone to emulate a dial-up modem and is not the way that modern smartphones do tethering anymore. There should be no need to use this value.

To set the property to 0 (= not required), you have to send a command in the smartphone’s shell.

Requirements

You need to:

Procedure

In the Developer Options menu, enable the following 3 items:

  • USB debugging
  • Install via USB
  • USB debugging (Security settings)

x-teth001

You’ll be prompted, if not already done, to login with your MI account.

It may happen that, when you enable an item, you get the following error:

x-teth002

The error means that your phone cannot contact Xiaomi servers, probably because of the Great Firewall of China.

You can solve the problem thanks to an app that establishes a VPN with a chinese server, that is behind the firewall. Install, from Play Store, the Flit VPN app and run it.

Choose Start without registration, then click on Connection Settings and choose one of the Jangsu server, the one with best performances:

x-teth003

Click on Connect and wait until the connection is established:

x-teth004Now you should be able to enable all the debugging items without any problems. When enabled, you can disconnect the VPN and uninstall the app.

Connect your smartphone to the computer via USB and run Minimal ADB and Fastboot.

Type the command adb shell to open a console connection with the phone.

Using the settings get global tether_dun_required command you can read the current value of the parameter. If you read null or 1 the setting is wrong:

mi-teth002

Use the settings put global tether_dun_required 0 command to set the value to zero. You can test if the command was successful re-entering the get command:

mi-teth003

NB: if you see a permission denied error, it means that not all the debugging features have been correctly enabled:

mi-teth001

Now you can use the tethering of your phone, happy surfing!

SeeedStudio, 3 PCBs for 1$

Here’s a new promo for the Fusion PCB service by SeeedStudio: from today untill September 21st, you can order 3 PCBs for 1$ (excluding shipping costs)

seeed-1dollar

To take advantage of the offer, simply enter the order from the dedicated page of their website.

The offer includes PCBs with the following characteristics:

  • 1/2 layers
  • max size of 10x10cm
  • thickness 0.6 -> 1.6mm
  • one of the 6 available colors (green/red/yellow/blue/white/black)

If you want to learn how to use the Fusion PCB service, here’s the link to my tutorial!

 

The smallest esp32 module (so far…)

Some months ago, Espressif announced the production of a new chip, named ESP32-PICO-D4.

esp-pico-001

It’s a complete SiP (System in Package), that is a chip which integrates the esp32 microcontroller, a 4Mbit flash memory, a crystal oscillator, filter capacitors and RF matching links. The chip datasheet is available on the official website.

Using this chip, it’s possible to create very small modules. I recently received one of those from Aliexpress:

esp-pico-002

esp-pico-003

To better understand how small it is, let’s compare its size with a “classic” ESP-WROOM-32 module and with a 1 euro coin:

esp-pico-004

The module includes a chip antenna; it’s also possible to connect an external antenna thanks to the presence of an I-PEX connector.

In conclusion, the availability of the ESP32-PICO-D4 SiP makes it possible to use the esp32 chip in applications where the available space is very small…

ESP32, PlatformIO

PlatformIO is an opensource ecosystem (as it’s defined in the homepage of the project) to develop IoT projects.

The heart of the platform is a software component named PlatformIO Core. This component includes:

  • a cross-platform compiler
  • a libraries and dependences manager
  • a serial monitor

PlatformIO Core is developed in Python and therefore it can run on different operating systems (Windows, Linux, MacOS).

Although you can use the Core component directly, PlatformIO’s strength lies in its IDE, which allows the development of multi-platform projects and integrates with the Core itself.

In this article I’m going to show you how to use PlatformIO to develop projects running on the esp32 chip.

Installation

PlatformIO IDE is provided as a plugin for two different development tools: Atom and VisualStudio Code. I tried both solutions and I preferred VSCode: both the installation and the use are simpler and more immediate.

Install VSCode after having downloaded the package from Microsoft’s website (the installer is available for Windows, Linux and MacOS).

Open the Package Manager:

pio-001

search the PlatformIO IDE package, then click on Install:

pio-002

wait until the installation is complete:

pio-003

Hello world

Now it’s time to develop your first program, which traditionally will display the sentence Hello world! on the terminal.

If it doesn’t show up automatically, open the PlatformIO’s homepage:

pio-004

then click on New Project:

pio-005

give a name to the project and choose a devboard based on the esp32 chip (in this example I’ll use a Lolin32 board by Wemos). PlatformIO supports both the esp-idf framework and the arduino-esp32 one. All my tutorials are based on the first one:

pio-006

PlatformIO automatically creates some folders for your project. Choose the src folder (it stands for source, that is the folder which will contain the source code) and create a new file:

pio-007

name the file main.c and type the simple program as it follows:

pio-008

run the compiler by clicking on the corresponding button in the bottom bar:

pio-009

the editor displays an error… indeed your code is using the printf() function without having included the library:

pio-010

add the missing line, now you should be able to complile the code without errors:

pio-011

PlatformIO can also upload the compiled program to your board. Thanks to its auto-detect feature, you usually don’t need to specify the serial port the board is connected to:

pio-012

Serial monitor

PlatformIO also includes a serial monitor you can use to test your program. By default, this monitor connects to the serial port with a speed of 9600 baud. The esp32 chip instead has a default speed of 115200 baud; you have therefore to change the platformio.ini file included in your folder as it follows:

pio-013

Now open the serial monitor; you should see the correct output of your program:

pio-014

Conclusions

I found the use of PlatformIO really immediate: after a few minutes I was able to develop, compile, load and test a program. Try it and leave a comment with your impressions!