ESP32 (28) – MQTT and SSL

Security is a very important aspect for MQTT brokers. In a previous article you’ve already learned how to implement authentication and authorization. The weakness in that configuration was that credentials were transmitted in cleartext; it was therefore possible, for an attacker who can sniff the network traffic, to read and use them to impersonate a legitimate client.

Today I’ll show you how to encrypt the communication channel between client and broker using SSL certificates. I’ll also explain how to write a program for the esp32 chip to send data to the broker using the secure channel…

SSL certificate

To be able to encrypt the communication, mosquitto requires a server certificate.

First generate the private key (RSA with a length at least 2048 bits):

openssl genrsa -out mosquitto.key 2048

Then create the CSR file:

openssl req -new -out mosquitto.csr -key mosquitto.key

type the required information; the most important of which is the common name that will identify the server:

mq-ssl-001

Now sign the CSR file with your Certificate Authority (or send it to a public / corporate CA) to generate the certificate:

openssl ca -config openssl.cnf -extensions server_cert 
 -notext -in mosquitto.csr -out mosquitto.cer

mq-ssl-002

Create the ssl subfolder in the folder where you installed mosquitto and copy into that folder the certificate, its private key and the certificate of the CA:

mq-ssl-003

Configuration

Open the mosquitto.conf file and add the following lines:

mq-ssl-004

The first line changes the TCP port mosquitto is normally listening to (1883) to the default port for SSL connection, 8883.

The following 3 lines set the path for server and CA certificates and for the private key that corresponds to the server certificate. The last one, which is not compulsory, forces the use of the TLS v1.2 protocol, the most secure one at the moment of writing.

Once the server has been configured, you can start it (-v is to enable the verbose output):

mosquitto.exe -c mosquitto.conf -v

To be able to use the mosquitto_pub and mosquitto_sub tools, you now have to add new parameters:

mosquitto_sub.exe -p 8883 -t test --cafile .\ssl\ca.cer --insecure
mosquitto_pub.exe -p 8883 -m 20 -t test --cafile .\ssl\ca.cer --insecure

With -p you specify the TCP port of the server, with –cafile the path of the CA certificate which signed the server certificate mosquitto uses and finally with –insecure you configure the two clients not to verify that the certificate’s common name (in my example mymosquitto.local) corresponds to the server name.

You can avoid using the –insecure switch if you generate a certificate with a common name the exact name of the server/PC which runs mosquitto or – alternatively – if you add a DNS alias which resolves the common name of your certificate with the IP address of the server and run the clients with -h commonName

esp32

Tuan PM developed a library (espmqtt) for the esp-idf framework that implements a complete MQTT client. Moreover, the library does support secure connections, you can therefore use it to connect to an MQTT broker with TLS enabled.

mq-ssl-005

Copy the content of the Github repository in the components folder of your project and include the library’s header file in your source code:

#include "mqtt.h"

The MQTT client is configured using the mqtt_settings struct:

mqtt-001

The most important parameters are:

  • the server (host) that runs the MQTT broker (you can use the IP address or the DNS name)
  • the TCP port (port) the server is listening to (default is 1883 or 8883 if SSL is enabled)
  • username and password if the server requires authentication
  • one or more callback functions the espmqtt library will call when the corresponding event occurs
the espmqtt library does not copy the parameters in an internal struct. It’s therefore very important that the mqtt_settings variable has a global scope and it’s defined outside a specific function.

Your program can interact with the MQTT client implementing its callback functions.

For example the connection or disconnection from the MQTT server occurs as follows:

mqtt-002

The connect_cb and disconnect_cb functions perform the “real” connection and disconnection, while connected_cb and disconnected_cb functions are executed after the corresponding activity is completed (= the client successfully connected to the server). Your program usually doesn’t need to re-implement the main functions, but will implement the ones related to events, to execute actions (for example subscribe a topic) when a specific event occurs.

After having configured the client, you can run it with:

mqtt_start(&settings);

Once connected to the server (connected_cb callback function) you can subscribe or unsubscribe a topic with:

void mqtt_subscribe(mqtt_client *client, const char *topic, uint8_t qos);
void mqtt_unsubscribe(mqtt_client *client, const char *topic);

and publish data to a topic with:

void mqtt_publish(mqtt_client* client, const char *topic, 
  const char *data, int len, int qos, int retain);

Demo

I prepared an example to show my esp32 devboard sending data to a mosquitto server, with SSL enabled.

I connected to the devboard an HTU21D sensor as explained in a previous article and my program reads, every 5 seconds, the temperature and humidity values and sends them to the broker. I used a very handy opensource program, HelloIoT, to create a dashboard and display the received data.

The source code of the program and the configuration of the HelloIoT dashboard are available in my Github repository; here’s a short video of the demo:

MQTT Mosquitto on a Pi Zero W in under 5 minutes // MickMake QuickBits

Setting up the Mosquitto MQTT Broker is pretty easy. In this video I’ll show you how to setup a Broker in under 5 minutes. Updating Raspbian If you followed my previous article on installing Raspbian without a keyboard or screen, Continue reading MQTT Mosquitto on a Pi Zero W in under 5 minutes // MickMake QuickBits

The post MQTT Mosquitto on a Pi Zero W in under 5 minutes // MickMake QuickBits appeared first on MickMake.

MQTT with lwip and NXP FRDM-K64F Board

lwip-ping-example

Erich Styger from MCU on Eclipse writes, “In this article I show the basic steps to get MQTT running on the NXP FRDM-K64F board using MCUXpresso IDE, lwip and MQTT. lwip ois a small and open source TCP/IP stack which is widely used. To keep things very simple in this first post, I’m using it in bare-metal (no RTOS) mode with no encryption/security. The principle applies to any IDE/toolchain, as long there is a lwip port available for your board and IDE/toolchain. I’m using the MCUXpresso IDE as it nicely integrates with the MCUXpresso SDK which includes a lwip port for the FRDM-K64F.”

More details at MCU on Eclipse homepage.

Raspberry Pi Home Automation for the Holidays

When you want to play around with a new technology, do you jump straight to production machinery? Nope. Nothing beats a simplified model as proof of concept. And the only thing better than a good proof of concept is an amusing proof of concept. In that spirit [Eric Tsai], alias [electronichamsters], built the world’s most complicated electronic gingerbread house this Christmas, because a home-automated gingerbread house is still simpler than a home-automated home.

fya59blixaq00y3-largeYeah, there are blinky lights and it’s all controlled by his smartphone. That’s just the basics. The crux of the demo, however, is the Bluetooth-to-MQTT gateway that he built along the way. A Raspberry Pi with a BTLE radio receives local data from BTLE sensors and pushes them off to an MQTT server, where they can in principle be read from anywhere in the world. If you’ve tried to network battery-powered ESP8266 nodes, you know that battery life is the Achilles heel. Swapping over to BTLE for the radio layer makes a lot of sense.

If you’re thinking that you’ve seen this before, maybe you’re thinking of [electronichamsters]’s previous feat of home-automation and cardboard, which is also great fun. If a web search with the keywords “IoT” and “hamsters” is what brought you here: Hackaday aims to please.


Filed under: Holiday Hacks, home hacks, Raspberry Pi

PURE Modules Aim to Make Prototyping Easier

[Sashi]’s PURE modules system wants your next wireless microcontroller and sensor module project to be put together using card-edge connectors. But it’s a lot deeper than that — PURE is an entire wireless gadget development ecosystem. Striking a balance between completeness and modularity is very difficult; a wire can carry any imaginable electronic signal, but just handing someone a pile of wires presents them a steep learning curve. PURE is at the other end of the spectrum: everything is specified.

So far, two microcontroller options are available in the system, the nRF52 series and TI’s CC2650. Both of these run the Contiki OS, so it doesn’t matter which of these you choose. Wired data is all transmitted over I2C and connects up via the previously-mentioned card-edge connectors. On the wireless side, data transport is handled through an MQTT broker, using the MQTT-sn variant which is better suited to small radio devices. At the protocol layer everything uses Protocol Buffers, Google’s newest idea for adding some structure to the data.

modules_in_arduinoWhat all this means is that it should be very easy to get code, devices, and WiFi-connected computers and services all working together. Besides the overview available at Hackaday.io, there are hardware and software repositories on GitHub, and [Sashi] even put together an Instructable to get you started making your own modules. It’s also up on Kickstarter at the moment, so check that out if you’re not feeling DIY.

PURE uses current open-source and interoperable technologies at every step of the way, coming together into what looks like a well thought-out system. Will something like this help you escape the connector zoo that characterizes our modern hacking scene? We have no idea, but it looks like a totally reasonable fifteenth standard.


Filed under: misc hacks

MQTT – enc28j60 and Adafruit IO

Adafruit IO is the new cloud platform (at the moment still in beta) by Adafruit, designed to allow simple data connections using standard APIs and web-based dashboards.

This new platform exposes MQTT APIs, you can therefore apply what you learned in my previous posts to create a complete project that takes advantage of all its features.

The project

Goal of the project is to connect the Arduino to a dashboard hosted on Adafruit IO:

  • Arduino reads the room temperature (thanks to the DHT11 sensor already used in a previous tutorial) and sends its value every t seconds to Adafruit IO
  • Arduino is also able, using an external relay module, to turn on/off an heater
  • The dashboard on Adafruit IO displays the temperature values on a chart and has a toggle button to turn the heater connected to the Arduino on and off

mqtt-io1

Adafruit IO

Let’s see how to create the dashboard using Adafruit IO.

First, you need to subscribe to the open beta program using the button on the official website https://io.adafruit.com:

io-01

After having logged in with your Adafruit account (or having created a new account) you’ll be redirected to the default dashboard.

Click on My dashboards:

io-02

next click on Create dashboard:

io-03

give a name to the new dashboard, then click on Create dashboard:

io-04

you’ll see a completely empty dashboard. To add new elements (charts, buttons…) click on Create a new block:

io-05

choose line chart among the available elements and click on  Create:

io-06

you have to link the chart to a feed (= topic, in the MQTT language). Create a new feed named temperature:

io-07

once created, you can link it to the chart (choose), then click on next step:

io-08

you can change some chart settings (for example type °C as y-axis label), then complete the wizard with create block:

io-10

You’ll see a new block in your dashboard that represents the chart, empty for the moment. Using the drag’n’drop you can move the block in the page or resize it.

Now add the toggle button, again using Create a new block:

io-20

create a new feed named control:

io-21

and link it to the button:

io-22

give a label to the button and to the two possible values, then create the new block:

io-23

Your dashboard is now complete!

Feeds, topics and MQTT APIs

As explained above, each element of your dashboard can be linked to one or more feeds, that are data flows. For example you linked to the chart a feed named temperature, while the button is linked to another one named control.

If you use the MQTT APIs, each feed corresponds to a topic named as it follows:

<username>/f/<feedname>

where username is the name of your Adafruit account.

Hence, my two feeds correspond to the following two topics:

  • lucadentella/f/temperature
  • lucadentella/f/control

To connect to the Adafruit’s MQTT broker (address io.adafruit.com) you need to specify a username and a password.

The username is the name of your Adafruit account (lucadentella for me), while the password is your Adafruit IO key, that can be retrieved using a button in the dashboard:

io-31

io-32

You can test your dashboard using the mosquitto_sub command to subscribe to the topic linked to the toggle button. Every time you click on the button, the dashboard sends to that topic a message with the actual status of the button (ON or OFF):

mosquitto_sub.exe -h io.adafruit.com -u <username> -P <AIOkey> -t <username>/f/control

io-33

Arduino

The sketch for Arduino is available in my Github repository.

First, some constants are defined: the parameters for the Adafruit IO service (username, password, topics…) and the PINs the DHT11 sensor and the relay are connected to:

#define CLIENT_ID       "ArduinoMQTT"
#define USERNAME        "adafruit_username"
#define PASSWORD        "adafruit_io_key"
#define PUB_TOPIC       "username/f/temperature"
#define SUB_TOPIC       "username/f/control"
#define PUBLISH_DELAY   5000
#define DHTPIN          3
#define DHTTYPE         DHT11
#define RELAY_PIN       6

In the setup(),  the MQTT client is configured. Unlike the previous example, this time I also define the callback function that is called every time the client receives a new message from a topic that was subscribed:

// setup mqtt client
mqttClient.setClient(ethClient);
mqttClient.setServer("io.adafruit.com", 1883);
mqttClient.setCallback(mqttCallback);

Indeed, when the sketch connects to the broker it subscribes the topic linked to the button in the dashboard to be notified when a user clicks on it:

void mqttConnect() {
 
  while(!mqttClient.connected()) {
 
    if(mqttClient.connect(CLIENT_ID, USERNAME, PASSWORD)) {
 
      Serial.println(F("MQTT client connected"));
      mqttClient.subscribe(SUB_TOPIC);
      Serial.println(F("Topic subscribed"));
    } else {
      Serial.println(F("Unable to connect, retry in 5 seconds"));
      delay(5000);
    }
  }
}

The callback function checks if the message received is ON or OFF and changes the relay status accordingly:

void mqttCallback(char* topic, byte* payload, unsigned int length) {
 
  if(strncmp((const char*)payload, "ON", 2) == 0) {
    Serial.println("ON message received, turning relay ON");
    digitalWrite(RELAY_PIN, HIGH);
  } else {
    Serial.println("OFF message received, turning relay OFF");
    digitalWrite(RELAY_PIN, LOW);
  }
}

The following video shows the project in action:

MQTT – Security

In the IoT they often don’t give much importance to the security aspects of the communcation. Proof of this is that many of the latest DDOS (Distributed Denial of Service) attacks have been launched by hacked Internet-connected smart devices.

In the previous posts, you learned how to configure mosquitto to receive messages published by clients and to forward them to all the subscribers. Today I’m going to show you how to configure the application security of your MQTT broker.

This post is about the standard settings mosquitto offers. On the Internet you can find several plugins to extend its functionalities and to implement advanced security settings, like storing accounts in different backends or using json web tokens.

Authentication

The first step to make our broker secure is to require clients who are connecting to authenticate themselves. In this way, only authorized clients can send/receive messages.

Open the mosquitto.conf file and, first of all, disable the anonymous access setting to false the corresponding parameter.

mqtt-sec01

When you change a setting in the file, make sure that the hash (#) character is not present at the beginning of the line, otherwise the parameter is not considered

The list of the users authorized to connect to the broker is defined in a dedicated file. The name of this file is configured with the password_file parameter (in the example below, the file containing users and passwords is named “pwdfile”, stored in the same folder of mosquitto):

mqtt-sec02

Using the mosquitto_passwd.exe command you can create (-c) the file and add a new user (“luca”). The command will ask to type the user’s password twice:

mqtt-sec03

If you need to add additional users, you can run the command again, without the -c option:

mosquitto_passwd.exe pwdfile sara

Let’s try the new configuration. First run mosquitto specifying the configuration file (mosquitto.exe -c mosquitto.conf), then try to publish a message as you learned in the previous tutorials:

mqtt-sec04

You can see that the connection is refused because of the client is not authorized: we indeed didn’t specify any users and anonymous access was disabled.

Try to run the command again, this time specifying the correct user and password and you’ll find that the message will be correctly published:

mosquitto_pub.exe -u <utente> -P <password> -t home/bedroom/temp -m 19

Authorization

After having defined who can connect to the broker, it’s now time to define what a user can do once connected. Again, the configuration of the ACLs (access control lists) is in a dedicated file, whose name is configured with the acl_file parameter:

mqtt-sec05

The ACL file contains blocks with the following structure:

  • the first line, user <nomeutente>, that defines the user the following ACLs are applied to
  • one or more lines, topic <read|write|readwrite> <nometopic>, that define which actions the user above can do on the specified topic

Let’s see an example, a file in which we define a user (writeclient) that can send messages to the secure/data topic and a user (readclient) that can read those messages. Both the two users can in addition send an receive messages from all the open/# topic (for an explanation about the use of wildcards read my previous post):

mqtt-sec06

(lines starting with # are comments)

Let’s now try to send a message using the readclient user, who’s not allowed to do that:

mqtt-sec07

The client doesn’t get any error message, but in the mosquitto logs you can clearly see that the publish command was denied, consistent with the configured ACLs.

If instead the right users are specified, the message is correctly routed from the publishing client to the subscribing one:

mqtt-sec08

Conclusions

It’s very important to secure our broker, especially if it’s connected to a public network (for example if it’s available on the Internet). In today’s blog post you learned how to define users and grants in mosquitto.

MQTT – Topics

In the first part of this tutorial, I introduced the concept of topics. Every message sent to an MQTT broker must be published to a topic and clients connected to the broker decide which messages to receive subscribing to one or more topics.

Today you’ll learn the format of the topics and different ways to subscribe to them.

A topic is a string (UTF-8 format) and consists of one or more levels, separated by a forward slash.

Here are some examples of topics:

commands
home/temperature
garden/tomatoes/humidity

A topic must have at least one character and can contains spaces (even if it’s not recommended).

Note that topics are case sensitive, therefore the following topics are different: Home/Temperature and home/temperature

You’re not required to define/create a topic on the broker to use it: a client sending a message can specify any topics, provided it’s syntactically correct.

Wildcards

A client can subscribe one or more topics. The subscription can be precise, specifying the exact name of the topic to be subscribed. Alternatively, you can use two wildcards, that are special characters used to subscrive a group of topics.

The first wildcard is the character. This subscribes to a single level of topics.

Let’s see an example: if you subscribe to the topic home/+/temperature, you subscribe to all the topics that contain an arbitrary string instead of the wildcard, for example:

  • home/living/temperature
  • home/kitchen/temperature
  • home/terrace/temperature

Here are on the contrary some topics thare are not subscribed:

  • garage/temperature
  • home/living/humidity
  • home/terrace/dx/temperature

The last example explains that the + character is wildcard for a single level (it therefore does not correspond to the two levels, terrace/dx, of the example).

The multilevel wildcard character is #. This must be the last character in the topic and preceded by a forward slash, /.

For example, if you subscribe to the topic home/indoor/# you’ll get the messages of the following topics:

  • home/indoor/temperature
  • home/indoor/kitchen/temperature
  • home/indoor/kitchen/humidity

while you won’t get messages from the following topics:

  • home/outdoor/temperature
  • garage/humidity
If you subscribe to the # topic (without specifying any root string) you’ll receive all the messages sent to the broker

Let’s try!

We can verify what explained above thanks to mosquitto. After having executed the server, let’s subscribe to the topic home/+:

mosquitto_sub.exe -t home/+ -v

(we use the -v option to display, together with the message, also the topic the message was published to).

If now we send messages to different topics, we can verify which of them are received and which not:

mqtt-wild4

We can do a similar test for the wildcard #:

mqtt-wild3

MQTT – enc28j60 and Arduino (22)

Being a comics fan, I couldn’t resist publishing a crossover between my “famous” tutorial about using the enc28j60 driver with Arduino and the fresh new tutorial about the MQTT protocol.

So, after having introduced the MQTT protocol and explained how to use mosquitto, today I’ll show you how to send data to an MQTT broker using the enc28j60 ethernet controller.

Libraries

We’re going to use the PubSubClient library by Nick O’Leary to “talk” with our MQTT broker (mosquitto). You can install the library using the Library Manager in the Arduino IDE (Sketch menu – Include Library – Manage Libraries…):

mqtt-arduino01

The library I chose is based on the Ethernet Client API of Arduino, usually available if you use the “official” ethernet chip (W5100). I’ve already blogged about the UIPEthernet library that allows to use sketches and libraries designed for the W5100 chip also with the enc28j60 one. After having installed also this library in the IDE, you’re ready to write your first MQTT sketch…

For this tutorial I decided to connect to the Arduino a temperature (and humidity sensor): the DHT11. I’m going to use the DHT sensor library by Adafruit to read values from that sensor, this library too can be installed through the Library Manager.

mqtt-arduino03

Sketch

The full sketch is available in my Github’s repository.

Let’s see the most important lines of code. After having included the required libraries, the sketch defines some constants, like the topic to send data to, the identifier of the MQTT client, the interval between two transmissions and the configuration of the sensor:

#define CLIENT_ID       "ArduinoMQTT"
#define TOPIC           "temperature"
#define PUBLISH_DELAY   5000
#define DHTPIN          3
#define DHTTYPE         DHT11

then it configures the MAC address of the network interface and the IP address of the MQTT broker (= the IP address of the PC where mosquitto runs on):

uint8_t mac[6] = {0x00,0x01,0x02,0x03,0x04,0x05};
IPAddress mqttServer(192,168,1,4);

the MQTT client is configured passing the Ethernet client to be used and the server (and its port) to connect to:

mqttClient.setClient(ethClient);
mqttClient.setServer(mqttServer, 1883);
Serial.println(F("MQTT client configured"));

in the main loop() the sketch checks is the interval between two transmissions is passed; if so it reads the new temperature value and sends it to the broker:

if(millis() - previousMillis &gt; PUBLISH_DELAY) {
  sendData();
  previousMillis = millis();
}

it’s very important to call the loop() command of the MQTT Client frequently to let it process the different incoming packets and commands:

mqttClient.loop();
every tot seconds, the sketch reads the temperature, displays it on the serial terminal and sends the value to the broker. The library returns a float value, it’s therefore required to convert it in string before sending (using the dtostrf command):
float t = dht.readTemperature();
Serial.print("Temperature: ");
Serial.println(t);
 
if(mqttClient.connect(CLIENT_ID)) {
  mqttClient.publish(TOPIC, dtostrf(t, 6, 2, msgBuffer));
}

Test

Before running the sketch, first you have to start mosquitto on your PC. Open a second command prompt and run mosquitto_sub to subscribe the temperature topic:

mosquitto_sub.exe -t temperature

if you now run the sketch, you should be able to see the temperature values sent to the broker and, from this one, to the subscriber:

mqtt-arduino02