ESP32 (28) – MQTT and SSL

Security is a very important aspect for MQTT brokers. In a previous article you’ve already learned how to implement authentication and authorization. The weakness in that configuration was that credentials were transmitted in cleartext; it was therefore possible, for an attacker who can sniff the network traffic, to read and use them to impersonate a legitimate client.

Today I’ll show you how to encrypt the communication channel between client and broker using SSL certificates. I’ll also explain how to write a program for the esp32 chip to send data to the broker using the secure channel…

SSL certificate

To be able to encrypt the communication, mosquitto requires a server certificate.

First generate the private key (RSA with a length at least 2048 bits):

openssl genrsa -out mosquitto.key 2048

Then create the CSR file:

openssl req -new -out mosquitto.csr -key mosquitto.key

type the required information; the most important of which is the common name that will identify the server:

mq-ssl-001

Now sign the CSR file with your Certificate Authority (or send it to a public / corporate CA) to generate the certificate:

openssl ca -config openssl.cnf -extensions server_cert 
 -notext -in mosquitto.csr -out mosquitto.cer

mq-ssl-002

Create the ssl subfolder in the folder where you installed mosquitto and copy into that folder the certificate, its private key and the certificate of the CA:

mq-ssl-003

Configuration

Open the mosquitto.conf file and add the following lines:

mq-ssl-004

The first line changes the TCP port mosquitto is normally listening to (1883) to the default port for SSL connection, 8883.

The following 3 lines set the path for server and CA certificates and for the private key that corresponds to the server certificate. The last one, which is not compulsory, forces the use of the TLS v1.2 protocol, the most secure one at the moment of writing.

Once the server has been configured, you can start it (-v is to enable the verbose output):

mosquitto.exe -c mosquitto.conf -v

To be able to use the mosquitto_pub and mosquitto_sub tools, you now have to add new parameters:

mosquitto_sub.exe -p 8883 -t test --cafile .\ssl\ca.cer --insecure
mosquitto_pub.exe -p 8883 -m 20 -t test --cafile .\ssl\ca.cer --insecure

With -p you specify the TCP port of the server, with –cafile the path of the CA certificate which signed the server certificate mosquitto uses and finally with –insecure you configure the two clients not to verify that the certificate’s common name (in my example mymosquitto.local) corresponds to the server name.

You can avoid using the –insecure switch if you generate a certificate with a common name the exact name of the server/PC which runs mosquitto or – alternatively – if you add a DNS alias which resolves the common name of your certificate with the IP address of the server and run the clients with -h commonName

esp32

Tuan PM developed a library (espmqtt) for the esp-idf framework that implements a complete MQTT client. Moreover, the library does support secure connections, you can therefore use it to connect to an MQTT broker with TLS enabled.

mq-ssl-005

Copy the content of the Github repository in the components folder of your project and include the library’s header file in your source code:

#include "mqtt.h"

The MQTT client is configured using the mqtt_settings struct:

mqtt-001

The most important parameters are:

  • the server (host) that runs the MQTT broker (you can use the IP address or the DNS name)
  • the TCP port (port) the server is listening to (default is 1883 or 8883 if SSL is enabled)
  • username and password if the server requires authentication
  • one or more callback functions the espmqtt library will call when the corresponding event occurs
the espmqtt library does not copy the parameters in an internal struct. It’s therefore very important that the mqtt_settings variable has a global scope and it’s defined outside a specific function.

Your program can interact with the MQTT client implementing its callback functions.

For example the connection or disconnection from the MQTT server occurs as follows:

mqtt-002

The connect_cb and disconnect_cb functions perform the “real” connection and disconnection, while connected_cb and disconnected_cb functions are executed after the corresponding activity is completed (= the client successfully connected to the server). Your program usually doesn’t need to re-implement the main functions, but will implement the ones related to events, to execute actions (for example subscribe a topic) when a specific event occurs.

After having configured the client, you can run it with:

mqtt_start(&settings);

Once connected to the server (connected_cb callback function) you can subscribe or unsubscribe a topic with:

void mqtt_subscribe(mqtt_client *client, const char *topic, uint8_t qos);
void mqtt_unsubscribe(mqtt_client *client, const char *topic);

and publish data to a topic with:

void mqtt_publish(mqtt_client* client, const char *topic, 
  const char *data, int len, int qos, int retain);

Demo

I prepared an example to show my esp32 devboard sending data to a mosquitto server, with SSL enabled.

I connected to the devboard an HTU21D sensor as explained in a previous article and my program reads, every 5 seconds, the temperature and humidity values and sends them to the broker. I used a very handy opensource program, HelloIoT, to create a dashboard and display the received data.

The source code of the program and the configuration of the HelloIoT dashboard are available in my Github repository; here’s a short video of the demo:

Java, IllegalFormatConversionException when importing an SSL certificate

Today I faced a strange problem. I needed to import an SSL certificate in a java keystore, using the “classic” keytool command. I was sure that the format of the certificate was correct, but I always received the following error:

java.util.IllegalFormatConversionException: d != java.lang.String

java-keystore01

The reason of the error is that – in the latest versions of java – by default the keytool command uses the language of the system (in my case italian).

I had to add the parameter -J-Duser.language=en to the keytool command to made it work correctly:

java-keystore02

ESP32 (21) – Mutual authentication

After having published my post about how to implement a webserver on the esp32 chip, some readers correctly warned me that everyone, once connected to the wifi network, could control the relay and asked me how to limit access to the webpage.

A classic solution (that I also used in this previous tutorial) is to ask the user to enter a password. It’s very easy to implement, but has a negative aspect, that is you have to type the password everytime you need to switch the relay.

You can avoid asking the password everytime the user access the page if you implement a login page and keep the user logged in using a session cookie as I’ll explain in a future tutorial!

Today I’ll show you how to protect your website using a feature of the SSL/TLS protocol: the mutual authentication.

Mutual authentication

Everytime you connect to a website using the https protocol, the server which hosts the site sends to your browser the SSL certificate of the site. Thanks to this certificate, you can verify the identity of the server and establish a protected connection to it:

ssl-02

The TLS protocol also offers the possibility for the server to require a certificate from the client: this means a two-way authentication, that is server and client authenticates each other using SSL certificates:

ssl-01

With this authentication method, a user has only to install on his device the client certificate (and the corresponding private key) and choose which certificate to send to the server if more than one is present:

ssl-03

The authentication with SSL certificates is therefore based on the principle of something I own (for example, a key) and not of something I know (for example, a password).

Certificates

Generally speaking, SSL certificates can be:

  • self-signed
  • signed by a Certification Authority

The first ones are perfect for internal use or for testing, while the second ones, normally signed by trusted CAs, are widely used in production environments or on the Internet. Sometimes you probably received this warning message when surfing on the Internet:

ssl-04

that means that your browser doesn’t trust the certificate sent by the server because it’s self-signed or signed by a non trusted CA.

For this tutorial, I’m going to use OpenSSL as a certification authority to create the certificates we need. If you’re using the toolchain provided by Espressif, you don’t need to install anything else because OpenSSL is included.

First create a folder that will contains all the files required by OpenSSL to operate as a CA in your home directory:

cd 
mkdir myCA

Now move into that folder and create some empty files:

cd myCA
mkdir csr certs crl newcerts private
touch index.txt
echo 1000 > serial

sslca-01

Copy the certification authority‘s configuration file (openssl.cnf) from my Github repository into the myCA folder. Open the file and change the main path accordingly to your system:

sslca-02

Now you have to generate the private key and the certificate for the CA. Let’s start with the key (its length will be 2048bit, enough to ensure good security):

winpty openssl genrsa -aes256 -out private/ca.key 2048

(all the commands must start with winpty only if you’re using Windows!)

You’re prompted for a password; it’s very important to remember it because you’ll have to type it everytime you use the CA:

sslca-03

Now generate the CA certificate (with a duration of 3650 days, that is 10 years):

winpty openssl req -config openssl.cnf -key private/ca.key -new -x509 -days 3650 -sha256 -extensions ca_cert -out certs/ca.cer

You have to enter some informations… the only mandatory is the common name which is the name of the CA, that will be included in all the certificates the CA will sign. At the end of the process you can open the file (ca.cer) and verify name and duration:

sslca-04

Let’s now generate the server and client certificates. The steps are the same for both:

  • generate a new private key – openssl genrsa
  • generate the CSR (Certificate Signing Request) file – openssl req
  • sign the CSR file with the CA to obtain the final certificate – openssl ca

Server certificate:

winpty openssl genrsa -out private/esp-server.key 2048
winpty openssl req -config openssl.cnf -key private/esp-server.key -new -sha256 -out csr/esp-server.csr
winpty openssl ca -config openssl.cnf -extensions server_cert -days 365 -notext -md sha256 -in csr/esp-server.csr -out certs/esp-server.cer

Client certificate:

winpty openssl genrsa -out private/esp-user.key 2048
winpty openssl req -config openssl.cnf -key private/esp-user.key -new -sha256 -out csr/esp-user.csr
winpty openssl ca -config openssl.cnf -extensions usr_cert -days 365 -notext -md sha256 -in csr/esp-user.csr -out certs/esp-user.cer

As said before, the client device must own both the certificate and the private key. You can merge certificate and key in a single p12 (or pfx) file with the following command:

winpty openssl pkcs12 -export -out esp-user.pfx -inkey private/esp-user.key -in certs/esp-user.cer

The way you install the P12 file on your device depends on the specific operating system it runs: on Windows you only need to double-click the file and follow the import wizard.

SSL webserver

The source code for the program running on the esp32 devboard is available in my Github repository.

First you have to copy in the prioject folder the certificates (CA and server ones) and the private key for the server certificate and include them – as text data – in the program (I’ve already explained how to include binary files in my previous post):

esp-ssl01

To implement the SSL protocol I’m using the mbedTLS library, included in the esp framework.

First include all the required header files:

#include "mbedtls/platform.h"
#include "mbedtls/net.h"
#include "mbedtls/ssl.h"
#include "mbedtls/entropy.h"
#include "mbedtls/ctr_drbg.h"
#include "mbedtls/debug.h"
#include "mbedtls/error.h"

Before being able to accept SSL connections, you have to define and initialize all the variables you’re going to use:

// mbed TLS variables
mbedtls_ssl_config conf;
mbedtls_ssl_context ssl;
mbedtls_net_context listen_fd, client_fd;
mbedtls_entropy_context entropy;
mbedtls_ctr_drbg_context ctr_drbg;
mbedtls_x509_crt srvcert;
mbedtls_x509_crt cachain;
mbedtls_pk_context pkey;
 
[...]
 
// initialize mbedTLS components
mbedtls_net_init(&listen_fd);
mbedtls_net_init(&client_fd);
mbedtls_ssl_config_init(&conf);
mbedtls_ctr_drbg_init(&ctr_drbg);
mbedtls_entropy_init(&entropy);	
mbedtls_x509_crt_init(&srvcert);
mbedtls_x509_crt_init(&cachain);
mbedtls_pk_init(&pkey);

Now parse the certificates and the private key and store them in the variables:

mbedtls_x509_crt_parse(&cachain, ca_cer_start, ca_cer_end - ca_cer_start);
mbedtls_x509_crt_parse(&srvcert, espserver_cer_start, espserver_cer_end - espserver_cer_start);
mbedtls_pk_parse_key(&pkey, espserver_key_start, espserver_key_end - espserver_key_start, NULL, 0);

Tell the library which variables to use for CA chain and own certificate and require the mutual authentication (MBEDTLS_SSL_VERIFY_REQUIRED):

mbedtls_ssl_conf_ca_chain(&conf, &cachain, NULL);
mbedtls_ssl_conf_own_cert(&conf, &srvcert, &pkey);
mbedtls_ssl_conf_authmode(&conf, MBEDTLS_SSL_VERIFY_REQUIRED);

After having configured other elements of the library (the random number generator and the debug function) you can bind it to the standard TCP port for the https protocol (443) and accept new incoming connections:

mbedtls_net_bind(&listen_fd, NULL, "443", MBEDTLS_NET_PROTO_TCP);
mbedtls_net_accept(&listen_fd, &client_fd, NULL, 0, NULL);

After having accepted a connection, you only have to call the function:

mbedtls_ssl_handshake(&ssl);

during the handshake process the library sends to the client its own certificate and require the one from the client, verifying that this is signed by the specified CA: the mutual authentication is implemented!

After the handshake, the program is similar to the previous example, you only have to use mbedTLS specific send/receive functions (mbedtls_ssl_read e mbedtls_ssl_write).

Test

Google Chrome, how to display the website’s SSL certificate

Users of the Google Chrome web browser were able to display the SSL certificate of a website published via https protocol with a click on the address bar. In the latest versions, this functionality has been removed:

chrome-ssl01

You now need some additional steps: first open the developer tools (CTRL-Shift-I):

chrome-ssl02

then select the Security tab and click on View certificate:

chrome-ssl03

And finally you can see the certificate used by the web site (in this example, www.google.com):

chrome-ssl04

Chris Conlon: Device Security 101

We all wring our hands over the security (or lack thereof!) of our myriad smart devices. If you haven’t had your home network hacked through your toaster, or baby cam, you’re missing out on the zeitgeist. But it doesn’t have to be this way — smart devices can be designed with security in mind, and [Chris Conlon] came to Pasadena to give us a talk on the basics.

He starts off the talk with three broad conceptual realms of data security: data in transit, data at rest on the device, and the firmware and how it’s updated. A common thread underlying all of this is cryptography, and he devotes the last section of his  talk to getting that right. So if you’d like a whirlwind tour of device security, watch on!

Data in Transit

Your smart device probably uses the Internet for some service or another. Your data, as it moves through the network, is vulnerable. You might want to keep it private and unmodified along the way, and also be sure that it travels to the right destination. Here, you’ve got basically two choices: either implement TLS/SSL at the session layer, or secure your connections deeper down at the network layer with IPSec.

handshakeYou’ve probably heard of TLS/SSL. It’s what keeps your web surfing safe when you type https:// into your browser. TLS (transport layer security) is actually made up of four components. One takes care of the initial connection — the Handshake — and the client and server agree on what encryption algorithm they’re going to use for the connection. This is the place where authentication happens: your client is sure that the server is who you think it is, and this depends on the X.509 certificate chain. Once the encryption is set up, that keeps your data private and unmodified. This all seems to work nearly effortlessly in the browser on your laptop.

But from an embedded device perspective, TLS is difficult. It involves cryptographic algorithms which can tax your microprocessor’s CPU, demand too much memory, and generally bog your device down. To help, you can choose a crypto algorithm that uses less RAM or ROM, depending on what you’re short of. You can buy devices with built-in hardware crypto peripherals to offload the CPU work. If you’re thinking of running an RTOS, look for TLS support before you get too deeply into your project.

Data at Rest

image17Data stored on the device itself is also vulnerable. If it has sensitive information on it, it should be at least encrypted, and if it’s possible to not store the information at all, that avoids the problem entirely. Presuming your firmware uses encryption, keeping the encryption keys secret is an important issue. Generating the keys locally on the device prevents them from being snooped in transit. [Chris] additionally suggests not re-using keys for different purposes, and changing them often.

But how does the device store the keys? Ideally your chip has a secure storage element: a TPM or HSM where the keys can be stored. If you don’t have any secure storage, you can always require the user to take some action to decrypt the keys, or zero them out in case tampering is detected.

Firmware Upgrades

If you’ve ever read our coverage of device hacking, you already know that the firmware upgrade procedure is rife with opportunities for mischief. Your device needs to be able to be remotely updated, but it also needs to be able to trust the source of the firmware, it needs to be encrypted in transit, and it needs to be sure that the new firmware arrived intact.

[Chris]’s method leans heavily on public-key encryption to get the job done. As long as the device is TLS-capable, it can use any transport mechanism in the middle. [Chris] suggests that it would be possible to use MQTT, secured by TLS, to handle the distribution of the firmware.

Cryptography

If you’ve been paying attention, you’ve noticed that all of the above relies heavily on keeping things secret, and that means thinking hard about the underlying cryptography. And crypto depends heavily on random numbers, which isn’t very easy on small, deterministic hardware devices.

keysizesThings you might think are random, but aren’t, include clocks and serial numbers. What you really want is a true random number generator — something that gets its randomness from the environment. If your device doesn’t have enough entropy coming in, you can seed a pseudorandom number generator periodically with whatever entropy you’ve got, and hope that’s good enough. Many devices these days have hardware random number generators in the silicon, which takes care of that.

Next, you need to make sure that you’re using a long enough key for the encryption, and this depends on both the algorithm you’re using and the actual number of bits in the key. The choice of algorithm is influenced its speed and memory requirements and your microcontroller’s resources. If the chip you’re using has a dedicated SHA-2 hardware peripheral, for instance, you’re probably going to want to use that, and then you can pick the appropriate key length accordingly. Then you know how much entropy you need, and can figure out how to get it.

Summary

[Chris] points out that this talk is a big overview. If you’re actually tasked with designing a secure hardware system, you’re going to need to dig in to each and every one of these points in real detail. Nonetheless, he’s got a summary slide packed with good advice, and it’ll give you a good footing to start your own learning process.


Filed under: cons, Hackaday Columns, security hacks