The Bus Pirate project currently has four firmware builds (v3/v4/v5/vNG1) under two toolchains (PIC C/ARM C). To make this more manageable, we use a cheap VPS to check for new code in our git repo and compile the firmware automatically. Fresh compiles are available for everyone immediately, without any intervention from developers or friendly forum members.
We wanted extremely tight integration with BusPirate.com so we rolled our own script instead of using a tool like Jenkins. This post covers:
- Installing MPLABX and PIC XC16 compiler for automated builds
- Installing ARM GCC and libopencm3 for automated builds
- A simple build service to periodically update repos and compile firmware
Is there new code to compile?
Creating builds for a bunch of platforms is boring, let’s automate it! First we find out if there’s new code to compile. With git we can check the log before and after a ‘pull’ command.
#git log --pretty=format:'%H' -n 1
Check the git log for the long hash of the current local repository.
Pull the latest commits and update the local repository.
git log --pretty=format:'%H' -n 1
Check the long hash again. The hash changed! Let’s compile some code!
make clean && make
Compile the latest code and then do something useful with the output.
A build server can definitely run in your basement or at the hackerspace, but a $5/month VPS from Vultr or DigitalOcean is a super slick option. We run Ubuntu 14.04 LTS 64bit because some of libraries we need aren’t available for newest versions of Ubuntu.
Install instructions for both toolchains are in shell script .sh format. You shouldn’t run these! Defaults and version numbers change. Some user interaction is needed. Open a shell terminal and paste the lines one at a time. Notes in the .sh file explain each step.
Setup PIC MPLABX and XC16 compiler for automated builds
PIC firmware can be compiled on a headless Linux server since Microchip released MPLABX and the XC compilers. Installation is a bit tedious, but the toolchain works great. bp-install-mplabx.sh documents our setup. Credit to this solution.
The Bus Pirate MPLABX project needs a little preparation before compiling on the server. There’s three active hardware versions to compile for the PIC-based Bus Pirate. Each hardware version needs a separate MPLABX project, the project configuration passes the hardware version #define via a compiler flag.
Setup ARM GCC compiler and libopencm3 for automated builds
ARM GCC is easier to setup than the PIC toolchain. Install the compiler, pull the source, then compile libopencm3 and the Bus Pirate NG firmware. bp-install-armgcc.sh documents our install.
Install and configure automated build service
buildv2.py runs ‘git pull’ every 10 minutes. If there’s new commits, the source is compiled and the firmware is uploaded to a server for further processing. Follow the steps in bp-install-autobuild.sh to install and configure the build service.
buildv2_tasks.py has settings for the repositories and builds. Each repository can have multiple build tasks. The PIC-based Bus Pirate repo is pulled once, then compiled for three hardware configurations (v3/4/5).
The service can be started from the command line with:
And stopped with:
If the service is installed in /etc/init.d it will start with the operating system. The script has been stable for months at a time, but we play it safe and use monit to restart the service if it crashes.
The firmware and build logs are packaged into a JSON file and uploaded to an API at BusPirate.com. Here’s an example of the JSON output, the important variables are described in the table below.
||0 if make executed, 1 if make returned system error (build failed with errors).
||start/stop build timestamp
||Commit hash before/after ‘git pull‘ command
||Output from the ‘git pull’ command
||Output from the ‘make‘ command
||The file extension of the firmware. Added to automate naming in the backend. PIC uses .HEX, ARM uses .BIN
||Base 64 encoded firmware file
BusPirate.com serves up the fresh build and notifies anyone subscribed to the mailing list. We also grab all the commit notes and change history from GitHub so it’s easy to see what’s in the build. You can check it out here.
For the rest of this week we’ll be testing a new version of Dirty Cables at DirtyCables.com.
Later next week we’ll receive a few hand-assembled Bus Pirate v5 prototypes. We ruined another Bus Pirate v5 prototype by tearing the USB Micro B connector and traces off the PCB. Instead of building another board in-house, we sent the parts to a Chinese PCBA that does two-off hand-assembled prototypes. Let’s see how they turn out!