USB Micro B is the connector of the moment, but we haven’t had a very good time working with it. The bog-standard Chinese generic Micro B connectors on Bus Pirate v5 and Bus Pirate NG1 break constantly. We tested a Micro B connector with through-hole support tabs on the latest Bus Pirate PCBs. It turned out really well.
Standard Chinese USB Micro B connector
A ton of mom and pop factories around Shenzhen churn these out by the bucket load. It seems like a great part: ubiquitous and cheap, doesn’t require a special board slot, versions with extended leads and centering pegs are easier to hand solder. Unfortunately, every connector soldered by every member of our team has eventually decided to exit the PCB.
This connector is optimized for paste stencil and reflow soldering. Most support should come from two solder pads under the connector that are impossible to reach with a soldering iron. Maybe it could be done with hot air, or the QFN “solder from the bottom of the board through a via” hack.
Connectors with supporting through-hole tabs
The generic Chinese connector (left) has six solder pads holding it to the PCB. Four can’t be reached with a soldering iron – the two under the front, and the two recessed pads at the back near the pins.
A Molex 47589-001 (right) with through-hole support tabs went into the next revision of the Bus Pirate boards. The tabs aren’t long enough to go completely through a 1.6mm PCB, but they can still be soldered in place from the top of the board. It’s much more expensive – essentially free vs $0.50 each – but we’re optimizing for hand assembly, not production.
Now we’re living the dream. The four tabs are really secure, and we don’t have to treat the prototype like glass. The next step is to find the most widely available Chinese equivalent.
These Bus Pirate v5 prototypes were hand-assembled by a random PCBA shop on Taobao. Assembly took two days and cost 80RMB (~$12) for each board, we provided the PCBs and components. Normally we relish a prototype build because it’s a source of so many design improvements, but this was an emergency.
The USB Micro B connector on the Bus Pirate v5 and the Bus Pirate NG1 keeps breaking. In fact, the connector on v5 broke while we took pictures for a post. We soldered it on well enough to finish the photos, but we need to replace the trashed board ASAP to continue working on BusPirate.com.
How it worked
We contacted a random PCBA shop on Taobao using QQ messenger. The assembler reviewed the gerber files and quoted 80RMB (~$12) to assemble each piece. That’s just the assembly cost.
PCBs and all the components came from our own suppliers. PCBs came from the Dirty PCB factory, parts came from JLC and a few Toabao suppliers. Sourcing the components for the “kit” took quite a bit of time, maybe the same as soldering a Bus Pirate. After two days we had the PCBs and parts in our Shenzhen office, then it all went to the assembler by same-day courier.
Communication with the assembler
The assembler relied on the BOM and the PCB silkscreen to stuff the board. They didn’t need an image of the schematic nor do they accept Eagle files. All interaction happened over QQ, which is pretty typical for everything in China.
They had a single question about the orientation of a component. The dots marking pin 1 of the BL1551s was on the wrong layer and didn’t get printed on the PCB. We’ll make sure all the orientation markings are clear for a speedier build next time.
The assembler found a problem with the parts we sent too. The 74HCT4066 is in the wrong package, it should be TSSOP instead of SOIC. We ordered a replacement part from JLC and chose SF shipping for 12RMB (~$1.90). The replacement arrived at the factory the next day.
Finished boards arrived in our office two days after the assembler received all the parts, about 5 days after we ordered parts. Soldering is very good, but you wouldn’t mistake it for a board done in a reflow oven. All the components are stuffed in the right place and in the correct orientation. There’s a tiny bit of flux around the LEDs, but otherwise the board is super clean.
Bus Pirates have a row of unpopulated indicator LED footprints on the back of the board named LED1A-LED4A. These are the same as the LEDs on the front (LED1-LED4), and just open up more case options at no additional cost. The BOM we sent to the assembler specified LEDs for LED1-LED4, and made no mention of LED1A-LED4A. The assembler soldered LEDs to both sides of the board.
Our instructions were too ambiguous. The LED silk labels are PWR, USB, MODE, and VREG, not LED1-LED4. There was no way for the assembler to tell which LEDs were supposed to be populated, so they stuffed them all. In the future we’ll make sure the BOM names match the PCB silk, and explicitly state which parts are “do not populate”.
Does it work?
Powered up, programmed a bootloader, flashed the latest v5 firmware freshie build. Every board works and passes the self-test.
Quick turn hand-assembly
It’s super nifty to send away parts and get back assembled boards a few days later. If the orientation of every part is really obvious on the PCB silkscreen then the assembler can probably handle the build without any questions. The boards all work perfectly, and outsourcing the build really did keep the project moving at a critical moment.
Sourcing the parts and kitting the components took a lot of time. It would be a lot easier if the assembler provided common resistors and capacitor values so we don’t have to find them. There’s also a lot of caveats: this was all handled in Chinese, requires Chinese payment methods, and our Shenzhen office was able to coordinated everything.
The Bus Pirate project currently has four firmware builds (v3/v4/v5/vNG1) under two toolchains (PIC C/ARM C). To make this more manageable, we use a cheap VPS to check for new code in our git repo and compile the firmware automatically. Fresh compiles are available for everyone immediately, without any intervention from developers or friendly forum members.
We wanted extremely tight integration with BusPirate.com so we rolled our own script instead of using a tool like Jenkins. This post covers:
- Installing MPLABX and PIC XC16 compiler for automated builds
- Installing ARM GCC and libopencm3 for automated builds
- A simple build service to periodically update repos and compile firmware
Is there new code to compile?
Creating builds for a bunch of platforms is boring, let’s automate it! First we find out if there’s new code to compile. With git we can check the log before and after a ‘pull’ command.
#git log --pretty=format:'%H' -n 1 cba17f0d1cf0f7a2b5e8ad5954390feae08b6ba3
Check the git log for the long hash of the current local repository.
Pull the latest commits and update the local repository.
git log --pretty=format:'%H' -n 1 42f05e5fe033f37d029a0483c967a01a81222c7c
Check the long hash again. The hash changed! Let’s compile some code!
make clean && make
Compile the latest code and then do something useful with the output.
A build server can definitely run in your basement or at the hackerspace, but a $5/month VPS from Vultr or DigitalOcean is a super slick option. We run Ubuntu 14.04 LTS 64bit because some of libraries we need aren’t available for newest versions of Ubuntu.
Install instructions for both toolchains are in shell script .sh format. You shouldn’t run these! Defaults and version numbers change. Some user interaction is needed. Open a shell terminal and paste the lines one at a time. Notes in the .sh file explain each step.
Setup PIC MPLABX and XC16 compiler for automated builds
PIC firmware can be compiled on a headless Linux server since Microchip released MPLABX and the XC compilers. Installation is a bit tedious, but the toolchain works great. bp-install-mplabx.sh documents our setup. Credit to this solution.
The Bus Pirate MPLABX project needs a little preparation before compiling on the server. There’s three active hardware versions to compile for the PIC-based Bus Pirate. Each hardware version needs a separate MPLABX project, the project configuration passes the hardware version #define via a compiler flag.
Setup ARM GCC compiler and libopencm3 for automated builds
ARM GCC is easier to setup than the PIC toolchain. Install the compiler, pull the source, then compile libopencm3 and the Bus Pirate NG firmware. bp-install-armgcc.sh documents our install.
Install and configure automated build service
buildv2.py runs ‘git pull’ every 10 minutes. If there’s new commits, the source is compiled and the firmware is uploaded to a server for further processing. Follow the steps in bp-install-autobuild.sh to install and configure the build service.
buildv2_tasks.py has settings for the repositories and builds. Each repository can have multiple build tasks. The PIC-based Bus Pirate repo is pulled once, then compiled for three hardware configurations (v3/4/5).
The service can be started from the command line with:
And stopped with:
If the service is installed in /etc/init.d it will start with the operating system. The script has been stable for months at a time, but we play it safe and use monit to restart the service if it crashes.
The firmware and build logs are packaged into a JSON file and uploaded to an API at BusPirate.com. Here’s an example of the JSON output, the important variables are described in the table below.
|‘error’||0 if make executed, 1 if make returned system error (build failed with errors).|
|‘timestamp’||start/stop build timestamp|
|‘starthashlong’/’endhashlong’||Commit hash before/after ‘git pull‘ command|
|‘gitoutput’||Output from the ‘git pull’ command|
|‘makeoutput’||Output from the ‘make‘ command|
|‘firmware_type’||The file extension of the firmware. Added to automate naming in the backend. PIC uses .HEX, ARM uses .BIN|
|‘base64encbin’||Base 64 encoded firmware file|
BusPirate.com serves up the fresh build and notifies anyone subscribed to the mailing list. We also grab all the commit notes and change history from GitHub so it’s easy to see what’s in the build. You can check it out here.
Later next week we’ll receive a few hand-assembled Bus Pirate v5 prototypes. We ruined another Bus Pirate v5 prototype by tearing the USB Micro B connector and traces off the PCB. Instead of building another board in-house, we sent the parts to a Chinese PCBA that does two-off hand-assembled prototypes. Let’s see how they turn out!
Up-to-date documentation makes a project easy to learn about, but its a really boring job that takes a lot of time. Even great documentation eventually has outdated examples and screenshots that don’t quite match the latest version.
BusPirate.com has a hacked together toolchain to keep the documentation fresh.
It’s a three part process:
- Test scripts run demos on actual Bus Pirate hardware
- Results files are uploaded to BusPirate.com
- Tutorial templates and the results files are merged to make updated docs
Hardware and firmware specific docs
Bus Pirate documentation is full of version caveats like “Hardware v4+ only” and “Firmware v5.1+”. BusPirate.com docs are versioned, and just shows the stuff that matters for your version. Choose a hardware and firmware from the menu to see the docs for that specific combination.
Developers still need to add new feature information manually, but the huge task of “refreshing” everything after a firmware release can be automated. We took a three step approach: test scripts that run commands on actual Bus Pirate hardware, results files that capture the test output, and templates that merge with result files to create version-specific documentation.
Test scripts are just a list of commands to run on the Bus Pirate, for example running a self-test. A test has multiple steps, and each step has one or more Bus Pirate commands. We build the demos with a simple editor, then dump them to JSON in a file. Here’s the test script that runs all the commands shown in the reference manual.
pipe.py sends commands in the test script to a Bus Pirate. It also records the terminal output to a results file. Result files are uploaded to BusPirate.com. Here’s the results file from the reference manual test script.
Demos and docs are blade templates. A template merges with a results file to show the tutorial exactly as it appears on a hardware and firmware combination. It’s not super elegant, the version specific stuff is done with a bunch of PHP if statements.
Taking it further
Ideally the update process will be triggered by an automated firmware release. An RPi in the workshop will bootload the new firmware into the Bus Pirate, run the test scripts, and upload the results without any intervention.
A test rig with a bunch of devices might be a cool way to do release testing. Scripts could test various firmware features on real devices. Comparing the scripted test results with a previous release would highlight things that may be broken. Our goal is to hack together something that does comprehensive release testing with no manual effort.
Thursday we’ll document the automated build server that’s kicking out freshies every time there’s new code in the git repo.
Bus Pirate documentation and demos are all buried in a huge wiki and around the blog, this is less than ideal for such an expansive project. We’ve been working on a new site just for Bus Pirate stuff.
This site is full of hacks that automate boring development stuff.
- Documentation and tutorials are updated through test scripts and a templating system
- The latest code in the git repo is compiled continuously and always ready to download
- Firmware releases are packaged and announced automatically
There’s a broad overview of the site features below, we’ll follow up with detailed posts later this week.
Big obvious menu options
It’s a simple thing, but it makes a big difference. Obvious menu options dedicated to the Bus Pirate are a nicer experience than digging through a wiki with stuff about dozens of (partially) completed prototypes.
Versioned tutorials you can download in PDF
Bus Pirate documentation is full of version caveats like “Hardware v4+ only” and “Firmware v5.1+”. BusPirate.com docs are versioned, they only show info that applies to a specific hardware and firmware combination.
Docs can be downloaded as a PDF. We tried to use PDF specific stylesheets so the docs look as good printed as they do in a browser. wkhtmltopdf handles the conversion to PDF, it’s by far the best conversion tool we’ve worked with.
Automated firmware builds and release packages
An automated build server compiles new firmware every time there’s a commit to the git repo. This is a huge improvement for everyone. Users get the latest bug fixes without compiling from source, and developers don’t have to make interim releases for a bunch of platforms.
Currently the build server compiles firmware for four hardware versions (v3/4/5/NG1) under two different compilers (MPLABX/XC16, ARM GCC). We’ll document the build server on Thursday.
Firmware releases are packaged and announced automatically. The release is a tidy archive with the latest firmware, update tools, readme.txt and updated history.txt.
Notification of new tutorials, releases, and firmware builds
Subscribe to be notified of new developments. Firmware builds and releases mailing lists can be limited to a specific hardware version to cut down on noise.
Taking it further
Later today we’ll post about the system to automate updates of the Bus Pirate tutorials and documentation. Thursday we’ll show how we setup an update automated build server to compile PIC and ARM firmware on a $5/month virtual private server.
Sjaak, fresh off his ARM adventures at SMDprutser, hacked together an STM32-based Bus Pirate prototype we’ve been calling NG1 (Next Gen version 1). NG1 uses a modern ARM chip with completely open tools and libraries, and sports an integrated logic analyzer . The firmware is very usable, but largely untested. Let’s have a look at the biggest changes.
STM32 ARM-based design
NG1 is built around an STM32F103CBT6, a modern ARM chip with 128K flash and 20K RAM running at 72MIPS.
All the extra resources let us use a proper printf function for terminal display, instead of the crappy obscure way of storing strings we invented for the v3.x hardware. The current firmware uses about half of the flash (78 of 128K). If we fill the chip, like we did with the Bus Pirate v3.x, STM32 has a full line of compatible chips with up to 512K of flash.
ARM has completely open toolchains, while the Microchip PIC used in previous Bus Pirates is still straggling with free-but-not-open-not-distributable–and-crippled compilers and libraries. The ARM GCC compiler is open source and freely available for multiple operating systems. Several active open source libraries support the chip’s features, such as multiple USB endpoints. NG1 uses an open source peripheral library, libopencm3, so the complete source and libraries can be redistributed under the GPL.
Like most chips, STM32 doesn’t have a Peripheral Pin Select feature. PPS on the PIC24FJ64GA002 is what makes the original Bus Pirate v3.x design so elegant (left layout). Most major hardware modules like UART, SPI, timers, PWM, etc can be assigned to almost any pin using PPS – just one PIC pin is needed for each of the 5 Bus Pirate IO connections.
STM32 hardware modules are fixed to specific pins. Each Bus Pirate IO pin is connected to up to five STM32 pins to provide the necessary features. This kind of design uses a lot more pins (28 vs 64), and the PCB is a lot more complicated (right layout).
On-board Logic Analyzer
Frequently we use the Bus Pirate along side a logic analyzer like the Logic Pirate. NG1 has an SRAM-based logic analyzer directly on-board. The logic analyzer records bus activity every time the Bus Pirate writes or reads commands.
This adds an additional layer of debugging when things go wrong, without any extra cables or connections. Talk to a chip through the terminal, then check the logic analyzer to verify what actually happened on the bus. Terminal mode and the logic analyzer work at the same time because they each have a dedicated USB endpoint in the STM32. Microchip 23LC1024 SRAMs provide 256K samples per channel at 20MSPS on the current prototype, but we’ve found an alternate SRAM chip with over 10M samples per channel.
Currently NG1 uses a 74LVC573 single direction latch between the logic analyzer and bus. The next revision uses a 74LVC245 bidirectional latch[link] so the logic analyzer can also work as a signal generator without any additional hardware. Maybe we could toss on a resistor ladder to make an analog signal generator too.
So far we’ve focused on capturing bus activity while the Bus Pirate is writing and reading commands, but the hardware should work as a stand-alone logic analyzer as well. The high latency of STM32 pin interrupts could contribute to really sloppy capture triggers, especially at high capture speeds, but there’s still a lot to learn about this chip.
Selectable 3.3volt, 5volt, external pull-up voltage
The Bus Pirate has on-board pull-up resistors (10K) to use with 1Wire, I2C, and any other situation where an open drain bus is used. The on-board pull-up resistors can be connected to one of three voltage sources straight from the Bus Pirate terminal. Select from the on-board 3.3volt or 5volt power supplies, or use an external supply through the Vpull-up pin. Selecting the pull-up voltage from the terminal is really convenient new feature.
Two BL1551 analog switches are chained together so that three input sources (3.3volt, 5volt, Vpu pin) are controlled by two pins. If this seems familiar, it’s the same switch hardware we tested on the v3.x update.
Integrated USB and USB Micro B connector
STM32F103CBT6 has integrated USB hardware with actively developed open source driver libraries. Not only is the USB interface faster than a USB-to-serial converter chip, it supports multiple USB endpoints at the same time. Currently we’ve implemented one “low speed” USB endpoint for the terminal and DFU bootloader updates, and a second “high speed” USB endpoint for the on-board logic analyzer.
USB Micro B is by far the most common USB connector at the moment. NG1 uses a Micro B connector so it works with the phone cable that’s probably already on your desk.
However, we don’t like the Micro B connector as much as the Mini B. Every hand-soldered USB Micro B connector eventually broke off of the PCB. After re-soldering it numerous times the pads and tracks get to the point of no repair.
On the most recent hardware revision we removed the cheap “standard” Chinese Micro B connector and used a much more expensive Molex 47589-001 USB connector with through-hole tabs for reinforcement.
1x10pin keyed locking connector
Previous Bus Pirates use a 2x5pin IDC connector. These are easy to use with 0.1” jumper wires from your parts box, but there’s not many options for probe cables using an ICD connector.
NG1 uses a 1x10pin 2543/TJC8S-10AW (equivalent to Molex 70553-0044)connector, a common 0.1” header inside a keyed/locking shroud. The pins are 0.1” pitch, so jumper wires still work great, but now we can make high quality keyed/locking probe cables with tangle-free silicone wire.
The NG1 has a hardware reset button and a user configurable button. The user configurable button is currently only used for activating the bootloader during power-up. We haven’t decided yet on a good secondary purpose for it, but we are open for suggestions.
A hardware UART is exposed for debugging without using USB. The code supports debugging via the UART, though we started debugging through the second USB CDC port after USB was up and running.
Taking it further
A logic analyzer, selectable pull-up resistor voltage supply, more memory and a totally open toolchain are probably the four biggest improvements to Bus Pirate NG1. The next revision will also support a hardware signal generator.
Hardware files are available here. The hardware is not final, but the current version addresses all known bugs. Quick order the latest PCB version at DirtyPCBs, or wait a week and we’ll have some to give away for free.
Firmware repository is here, it builds with the ARM GCC compiler. Use the command ‘git clone –recursive https://github.com/DangerousPrototypes/bus_pirate_ng‘ to clone our git repository. That command should also take care of getting the right libopencm3 version. Be sure to run make in the ‘/libopencm3/‘ folder first, then switch to the ‘/source/‘ folder and run make again to built the firmware. We’ll walk through the toolchain setup next week.
Brandon Vandegrift @bmv437 tweeted, “Got my @defcon badge working again! I must have shorted out either R9 or R10, which are pull-up resistors for the I2C data lines. Without that, the PIC32 can’t communicate with the LED driver board. @dangerousproto Bus Pirate to the rescue, with it’s built in pull-up resistors!”
Development on DirtyPCBs.com is winding down, so I’ve had some time to play with hardware. I’ve said the same thing for a few years now, but this time it really happened!
This update of Bus Pirate v3.x crams in a major new feature, and slightly lowers the total cost. Two China-sourced analog switches enable pull-up resistor voltage selection – 3.3volts, 5volts, or External – directly from the terminal menu. A new IO header is compatible with fancy tangle-free silicone wire probe cables. An updated USB to serial converter chip reduces the BOM to offset the cost of the new features.
Pull-up voltage select 3.3v/5v/Vpu pin
Bus Pirate v3.x has on-board pull-up resistors for 1Wire, I2C, and any other situation where an open drain bus is used. Currently the pull-up resistors are fed through the Vpull-up pin (Vpu). In almost all cases I use a wire to connect the Vpu pin to the on-board 3.3volt or 5volt power supply.
It would be so much more convenient to select one of the on-board power supplies from the Bus Pirate menu, instead of connecting an extra wire to the Vpu pin. The v4 hardware made an attempt at this, but with a circuit that creates a lot of voltage drop.
I spotted the BL1551 analog switch while browsing Chinese chip datasheets at JLC. BL1551 is a $0.04 analog switch with low on-resistance (2.7ohm at 5.0volts) and high current capacity. It seemed like a good candidate for switching the pull-up resistor source.
Two BL1551 are chained together so that three input sources (3.3volt, 5volt, Vpu pin) are controlled by two pins. All the PIC microcontroller pins are already used, but a little hack lets the hardware version ID pins drive the BL1551.
1 x 10pin keyed locking connector, corrected pinout
Bus Pirate v3.x has always used a 2x5pin IDC connector. These are super common and easy to use with 0.1” jumper wires from the parts box. Unfortunately there aren’t many good cable options for an IDC connector, a probe cable made from ribbon wire always feels cheap. I rolled a few versions with various JST connectors, but a custom cable makes everything less handy.
Eventually I settled on a 1x10pin 2543/TJC8S-10AW connector (equivalent to Molex 70553-0044), a common 0.1” pin header inside a keyed/locking shroud. Jumper wires still work great because the pins are 0.1” pitch, but now we can make high quality keyed/locking probe cables with tangle-free silicone wire.
The pinout on the new connector is corrected to MOSI-CLOCK-MISO-AUX-ADC-Vext-3.3V-5V-GND. The original v3.x pinout was mangled in early revisions, and the current mess has been grandfathered-in since the first production run.
FT230X USB to serial converter
The FT232RL has been the go-to USB-to-serial converter chip for a decade (IC2, left). It’s used on the Bus Pirate v3.x, as well as oh-so-many Arduinos. FT230X is a new version that uses the same FTDI drivers everyone already has installed, but it’s half the price and comes in a smaller package (IC4, right).
Moving to the FT230X frees up board space for some PCB tweaks, and reduces the BOM cost by about $2 in single quantity.
USB Micro B
What kind of USB cables do you have laying around these days? I’ve got a ton of USB Micro B cables from phones and phone chargers, and those are slowly being replaced by USB C cables. The Mini B connector on v3.x is a relic.
My updated board uses a USB Micro B connector. It’s not hand-solder-hobby-friendly like the USB Mini B connector. To be completely honest, every hand-soldered prototype connector eventually broke off the board, often lifting traces with it. There are two versions of this hardware in git – one with a generic Chinese USB connector, and one using an expensive Molex connector with through-hole reinforcement.
5cmx5cm PCB size
Swapping the IO connector and USB chip made it possible to fit everything on a 5cmx5cm PCB. This version fits on the super cheap 5x5cm PCB prototype packs sold at most board houses.
Taking it further
This Bus Pirate has been on my bench for about six months, and it’s hard to go back to an older version. Selecting the pull-up voltage from the terminal is really convenient. The IO header pinout makes a lot more sense, and the cable options are pretty sweet. The Micro B connector may be the best update, now the Bus Pirate works with the phone cables I always have around.
The Eagle schematic and PCB files are in git. The hardware is significantly different from v3.x and needs a custom firmware build, so I called it v5. The version isn’t set in stone. There is also a firmware branch with support for the new hardware.
There are no plans to produce this version of the board without lots more testing and community feedback. PCBs for the Molex USB version should be available in the free PCB drawer in a few days.