Unit Test Tower - Parallel Arduino Testing Supercomputer

i.MX RT1010 EVK Based Unit Test Tower

The Arduino UNO is programmed and debugged with a Serial Port over USB at 115Kbaud. But what if you want to program and control 50 Arduinos at the same time! This was the challenge I put to the i.MX RT1010.

The i.MX RT1010 EVK is used to host up to 10 Test Shields each with 5 VBBMicro sandbox's to host up 50 Virtual Arduino UNOs in total. Each Virtual Arduino connects via it's standard UART at 115200 baud..

Handling the resulting datastream of 50 x 115200 = 5.7Mbit/Sec was the task at hand. The resulting solution is a USB:SPI Router created by merging several libraries and examples provided in MCUXpresso SDK and toolkit.

The i.MX RT1010 EVK features used

• High Speed USB
• SPI Master
• Digital IO
• Arduino Shield Breakout

Background

Why would I want to program and control 50 Arduino's? So I can run 50 Unit Test's in parallel of course!

Unit Testing for Microcontrollers

Unit Testing is at the heart of almost all serious software development with most major software tools like Visual Studio and Eclipse featuring integrated unit testing support.

Integrated circuits developers' work with a type of unit testing called Boundary Scanning which is the idea of creating test cases to scan the outside pins and voltages to verify a chip. To achieve this you need an external hardware probe and test logic. In the Integrated Circuit world this is commonly associated with JTAG.

But what about microcontroller circuits and firmware?

VBBMicro for Arduino

The VBBMicro takes Boundary Scanning to the microcontroller level by embedding a type of smart boundary scanning "test probe" directly on a target chip in a pin-compatible format to the host chip.

The test enabled chip can then be used in-circuit to inject tests and component virtualizations driven by the Virtual Breadboard circuit design. It's the MATRIX.. only smaller.

VBBMicro is a micro-MATRIX sandbox!

Unit Testing

The latest version of Virtual Breadboard (1.30 or later) supports a Unit Test framework based on JUnit concepts which works with the VBBMicro Avatar.

Unit Testing with VBB deserves it own Tutorial project series but the key point for this project is this:

Unit Tests run on the VBBMicros connected via a VCOM driver at 115200.

Hence scaling testing can be achieved by parallelising connections. Minutes of testing become seconds making Test Driven Development possible.

i.MX RT1010 MCUXpresso Development

Most of my own work involves low power Microchip 8-bit devices but I really needed something with more firepower to get the USB throughput I needed for this application. However with the step-up in power comes with an even bigger step-up in complexity. Fortunately NXP have done really good job at bridging the complexity gap with first class tooling.

The MCUXpresso IDE is based on Eclipse which is a tool I use a lot for java development. So while I can imagine that if you have only ever used the Arduino IDE it could be overwhelming I immediately felt orientated.

I was able to get the i.MX RT1010 EVK for the discounted $10.10 although this was somewhat offset by shipping and taxes which increased the price 5 fold!

Nevertheless it's a really useful development board and the onboard programmer/debugger is especially good. I have been amazed how far Arduino has come without a debugger, but like a smart-phone, you don't know how you lived without it once you have it.

I found the debugger especially useful to interactively explore the examples which I ultimately used to sew together the solution.

Gotchas's

Not to say I didn't run into some gotcha's...

Flex Pins not broken out to the Arduino Pins

My first idea was to use the Flex Pins to create a parallel array of UARTs' directly from FlexIO. However I discovered the Muxing options while quite good are limited enough that you can't just map any pin wherever you like. As it turns out the FlexIO are not mapped to the Arduino pins on this board so I moved to the SPI solution.

FlexIO Bricked the Board

Before I really knew what I was doing I ran a FlexIO example which didn't match the board. Somehow, I managed to lock-up some pins which locked out the programmer/debugger bricking the unit even after power reset. Actually I though it was game over and I wouldn't be able to complete the project. Somehow I used the DIP switches to change the programming mode and reprogram a new firmware with the FlexIO. Note to self - be careful with FlexIO!

Shared IO

Many of the pins functions are physically shared using very small 0 Ohm smd resistors. The only way I could figure out how to get the functionality I wanted was to look up the schematic in the build files and manually hunt for the SMD resistors to solder/desolder using a magnifying glass! I couldn't find a searchable PDF for the resistor names and the build files are gerber so I couldn't figure out how to search those for resistor locations.

i.MX RT1010 USB : SPI Bridge

I have been researching how to get high data transfer over USB for a while and it's not that easy. HID USB has a theoretical limit of 64Kbytes/second but actual realised speeds much lower. High speed FTDI chips require drivers and are not supported by UWP.

Virtual COM is the best solution because of the driverless UWP support so when I saw this post on the NXT forum I was excited to see what the RT1010 could do:

RT1020 VCOM USB throughput at 26Mbytes/s

Inspired by this I merged the following two projects, replacing the echo inner loop with an SPI Master.

• evkmimxrt1010_dev_cdc_vcom_lite_bm
• evkmimxrt1010_lpspi_polling_b2b_transfer_master

Inner Loop

These SDK examples are frameworks which wrap an enormous amount of functionality to kickstart solutions. The APP_task routine hosts the user code and this is where the inner loop is hosted.

void APP_task(void)

..
/* User Code */
..
while( DecodeIncomingFrame() ){
    if(VerifyFrameFormat())
    {
        ExchangeDataWithSlaves();
    }
    _frameWritePos = 0;
}

ExchangeDataWithSlaves()

The core routine is to exchange a DataFrame with the array of slaves. I used the default size of 64 bytes used in the SPI master example except replacing the first byte with the length of actual data.

Slave exchange packet : [ N length ] [ 0.. 63 data ]

SPI sends and receives at the same time, so 64 bytes are always sent/received even if less than 63 bytes of actual data are available to send. This is because the slave can have a fully populated data frame to return.

USBDataFrame

Unlike HID which is packetised the USB CDC profile is a stream so the DataFrame packets need to be recovered from the stream. To encoded the frames I used SLIP ( Serial Line Internet Protocol )

SLIP Encoding

The SPI sends the full 64 bytes but in the case of the USB Dataframe the data is packed.

To illustrate: This test USB frame contains outgoing data only to slave 1.

testFrame[0] = FRAME_START;
testFrame[1] = 2;	  //Slave 1
testFrame[2] = 'H';
testFrame[3] = 'I';
testFrame[4] = 0; //Slave 2
testFrame[5] = 0; //Slave 3
testFrame[6] = 0; //Slave 4
testFrame[7] = 0; //Slave 5
testFrame[8] = 0; //Slave 6
testFrame[9] = 0; //Slave 7
testFrame[10] = 0; //Slave 8
testFrame[11] = 0; //Slave 9
testFrame[12] = 0; //Slave 10
testFrame[13] = FRAME_END;

SPI : UART Slaves

The protocol exchanges data with 10 SPI slaves. SPI slaves are selected using the (SS) Slave Select Line which are broken out using the Arduino GPIO header pins.

+-----+----------------+------------+----------+--------+
|     |   Schematic    | iMX.RT1010 |  Pin No  | Slave# |
+-----+----------------+------------+----------+--------+
| D0  | LPUART1_RXD    | GPIO_09    | UART     |        |
| D1  | LPUART1_TXD    | GPIO_10    | UART     |        |
| D2  | INT1_COMBO     | GPIO_AD_05 | SPI*     |        |
| D3  | INT2_COMBO     | GPIO_AD_06 | SPI*     |        |
| D4  | FLEXPWM1_PWMA3 | GPIO_08    | GPIO1.8  |      1 |
| D5  | FLEXPWM1_PWMB0 | GPIO_01    | GPIO1.1  |      2 |
| D6  | ADC12_1        | GPIO_AD_01 | GPIO1.15 |      3 |
| D7  | ADC12_2        | GPIO_AD_02 | GPIO1.16 |      4 |
| D8  | GPIO_SD_02     | GPIO_SD_02 | GPIO2.02 |      5 |
| D9  | FLEXPWM1_PWMB1 | GPIO_03    | GPIO1.3  |      6 |
| D10 | LPSPI1_PCS0    | GPIO_AD_05 | SPI      |        |
| D11 | LPSPI1_SDO     | GPIO_AD_04 | SPI      |        |
| D12 | LPSPI1_SDI     | GPIO_AD_03 | SPI      |        |
| D13 | LPSPI1_SCK     | GPIO_AD_06 | SPI      |        |
| A0  | ADC12_3        | GPIO_AD_07 | GPIO1.21 |      7 |
| A1  | ADC12_4        | GPIO_AD_09 | GPIO1.23 |      8 |
| A2  | ADC12_5        | GPIO_AD_10 | GPIO1.24 |      9 |
| A3  | ADC12_6        | GPIO_AD_14 | GPIO1.28 |     10 |
| A4  | ADC12_1        | GPIO_AD_01 |          |        |
| A5  | ADC12_2        | GPIO_AD_02 |          |        |
+-----+----------------+------------+----------+--------+

Each Test Shield uses a PIC18F27Q43 to as a SPI slave. These are the latest 8-bit micro's from microchip with 8Kb ram for buffering, High speed SPI with DMA and 5 hardware UARTS to implement a SPI : UART bridge.

Performance

Using the test tool the performance is hitting the 5Mhz SPI + packet overhead which is already sufficient for this application without any optimisation. The limit is the speed on the SPI Slave devices. There are 2 SPI channels on the iMX.RT1010 so when I create a custom PCB I can boost speed further by serving additional slaves on the second SPI channel.

Next Steps

The i.MX RT1010 has proven itself as an ideal high speed USB:SPI bridge for this application. Next I want to release the Test Tower as a product.

Custom PCB Base

The next step is to figure out what's needed to roll-my own stripped down version of the i.MX RT1010 EVK. Maybe with a bigger power supply to support higher powered devices in the future.

Casing

I love the idea of creating a fancy 3-D printed casing.. something like this.. Very matrix like.

Kickstarter

Watch this space.

Thanks!