Say “Hello” to Google Coral

Update: I’ve now managed to get some hands-on time with the Coral Dev Board, and the USB Accelerator. You can use a Raspberry Pi to help setup the Dev Board.

During Injong Rhee’s keynote at last year’s Google Next conference in San Francisco, Google announced two new upcoming hardware products: a development board and a USB accelerator stick. Both products were built around Google’s Edge TPU, their purpose-built ASIC designed to run machine learning inference at the edge.

Almost a year on, the hardware silently launched “into Beta” under the name “Coral” earlier today, and both the development board and the USB accelerator are now available for purchase. The new hardware will officially be announced during the TensorFlow Dev Summit later this week.

“The Edge TPU is a small ASIC designed and built by Google that provides high performance ML inferencing with a low power cost . For example, it can execute state-of-the-art mobile vision models such as MobileNet v2 at 100+ fps, in a power efficient manner. This allows you to add fast ML inferencing to your embedded AI devices in a power-efficient and privacy-preserving way.”

Machine learning development is done in two stages. An algorithm is initially trained on a large set of sample data on a fast powerful machine or cluster, then the trained network is deployed into an application that needs to interpret real data. This deployment stage, or “inference,” stage is where the Edge TPU and the new hardware comes in, with the ability to run these trained networks “at the edge” nearer the data, enabling developers to put the smarts on the smart device, rather than in the cloud. Allowing them to build smart devices that uses machine learning without a network connection at all.

“The Coral USB Accelerator is a plug-in USB stick that brings powerful ML inferencing capabilities to existing Linux systems. With the Edge TPU connected over USB 3.0 interface, it allows for quick prototyping of local AI applications.”

Much like the Intel’s Movidius Neural Compute Stick released a year and a half ago, the Coral USB Accelerator wraps their custom ASIC in an easy to use stick form factor that looks a lot like a flash disk. However comparing the two side-by-side and there are obvious differences. For a start, the Coral stick is a lot smaller. That doesn’t seem all that important until you realise that the Intel stick was so large it tended to block nearby ports, or with some computers, be hard to use at all.

Part of that size difference is that the Coral USB Accelerator stick has a USB 3.0 Type-C socket for data and power. However as it comes with a USB-C to A cable if you don’t have a USB-C laptop, like one of the latest generations of Mac, you can still use it straight out of the box.

Also unlike the Movidius stick, which had a lot of early problems with Arm-based computers like the Raspberry Pi, the Coral stick should work out of the box with the Raspberry Pi, albeit at USB 2.0 speeds. In fact, it should also work with any 64-bit Arm, or x86, platform supported by Debian Linux.

The stick is built around an 32-bit Arm Cortex-M0+ microprocessor running at 32MHz, with just16KB of flash, and 2KB of RAM, which provides all the needed support for the Edge TPU inside the stick. Interestingly, the datasheet suggests that the USB Accelerator should only draw 500mA when running.

“The Coral Dev Board is a single-board computer with a removable system-on-module (SOM) that contains SOC, eMMC, wireless radios, and Google’s Edge TPU. It’s perfect for IoT devices and other embedded systems that demand fast on-device ML inferencing.”

Running Debian Linux, the Coral Dev Board consists of a base board with peripheral connections; including USB 2.0/3.0 ports, DSI display interface, MIPI-CSI camera interface, a Gigabit Ethernet port, a 3.5mm audio jack, 2.54mm 4-pin terminal for stereo speakers, a full sized HDMI 2.0a connector, two Digital PDM microphones, and a “Raspberry Pi like” 40-pin GPIO header.

Attached to the base board is a removable 40×48mm system-on-module (SoM) built around an NXP i.MX 8M processor, and the Edge TPU itself. The SoM has a cryptographic coprocessor, onboard Wi-Fi and Bluetooth 4.1 support, as well as 1G of LPDDR4 RAM, and 8GB of eMMC. Full details of the specification can be found in the board’s datasheet.

Unlike the USB Accelerator, which anyone can pick up and use, it’s obvious that the development board is a supposed to serve as an example board for the SoM board which Google is planning to make available “in volume” later to be combined with your own custom PCB hardware using a board-to-board connector for integration directly into a commercial product.

Intended as a product development tool, this board is intended for hardware developers use to prototype IoT devices and other embedded systems that demand fast on-device machine learning inferencing.

Also available is a camera to accompany the development board. Built around an 5 mega-pixel (2582×1933 pixel) Omnivision OV5645 sensor, the camera connects to he development board’s MIPI-CSI connector. It has a focal length of 2.5mm, with a focal range from 10cm out to infinity, and a field of view of 84 degrees.

Both the development board and the USB accelerator sit on top of Google’s software stack for the Edge TPU. Models need to be developed in TensorFlow Lite rather than TensorFlow, and then compiled to run on the Edge TPU using a web compiler. Some pre-compiled models are also available.

During the current beta period the compiler the Edge TPU compiler has some restrictions. But these restrictions should be lifted when Coral comes out of Beta testing next month.

Using a web compiler is a neat move by Google to get around a problem faced by the Intel Movidius hardware that required another x86 based development machine to compile your models to use on the accelerator hardware if you were using it with an Arm-based board like the Raspberry Pi.

Model inference performance with the Edge TPU should be significantly faster in comparison to using a desktop or embedded CPU. About a month ago performance figures of a “mysterious” device called Google Coral were spotted on Geekbench, and assumed to be an upcoming Android phone.

However comparison with CPU probably not the really interesting comparison. The really interesting numbers will be when the Coral hardware is out in the wild and we can compare it with other custom accelerators, like the Movidius Neural Compute Stick, updated at the tail end of last year. So look out for those when they arrive.

The Coral development board is priced at $149.99, and the USB accelerator is priced at $74.99. While Google’s own site asks you to “…call Mouser and order using the part number,” the Mouser pages for the board and the accelerator are live. The page for the accelerator is indicating that they’re in stock and ready to ship, while the page for the development board is indicating a factory lead time of “17 weeks.” Yet don’t be fooled, that’s speak for “out of stock” or other supply problems for new products and my sources indicate that the development board should be shipping “…by next week.” So if you order now it’s likely that you should get your hardware a lot sooner.

A PCI-e Accelerator board built around the Edge TPU is also in development, and Google intend to offer the 40×48mm system-on-module (SoM) used by the Coral development board separately for large volume purchases. However both of the PCI-e board, and the SoM, will not be available until “…later in the year” so if you’re looking to get hands on with the Edge TPU picking up a USB accelerator stick and a Raspberry Pi is going to be the fastest route right now.

UPDATE (March 6th): Coral was officially announced today as the TensorFlow Dev Summit kicked off in Santa Clara, CA, with June Tate-Gans, who is the lead software engineer for Coral at Google, demonstrating on-device transfer learning with the Coral Dev Board on stage.

There were also some interesting snippets in the official announcement post on the Google Developers blog, with the news that Google will “…be releasing the baseboard schematics for those who want to build custom carrier boards [for the SoM].”

Update (March 26th): I’ve now got hands on with both the Coral Dev Board and the Coral USB Accelerator. More, including benchmarking against the Intel and NVIDIA competitors, coming soon…