A Rockchip RK1808-Based USB Stick for Machine Learning

Over the last six months I’ve been looking at deep learning on the edge, and investigating the new generation of custom silicon designed to speed up machine learning inferencing on embedded devices. The original accelerator hardware was launched by Intel back in 2017, but since then we’ve seen more hardware from Intel, Google, NVIDIA, and others.

Right now, we’re seeing a deluge of new hardware based around the Intel Movidius, and the Gyrfalcon Lightspeeur chips. I’m also expecting to see hardware based around Google’s Edge TPU later in the year when their System-on-Module (SoM) is finally available in volume.

However, there are other less known players in the accelerator market, one of these is Rockchip with their Neural Processing Unit (NPU). So say “Hello” to yet another USB stick of Deep Learning, the Toybrick AI Compute Stick now available to pre-order from Seeed Studio.

Despite the odd name given to it by Seeed Studio on their site, “RK1808 AI Compute Stick — Intel Core i3 Processor,” there is no Intel Core i3 processor inside this stick. The only mention of an Intel Core i3 I can find in the user manual in the part where it describes the minimum system requirements for the host machine. But I’m guessing, just like Intel’s Neural Compute Stick 2, this is only needed for training rather than inferencing. But, your milage may vary there, so you should be careful if you’re ordering this hardware with the intention of using it with a Raspberry Pi or other Arm-based boards.

Instead, the Toybrick branded stick—which is the AI platform brand of Fuzhou Ruixin Microelectronics or, in other words, Rockchip itself—is built around the Rockchip RK1808 NPU.

The Rockchip RK1808 is a standalone version of the NPU that we first saw built in to the Rockchip RK3399Pro used in Pine64’s RockPro64 board earlier in the year. Based on a dual-core Arm Cortex-A35, with 2MB of SRAM, the on-chip NPU is advertised to offer performance up to to 3 TOPs. So that’s roughly equivalent to the Intel MovidusX, which advertises performance of 4 TOPs, although as we’ve seen the real life performance of the MovidusX is somewhat lower. The RK1808 can be had in volume for $6 — $12 per unit, it’s really hard to say what the per-unit chip cost of the MovidiusX is, let alone the Edge TPU.

Looking at the specifications on the Seeed Studio site the Toybrick stick, like both the Intel Neural Compute Stick 2 and the Coral USB Accelerator from Google, connects to a host computer via USB 3. The stick also has 1GB of LPDDR memory, and 8GB of EMMC storage, on board.

The only official information we have around the stick, at least from Seeed Studio, is the user manual. This suggests that the stick is restricted to x86 platforms only, that “Intel Core i3” in the minimum system requirements is somewhat suggestive.

However, if we go to the announcement in the Toybrick forums we see, albeit somewhat roughly translated that, “…built-in high computing RK1808 NPU processor, low power consumption, strong compatibility; support Windows, Linux, MacOS, Arm Linux and other platforms; AI application development SDK supports C/C++ and Python; based on USB 3.0 Type A interface.” So somewhat intriguingly, it appears that the new Toybrick stick might support Arm Linux, and hence the Raspberry Pi, after all.

It’s likely the new Toybrick stick might well be a rebranded version of the Rockchip stick we saw in Los Vegas at CES 2019 back at the start of the year, which afterwards more or less disappeared from view.

Like the Intel Neural Compute Stick 2, which the new Toybrick AI Compute Stick is obviously attempting to imitate with its bright blue shell, we can’t use TensorFlow directly with the stick. While the Intel Movidius-based Neural Compute Stick hardware requires you to use OpenVINO, the Toybrick RK1808-based hardware also uses a custom framework, the RKNN toolkit with development supported in both C++, and Python.

Interestingly, and uniquely as far as I’m aware, the new stick also acts as a USB disk with the documentation and software tools stored on the stick itself. Although there appears to also be a GithHb repo and a Wiki for the RKNN toolkit, which seems to make some of the same materials available along with some advice on model conversion from TensorFlow and other popular machine learning platforms like Caffe and PyTorch.

The RK1808 AI Compute Stick is available to pre-order from Seeed Studio today, with an estimated availability date of August 30th. The stick is priced at $86 plus shipping.

It looks like my final benchmarking post might not be as final as I thought?

It’s going to be really intriguing to see how the Rockchip RK1808 measures up to the Intel MovidiusX or the Google Edge TPU. Expect to see a hands-on with this sometime around mid-September, just as soon as Seeed Studio can get one to me.