Jeff Geerling Gets a Raspberry Pi 5 Talking to a Google Coral TPU PCIe Accelerator for Faster ML, AI
Proof that the Raspberry Pi 5's PCIe connection is good for more than high-speed storage is here, though a few workarounds are needed.
Jeff Geerling has become the first to give a Raspberry Pi 5 a short in the arm for on-device machine learning projects, successfully bringing up a Google Coral Tensor Processing Unit (TPU) accelerator on the board's PCI Express connection.
"Coral.ai TPUs are AI accelerators used for tasks like machine vision and audio processing. Raspberry Pis are often integrated into small robotics and IoT products — or used to analyze live video feeds with Frigate," Geerling explains by way of background to his project. "Until today, nobody I know of has been able to get a PCI Express Coral TPU working on the Raspberry Pi."
Part of that, of course, is that previous releases of the popular single-board computer have lacked user-accessible PCI Express connectivity — though the addition of a USB-connected Coral TPU was a popular choice. The Raspberry Pi Compute Module 4 included a single user-accessible PCI Express lane, if not used by the carrier board of the user's choice, but quirks in the way PCIe was implemented on the device meant high-performance non-storage devices like the TPU and graphics cards typically failed to work.
The Raspberry Pi 5 launched last month as the first consumer-facing model to include an easily-accessible PCIe lane — though brought out to a Flat Flexible Circuit (FFC) connector which requires an adapter, the official model of which has yet to be released. Using a third-party alternative, Geerling has worked through some issues to get the PCIe Coral TPU talking to the Raspberry Pi 5 — and has documented how you can do the same.
"To get the TPU working, we are going to need to overcome some hurdles," Geerling explains. "Coral's drivers only work on 4K page size, so we need to switch from the default Pi kernel. The Coral is a bit picky with PCIe timings, so (for now at least) we need to disable PCIe ASPM [Active State Power Management].
"The Pi's default device tree sets up the PCIe bus to not have enough MSI-X [Message Signaled] interrupts," Geerling continues, "so we need to change it. Pi OS 12 'Bookworm' ships with Python 3.11, but Coral's PyCoral library only runs on 3.9, so we need to run inside Docker (or install an alternate system-wide Python version). [And] there's no A+E key adapter/HAT for the Raspberry Pi (yet), so we need a hardware interface to plug the Coral TPU into the Pi 5's PCIe header."
Once those issues have been resolved, the Coral TPU shows up for the acceleration of on-device machine learning workloads, and without the overhead of a USB connection — and can be used alongside the Raspberry Pi 5's CPU and GPU cores, both of which show a considerable bump in performance over the Raspberry Pi 4 and Compute Module 4.
Geerling's full write-up is available on his website.