Edge Impulse Raspberry Pi 5 Santa Tracker

Idea 🤔 💡

Edge Impulse have long supported single-board computers like the Raspberry Pi 4 and NVIDIA Jetson Nano, but having been lucky enough to get my hands on the new Raspberry Pi 5, I wanted to do the community a favour and give it a quick shakedown to make sure everything still works on the latest hardware. And because I'd just read a book to my daughter about children staying up to try to catch a glimpse of Santa, I thought why not automate that task, so she can get some rest on Christmas Eve while a Raspberry Pi does all of the work! 🤔🔎🎅

Getting Started 🔰 👩‍💻

There are countless guides on setting up a Raspberry Pi, so I won't reproduce that here — gone are days of having to dd a raw image onto an SD card — in fact, I just used the Raspberry Pi Imager to flash Raspberry Pi OS with desktop (do make sure it's the 64-bit version) onto a mSD card and popped it in and let it boot. After connecting to my local Wi-Fi, and attaching the webcam via USB, I was ready to install the Edge Impulse dependencies:

sudo apt update
curl -sL https://deb.nodesource.com/setup_12.x | sudo bash -
sudo apt install -y gcc g++ make build-essential nodejs sox gstreamer1.0-tools gstreamer1.0-plugins-good gstreamer1.0-plugins-base gstreamer1.0-plugins-base-apps npm
sudo npm install edge-impulse-linux -g --unsafe-perm

Note that this package list differs slightly than EI's own instructions, since my Pi did not have npm installed, which is necessary for the last step. Once everything is installed, create a project in Edge Impulse Studio and run the following in the Pi's terminal:

edge-impulse-linux --disable-microphone

I used --disable-microphone because this is a vision project and I didn't have a microphone connected; you can also use --clean to switch accounts/projects. You can then head over to your project and confirm that the Pi 5 is connected under Devices.

Building a Dataset 📸 💾

I was excited to leverage the Pi 5's power and pull out all of the stops with whatever crazy model and image resolution I wanted, so I initially started out by experimenting with FOMO and MobileNet SSD multiple-object detection, but experienced a number of false positives, which won't do when trying to catch Santa, so I simplified with regular single-object image classification.

Edge Impulse Studio makes it really easy to build a dataset; with your Pi 5 all set up and the camera connected, you can use your browser on another computer, or the Pi itself (the Pi 5's incredible performance finally makes this not just bearable, but trivial), by just pointing at the object, giving it a label, and hitting Start sampling under Data acquisition. In our case, we captured around 30 pictures of various Santas under the santa label, and 30 pictures of not-Santas under unknown.

Now is a good time to reserve some images for testing using Select multiple items, then Move to test set. Six from each class is 20%, which is a good amount.

With 30 images collected for each class, and 20% of them reserved for testing, we're ready to train our model.

Training our Model 👨‍🔬 🧠

With plenty of images to train on, it's time to design our "impulse." Under Create impulse, configure your image, preferably based on existing model resolutions like 96x96 or 160x160, then add Image and Transfer learning (Images) blocks. Hit Save Impulse and move to the next tab below, Image. Our Pi 5 has plenty of power, so we'll stick with RGB and hit save. From here hit Generate Features and regard the pretty Feature explorer which should serve to reassure you that your classes are sufficiently differentiable.

Next click Transfer learning and set:

Number of training cycles: 20

Learning rate: 0.0005

Data augmentation: checked

For the model, we might as well go all out. In the Mega or Mini? project, we were severely constrained by the OpenMV Cam H7's 1MB of RAM, but the Pi 5 has GBs, so we can luxuriate ourselves in the full MobileNetV2 160x160 1.0 model as we click Start training and laugh in the face of the resulting RAM usage and model size.

Now click on Model testing in the left nav, and Classify all to run the model against the previously-unseen images we reserved for testing. Here we achieved 91.67% accuracy, and if we select Show classification under the three dots (⋮) next to the single flagged image, we see it was a Santa that was wandering off to the side and had an unusually large white-to-red ratio in profile, yet still technically got classified correctly with 51% confidence that he was a Santa, so we can safely ignore that and move on to on-device testing.

Running our Model 📸 🧠

Deploying and running a model on Linux with Edge Impulse is outrageously easy! Just enter:

edge-impulse-linux-runner

at the command line (after killing edge-impulse-linux if you were still running that from data collection) and your model is automatically downloaded, compiled, and executed, and even served as a convenient web-based tool that lets you monitor your camera feed and resulting classification from any browser on the same network!

Results and Conclusions ⚗️ 📊

The MobileNetV2 model exceeded predictions with reliable classification of Santa and not-Santas in tens of milliseconds:

Mrs. Claus and a small Santa candle were correctly classified as Santa-like, while Kirby, a jack-o-lantern, and a Christmas tree were all marked unknown with extremely high levels of confidence. 🔎🎅💯

Next Steps/Future Enhancements 🔜 🔮

Even using 160x160 RGB with the full MobileNetV2 160x160 1.0 only took a few MB of RAM and storage and the Pi 5 was typically inferencing in around 14-15ms. There is clearly much, much, much more performance potential here, so I'd love to throw some really huge models at it and see how it fairs. 🤯 Until then, it's time to set it up in front of the fireplace and see if we can catch the big fella in the act this year! 🔎🎅🎁