reTerminal Machine Learning Demos (Edge Impulse and Arm NN)

Last few articles I published were about TinyML with Wio Terminal - a Cortex M4F based development board with an LCD screen in sturdy plastic case. Seeed studio, the company made Wio Terminal decided to take this idea a step further and recently announced reTerminal - a Raspberry Pi 4 Compute Module based development board with an LCD screen in sturdy plastic case.

I got my hands on one of the reTerminals and have made a brief unboxing video, which also included a few demos and an explanation of possible use cases for the device. The article is means as a supplement for the video, expanding on how to set up the environment and run the machine learning demos.

UPDATED 03/29/2022. I try my best to keep my articles updated on a regular basis and based on your feedback from YouTube/Hackster comments section. If you'd like to show your support and appreciation for these efforts, consider buying me a coffee (or a pizza) :) .

Specifications

reTerminal is powered by a Raspberry Pi Compute Module 4 (CM4) with Quad-core Cortex-A72 CPU running at 1.5GHz. CM4 module version used for reTerminal has 4 Gb of RAM and 32 Gb of eMMC storage, which shortens boot-up times and gives smoother overall user experience. Peripherals-wise, there is a 5-inch IPS capacitive multi-touch screen with a resolution of 1280 x 720, accelerometer, RTC module, buzzer, 4 buttons, 4 LEDs and a light sensor. And for connectivity, the new board has dual-band 2.4GHz/5GHz Wi-Fi and Bluetooth 5.0 BLE, plus Gigabit Ethernet port on the side.

reTerminal can be powered by the same power supply used for Raspberry Pi 4, 5V2A, however in the official description 4A power supply is recommended, especially when you connect more peripherals. For the demos I have used a run of the mill unknown company 5V2A wall plug power supply and didn't get under-voltage warning. Having said that, if in doubt, use 5V4A.

By default reTerminals ship with pre-installed 32-bit Raspbian OS, with devices drivers installed. However, since for machine learning applications, a 64-bit OS can give a significant boost, Seeed studio will also provide a 64-bit version of the Raspbian OS image with reTerminal-specific drivers pre-installed.

Onboard screen keyboard and a simple QT5 demo are included as well. Touchscreen is responsive, but since Raspbian OS is not mobile operating system and not optimized for touchscreens it sometimes can be a bit troublesome to press on smaller UI elements. Having a stylus helps a lot.

Onboard screen keyboard pops up when you need to type the text and disappears after that. You can modify that behavior in the settings. So it is possible to use reTerminal as portable Raspberry Pi, although for that you might want to have a look at another OS, for example Ubuntu touch, which works with Raspberry Pi 4, but currently is in beta development stage and highly experimental. The main use case for reTerminal is displaying user interfaces made with QT, LVGL or Flutter. Let’s launch a sample QT application, that shows device specifications and parameters, data from sensors and example control board for imaginary factory. When interface elements are large, the touch screen is much pleasant to use.

Edge Impulse Object Detection

We’re going to make use of latest feature of Edge Impulse development platform, Linux deployment support. We can easily train an object detection model by collecting samples with camera connected to reTerminal, then train in the cloud and automatically download and run the trained model with edge-impulse-linux-runner.

The installation procedure of Edge Impulse CLI is described in documentation. It all comes down to a few simple steps:

curl -sL https://deb.nodesource.com/setup_12.x | sudo bash -
sudo apt install -y gcc g++ make build-essential nodejs sox gstreamer1.0-tools gstreamer1.0-plugins-good gstreamer1.0-plugins-base gstreamer1.0-plugins-base-apps
npm config set user root && sudo npm install edge-impulse-linux -g --unsafe-perm

After Edge Impulse CLI is installed, make sure you have the camera connected - I used a simple USB Web camera, if you use Raspberry Pi camera, remember to enable it in raspi-config.

Before you start collecting the data for object detection, make sure that in Dashboard under 'Project info > Labeling method' 'Bounding boxes (object detection)' is selected.

Take at least 100 images for each class you want to recognize. Currently you can upload your own images, by pressing on Show options - Upload data in Data Acquisition tab. But it's not possible to upload the bounding boxes annotations yet, so the images you upload will still need to be manually labeled. Aftr you have enough images annotated, go to Create Impulse, choose Image for processing block and Object Detection(Images) for Learning block.

The amount of images one user can collect and annotate is not nearly enough to train a large network from scratch, that's why we fine-tune a pre-trained model to detect new classes of objects. In most of cases you can leave the default values for number of epochs, learning rate and confidence. For object detection, a custom code is used, so we cannot tweak it in Expert mode, as it is possible with simpler models.

Training is done on CPU, so it takes a bit of time, depending on number of images in your dataset. Have a cup of your favorite beverage, while you at it.

One of the best things about newly added Linux support for Edge Impulse is the edge-impulse-linux-runner. When the model training is finished and you're satisfied with accuracy on validation dataset (which is automatically split from training data), you can test the model in Live classification and then go on to deploying it on the device. In this case it is as simple as running

edge-impulse-linux-runner

in terminal. The model will be automatically downloaded and prepared, then the inference result will be displayed in browser, you will a line on your terminal, similar to:

Want to see a feed of the camera and live classification in your browser? Go to http://192.168.1.19:4912

Click on the link in your terminal to see the camera live view.

The backbone model used for transfer learning is MobileNetv2 SSD and is quite large, so even with all the optimizations we get about 2 FPS or ~400 ms. for a frame - the video stream looks quite responsive, but that’s because inference is not performed on every frame, you can clearly see that if you detect an object and then it disappears from the image, the bounding box for it stays on for some time. Since Linux support is a very new feature in Edge Impulse, I'm sure it will receive a lot of improvements in the near future, allowing for faster inference and upload of user-annotated data.

ARM NN Accelerated Inference

While we know that Raspberry Pi 4 is not the best board for machine learning inference, since it doesn’t have any hardware accelerator for that, we still can achieve higher than real time inference speed by

a) using smaller mode

b) making sure we utilize all 4 cores and Single Instruction Multiple Data (SIMD) instructions, where multiple processing elements in the pipeline perform operations on multiple data points simultaneously, available with Neon optimization architecture extension for Arm processors.

I trained a simple single class face detector on WideFaceNet using MobileNet v1 alpha 0.25 feature extractor with the help of my personal project, aXeleRate, a Keras-based framework for AI on the edge.

You can train on a local computer with NVIDIA GPU or entirely in Google Colab - by specifying “tflite” as converter type you will receive a.tflite model, that can be run with some adjustments using ARM NN - a optimization toolkit, that allows us utilize 4 cores of compute module and NEON optimization architecture extension. Follow the steps in Colab notebook I shared to train a similar face detection model or download the exact same model that I used for demo here.

ARM NN is only distributed through apt-get for Ubuntu 20.04 or Debian Bullseye. Which means that to install it on Raspberry OS (Debian Buster) the only "proper" way is daunting compilation from source (including compilation from source of Tensorflow). I'll be researching on better ways to install ARM NN on Debian Buster, but meanwhile there is a hacky solution, that will allow you to try out ARM NN. Add

deb http://deb.debian.org/debian bullseye main contrib non-free

to your /etc/apt/sources.list and then do

sudo apt-get update

After that you will be able to install ARM NN with the following command

sudo apt-get install python3-pyarmnn libarmnn-cpuacc-backend22 libarmnn-cpuref-backend22

It's very likely that apt-get will install and update many other different packages, including your system Python 3 while installing ARM NN - as I said it is a hacky temporary solution, that will work for development and testing purposes.

After ARM NN is installed, git clone aXeleRate repository

git clone --branch dev https://github.com/AIWintermuteAI/aXeleRate.git

And then execute run_video_file.py or run_video_stream.py in example_scripts/arm_nn/face_keypoints folder, e.g.

python3 run_video_file.py --fd_model_file_path YOLO_best_mAP.tflite --kp_model_file MobileFaceNet_kpts.tflite --video_file_path test_s.mp4

or

python3 run_video_stream.py --fd_model_file_path YOLO_best_mAP.tflite --kp_model_file MobileFaceNet_kpts.tflite

You can download reference models (face detector and key-points detector) by clicking on this link. To see how you can optimize your own models, have a look at example source code and object detection with ARM NN tutorial and additionally, image recognition with ARM NN tutorial.

Here is the demo result. You can see the face keypoints (right eye, left eye, nose, right corner of the mouth, left corner of the mouth) as blue circles.

The demo won’t be included with reTerminal, but if you follow the steps in the accompanying article, you’ll be able to reproduce the result yourself or train a model for different objects. I went a step further and added second stage inference on detected face - face keypoint calculation. This is second stage of face recognition pipeline, which I will implement and demonstrate in my later video about face recognition on Raspberry Pi 4.

Open-domain Speech Recognition with DeepSpeech

Last, but not least, let’s run real-time speech recognition demo with latest version of DeepSpeech.

To install DeepSpeech on reTerminal, simply type

pip3 install deepspeech

To run real-time speech recognition demo, install the Device Tree overlay for reSpeaker (of course you can use other microphone as well), then download DeepSpeech model and scorer

wget https://github.com/mozilla/DeepSpeech/releases/download/v0.9.3/deepspeech-0.9.3-models.tflite
wget https://github.com/mozilla/DeepSpeech/releases/download/v0.9.3/deepspeech-0.9.3-models.scorer

Finally you will also need to git clone the repository with user-contributed examples for DeepSpeech

git clone https://github.com/mozilla/DeepSpeech-examples.git

Install the dependencies for mic_vad_streaming example and then run

python3 mic_vad_streaming.py --model [path-to-tflite-model] --scorer [path-to-scorer]

Depending on level of noise in your environment, you might want to experiment with VAD (voice activity detection) aggressiveness setting, which can be set with -v argument (from 0 to 3, 0 being the least aggressive VAD).

Conclusion

What are my thoughts about reTerminal? Machine Learning loads wise, since it just plan Raspberry Pi CM 4 with a generous amount of RAM, you can run simple computer vision/ speech recognition in real time or even faster, provided you optimize networks well enough. Just don’t expect it to be able to juggle 4 different models inference at the same time like Xavier NX.

On the other hand, I found it convenient to use as a portable Pi, although using stylus for desktop interface is a must. Hopefully in future an official image for Ubuntu Touch will be provided, which will increase the touch screen usability for interface interaction greatly. If you plan to use reTerminal as OEM controller with user interface, you’ll write your own UI, so that won’t be an issue. Presence of the touch screen makes it also possible to mount it on a robot for debugging or user interaction. Some people in comments already noted that a battery extension would be a great addition.

There are other extensions planned to be released. What extension would you like to see available? Share your ideas in the comments below - I’ll make sure to pass them to hardware engineers and product manager.