Running TensorFlow Lite "Person Detection" on RTL8722DM-M...

Ameba RTL8722DM-MINI is an easy-to-program platform for developing all kinds of IoT applications

Hardware1x RTL8722DM-MINI

3x LED

1x Arducam Mini 2MP Plus OV2640 SPI Camera

Introduction to Google TensorFlow

TensorFlow (TF) is an end-to-end open-source platform for machine learning. It has a comprehensive, flexible ecosystem of tools, libraries, and community resources that lets researchers push the state-of-the-art in ML, and developers easily build and deploy ML-powered applications.

TensorFlow Lite for Microcontrollers

While TensorFlow Lite (TFL) for Microcontrollers is designed to run machine learning models on microcontrollers and other devices with only a few kilobytes of memory. The core runtime just fits in 16 KB on an Arm Cortex M3 and can run many basic models. It doesn't require operating system support, any standard C or C++ libraries, or dynamic memory allocation.

Ameba and TFL for Microcontrollers

Ameba is an easy-to-program platform for developing all kinds of IoT applications. AmebaD is equipped with various peripheral interfaces, including WiFi, GPIO INT, I2C, UART, SPI, PWM, ADC. Through these interfaces, AmebaD can connect with electronic components such as LED, switches, manometer, hygrometer, PM2.5 dust sensors, …etc.

Materials

• Ameba D [RTL8722 CSM/DM/DM-MINI] x 1
• Arducam Mini 2MP Plus OV2640 SPI Camera x 1
• LED x 3

Example Procedures

Connect the camera and LEDs to the RTL8722 board following the diagram.

Download the Ameba customized version of TensorFlow Lite for Microcontrollers library at ambiot/ambd_arduino. Follow the instructions at Arduino - Libraries to install it.

Ensure that the patch files found at ambiot/ambd_arduino are also installed.

You will also need to install the Ameba_ArduCAM library, found together with the TensorFlow Lite library.

In the Arduino IDE library manager, install the JPEGDecoder library. This example has been tested with version 1.8.0 of the JPEGDecoder library.

Once the library has been installed, you will need to configure it to disable some optional components that are not compatible with the RTL8722DM. Open the following file: Arduino/libraries/JPEGDecoder/src/User_Config.h

Make sure that both #define LOAD_SD_LIBRARY and #define LOAD_SDFAT_LIBRARY are commented out, as shown in this excerpt from the file:

• //#define LOAD_SD_LIBRARY // Default SD Card library
• //#define LOAD_SDFAT_LIBRARY // Use SdFat library instead, so SD Card SPI can be a bit bashed

Open the example, “Files” -> “Examples” -> “TensorFlowLite_Ameba” -> “person_detection”.

Upload the code and press the reset button on Ameba once the upload is finished.

Once it is running, you should see the blue LED flashing once every few seconds, indicating that it has finished processing an image. The red LED will light up if it determines that there is no person in the previous image captured, and the green LED will light up if it determines that there is a person.

The inference results are also output to the Arduino serial monitor, which appear as follows:

Code Reference and Q&A

• More information on TensorFlow Lite for Microcontrollers can be found at: https://www.tensorflow.org/lite/microcontrollers
• For more examples, please visit our official website: Realtek IoT/Wi-Fi MCU Solutions
• Please join our FB Forum (English version) to further discuss with us: Ameba IoT Facebook Group
• Or join our forum to bring up any questions: Realtek Ameba IOT Developers Forum