Smart Home Pet Surveillance on BeagleBone AI-64

What is AI and how is it used?

AI is integrated into existing solutions for added functionality. A surveillance camera can be smarter with edge AI functionality by implementing automatic object detection, adaptive zoom of interest and real time decision making. Traditional vision analytics uses pre-defined rules to solve tasks such as object detection and facial recognition. As opposed to these conventional computer vision methods, deep learning employs neural networks to process the images.

Object Detection Models

Object detection is an important problem in Computer Vision. These computer vision models are designed to identify multiple objects simultaneously and can output the location of each detected object as bounding boxes along with corresponding class labels. For this project, we will be using a YOLOX with MMDetection model trained on the COCO dataset. COCO data sets are commonly used for large scale object detection. YOLOX models are an excellent set of models with low complexity and high accuracy. MMDetection is a popular training framework for object detection and an open-source toolbox based on PyTorch.

For more information about the fundamentals of AI development with hands on coding projects for beginner and advanced users, I recommend viewing TI’s free Edge AI Academy.

Platform: BeagleBone AI-64

The BeagleBone AI-64 is a 64-bit Single Board Computer (SBC) with two Cortex-A72 microprocessors. This board has a complete system for developing AI and machine learning solutions with the convenience and expandability of the BeagleBone platform and the peripherals on board to get started right away learning and building applications.

Introduction

Having a cat with a condition that causes them to consume inedible objects means I must keep a more watchful eye for their safety. My cat wears a collar with a bell which helps me keep track of where he is in my apartment, but I often feel like I’m having to leave a room just to see what he is up to in the other room. I had my hands on a BeagleBone AI-64 board so I decided to set up a surveillance system that uses pretrained models to detect, identify and monitor my cat as he comes into frame. My cat often gets into places he shouldn't, and it is difficult to keep an eye on him since I live alone so I set up an IMX219 camera sensor pointed at the area my cat often tries to sneak around at to make my life easier and keep him safe.

This project is based on the following instructions:

• BeagleBoard — Getting Started
• Overview — Processor SDK Linux for SK-TDA4VM Documentation

Board Setup

1. Download the latest image for BBAI-64 (BeagleBone AI-64): beagleboard.org/latest-images

Full instructions for all beagleboards, including the BBAI-64 we are using, can be found here: beagleboard.org/getting-started

2. Use etcher tool such as BalenaEtcher to install the image to microSD Card.

3. Once flashed, put the microSD card in the BBAI-64 and the USB-C and ethernet cables in their respective ports

4. Connect to a HDMI display using HDMI cable and an active mini DisplayPort to HDMI adapter for the mini DisplayPort on the board

5. Connect your camera via USB port or CSI port

6. Optional – connect your keyboard and mouse to the board to use the terminal

Software Setup – Out of Box:

1. After booting prompt is finished you will sign in with your debian as your username and temppwd as your password. This is the default, but you can change it which is recommended. Now to view the edge AI pages run the command sudo su. You will now see your directory change to opt/edge-ai-apps/.

1a. There are many applications in the default Debian image but specifically, we want to look at the edge AI applications that come from Texas Instruments. You'll find those in the opt/edge-ai-apps/ directory. You’ve got apps in written in C++ as well as a python. These applications help you utilize GStreamer to create your pipelines. For more information on these dataflows and pipelines I recommend viewing this page. To get a feel for how applications such as object detection are run, view this page. The unaltered object_detection.yaml file will be using pre-recorded video, but the inference is being done in real time utilizing the accelerators and then rendered through the GStreamer pipeline.

2. Going into the model_zoo directory and looking at the dataset.yaml file under the ONR-OD-8200-yolox-nano-lite-mmdet-coco-416x416 directory, I was able to see that “cat” was listed as a detectable object. In order to have the application detect only “cat” data, I added an if condition to the overlayBoundingBox function in the post_process_image_object_detect.cpp file which is responsible for implementing the image-based object detection post-processing logic. This file is found in the following directory: /opt/edge_ai_apps/apps_cpp/common/src/post_process_image_object_detect.cpp

3. After you have made these changes, you must be re-built on the target using the following steps:

root@BeagleBone-AI64: /opt/edge_ai_apps/apps_cpp# rm -rf build bin lib
root@BeagleBone-AI64: /opt/edge_ai_apps/apps_cpp# mkdir build
root@BeagleBone-AI64: /opt/edge_ai_apps/apps_cpp# cd build
root@BeagleBone-AI64: /opt/edge_ai_apps/apps_cpp/build# cmake ..
root@BeagleBone-AI64: /opt/edge_ai_apps/apps_cpp/build# make -j2

4. You may edit the existing object_detection.yaml file found in the configs directory or create your own.yaml file. Follow this page for detailed instructions on how to format this file.

4a. If using IMX219/RPiV2 sensor in the CSI0 slot, you must apply the BBAI64-CSI0-imx219.dtbo to /boot/firmware/extlinux/extlinux.conf as such:

label Linux eMMC
kernel /Image
append root=/dev/mmcblk0p2 ro rootfstype=ext4 rootwait net.ifnames=0 quiet
fdtdir /
fdtoverlays /overlays/BBAI64-CSI0-imx219.dtbo
initrd /initrd.imgb.

4b. Run v4l2-ctl --list-devices or run init_script.sh to verify your camera has been detected. Take note of the /dev/video number as well as the subdev_id for IMX219 sensor and make sure it matches the number you have as your input source in your.yaml file.

4c. You have the option to view the inference live on your display, save to a file or view remotely.

4d. C++ files are cross compiled while packaging. Run your application as such:

root@BeagleBone-AI64: /opt/edge_ai_apps/apps_cpp# ./bin/Release/app_edgeai ../configs/object_detection.yaml

Going further

If you plan on creating a similar project but require a custom model, a great option would be to use Edge Impulse studios. Edge Impulse is a platform that makes AI model development easy for a number of processors including Texas Instruments TDA4VM,AM62A and AM68A processors.