Published May 13, 2024 © MIT

Step Guard 3.0

It all started with a simple distance sensor, now drunks are detected with AI, reported with BLE and dispersed with dog barks.

IntermediateFull instructions provided14 hours362

Things used in this project

Hardware components

Seeed Studio XIAO ESP32S3 Sense

Seeed Studio Grove Vision AI Module V2

DFRobot UNIHIKER - IoT Python Programming Single Board Computer with Touchscreen

Raspberry Pi Camera Module

Software apps and online services

Autodesk Fusion 360

Arduino IDE

Story

Version 3 with AI detection, dog mode and BLE link

A few months ago, I embarked on a mission to dissuade noisy drunks from loitering on my doorstep. My initial system relied on an outdoor ESP32-based distance sensor, which communicated with an indoor Linux/Python Unihiker. When the doorstep remained occupied for a specified duration, a notification was dispatched to a Telegram group, and a sequence of sounds escalated—from a simple alert to LRAD-type tones.

While the system demonstrated some effectiveness (videos of our “beta testers” are available), I wasn’t entirely satisfied with the detection procedure. On one occasion, the system reported activity, only to discover it was a promotional flyer with scotch tape obstructing the distance reading.

I had a Grove Vision AI V 2 waiting in the desk and I thought it was time for an AI upgrade.

Grove Vision AI V 2

SenseCraft page to load and test the model.

The Grove Vision AI Module V2, a compact device developed by Seeed Studios, harnesses the power of Arm Cortex-M55 & Ethos-U55. It seamlessly integrates with TensorFlow and PyTorch frameworks and is fully compatible with the Arduino IDE. Trained models can be effortlessly tested and deployed to the module using a simple web browser.

My journey began by connecting an Rpi 1.3 camera and a USB-C cable. Navigating to SenseCraft-Web-Toolkit, I selected the Grove Vision AI 2 and established a connection. With ease, I chose a pre-trained model for person detection and deployed it to the board.

Grove Ai Vision V2

Next, I linked the XIAO ESP32S3 to the Grove Vision module. Installing the Seeed_Arduino_SSCMA library, I obtained immediate inferences.

However:

Issue #1 No Serial Communication from Xiao ESP32S3: After some troubleshooting, I discovered that adjusting the baud rate to 460800 allowed the serial monitor to receive data from the board.

Issue #2 Upload Errors: Despite several successful uploads, an ominous message appeared: “No serial data received. Failed uploading: uploading error: exit status 2.” Fortunately, Seeed Studios’ support swiftly provided a solution—simply press the boot and reset buttons. The bad part is that this step was required from this point for all the code updates and boot and reset are tiny buttons that you barely see them, leave aside pressing them. I was able to do it with the help of 2 Pencils. Anyway… back on track.

Here are the IDE Settings just in case

BLE

ESP32 BLE Code

I could have used also WiFi but the sensor unit and the router are far away, so BLE to the Indoor unit seemed the best choice. I have configured ESP32S3 as a BLE Server, then I have added a characteristic that switch from "Clear" to "Detected" according to AI person detection.

Outside unit ESP32S3 and Grove AI module

From Python side, I have used the bluepy library to connect to the outside unit and receive the characteristic updates.

Python Unihiker Code

Other changes

Dog mode

I have also added Telegram notifications from the Unihiker with a direct requests.post and the ability to configure dog barks for the first step of sounds triggered. Then a recorded message “Clear the entry” and last, an annoying loud sound.