Haunted Halloween Picture

When Pete Warden (of TensorFlow Lite Micro fame) announced that he had started a new company developing affordable TinyML sensors, I was excited. When he sent me a sensor to try out prior to full-scale release to test out, I was BEYOND excited!!

Pete sent me a small (yet powerful) face identification board that identifies faces, then sends the bounding box location of the faces within the frame via an I2C interface. All the inferencing is done directly on the board, so the only information that is sent externally from the board is the metadeta. This results in a secure board that prioritizes privacy and anonymity, making it an excellent candidate for public and wide-spread applications. The board also has a cool feature of being able to assign IDs to faces it has previously recognized, allowing for personalized logic while still maintaining privacy! The board runs on 3.3V and runs at about 150 mW, although the team is working on improving that to make it more power efficient. It uses a Quiic/Stemma QT connector for the I2C connection.

I wanted to test my skills and the board and see what I could come up with. Given that we are about a month from Halloween, I decided to make a Haunted Picture Frame! The premise of the project is to conceal the face ID board within an old looking/somewhat creepy picture in a frame. If someone walks in front the frame, the picture says in a spooky voice: "Hey, who are you? I don't know you!". It then listens for a response. If it doesn't recognize you, it asks the question again. If you respond with the name associated with the ID of the face captured by the sensor, it will respond in a friendly tone. Check out the demo below:

For this project, I built off of a prior project I did making Darth Vader listen and talk using the Syntiant Tiny ML board. The major difference this time was adding the face ID sensor via I2C. Once again Atul Gupta from Syntiant was supportive in getting the I2C interface to the board up and running. Once I was able to confirm the I2C connection (running an I2C scan is very helpful before running any code), I was ready to start coding!

The essential flow of the code (developed in the Arduino IDE) is to set the appropriate pins, address, and registers to establish the I2C connection. In the loop() method, an I2C read() function is called to read the metadata from the face ID sensor. If a face is detected, it should have an ID of -1, indicating that no ID is associated with that face. The picture will then ask "Hey, who are you?" Currently, the audio recognition model I have only recognizes my name, so that logic is hard-coded in, but I could eventually expand the model and code for additional members of my family. If the name is recognized, a register is set to start tracking IDs of the face it recognizes.

Overall, it is a fairly straightforward interface, which I think is the goal of Pete and the Useful Sensors team: send all the results though I2C, so you can run inference on even the most-constrained of boards (my first successful demo with the face ID board was on an original Arduino Uno R3).

For the Edge Impulse model, I followed a similar process flow as my Darth Vader project. Instead of classifying 3 objects though, I just classified 2 (my name and the z_openset). If you have any questions on how that works, check out that Hackster.io project and the public project on Edge Impulse.

This project was a lot of fun. It was great to try out a new piece of hardware and I'm excited to develop new (and hopefully more useful) use cases in the future!