Controlling a Display of Lights with Fun Poses

The idea

A couple of months ago, Lorraine from element14 Presents designed and built an interactive window project that used a series of capacitive touch pads stationed in her front lawn which could be touched to modify the current color of the LEDs in the display. However, in her newest iteration of the project, she wanted to expand upon this idea by making the entire system touch-less and usable from a distance so that it is more convenient to interact with.

A previous design

The previous system was comprised of two main parts. The first and main component was a micro:bit that was stuffed inside a waterproof box and had three capacitive buttons run along the outside that could be touched. After a user interaction had occurred, data would be transmitted along several underground wires to a Raspberry Pi positioned just inside the house, and its job was to control two sets of RGB lights. Unfortunately, this system turned out to be quite unreliable and would give inconsistent responses, especially during inclement weather.

Components required

In order to improve upon the previous design, Lorraine started by determining which components could stay and which ones needed to be replaced. Her initial thought was to incorporate a time-of-flight sensor into the waterproof housing and utilize the preexisting micro:bit/Raspberry Pi setup to send data when an interaction happens. To go with the VL53L0X ToF module, she also picked out the Raspberry Pi NoIR Camera V2 module for computer vision when it's dark outside.

Using time-of-flight

After loading the MakeCode visual coding development environment for the micro:bit and connecting the VL53L0X via I2C to the board, Lorraine created a quick program that constantly reads data from the sensor and logs it via UART. Finally, two thresholds were added that make the micro:bit only send data to the Pi when a user is within a predetermined range, which cuts down on the amount of unnecessary data transfers. On the Pi's side, three separate thresholds were created that switch between red, green, and blue depending on the current distance value.

Detecting poses

Although this somewhat worked, the readings were quite inconsistent and had a large variation, so a plan B was made that incorporated the camera module and computer vision to control the lights instead. A switch to the Ubuntu operating system was required in order to get pose estimation working correctly, and after replacing the sadly broken Pi NoIR camera with a second one, the Pi could successfully see poses.

Party on the front lawn

In element14's demonstration video/guide, you can see how Lorraine could raise her right hand to turn the lights red or raise her left hand to turn them blue. However, the party mode, which is triggered by raising both hands simultaneously, is probably the best one, as it rapidly alternates between red and blue for a strobe light effect. To see the bill of materials list and code used in this project, you can visit its blog post here on element14.