Eivind Holt's Portenta H7-Powered TinyML Camera Tracks Dangerous Icicle Formations
Trained using simulated data, rendered as realistically as possible, this low-power gadget aims to keep pedestrians safe in colder climes.
Software engineer and self-described Internet of Things (IoT) enthusiast Eivind Holt has put some artificial intelligence to work in protecting pedestrians from falling icicles — by monitoring their formation with an Arduino Portenta H7 and its Vision Shield accessory.
"The portable device created in this project monitors buildings and warns the responsible parties when potentially hazardous icicles are formed," Holt explains. "In ideal conditions, icicles can form at a rate of more than 1cm (0.39in) per minute. In cold climates, many people are injured and killed each year by these solid projectiles, leading responsible building owners to often close sidewalks in the spring to minimize risk. This project demonstrates how an extra set of digital eyes can notify property owners icicles are forming and need to be removed before they can cause harm."
The heart of the project, brought to our attention by the Arduino blog, is an Arduino Portenta H7 development board with the Vision Shield — LoRa accessory, which adds both a camera and a LoRa transceiver for low-power long-range communication. "A powerful platform combined with a high resolution camera with fish-eye lens would increase the ability to detect icicles," Holt admits. "However, by deploying the object detection model to a small, power-efficient, but highly constrained device, options for device installation increase. Properly protected against moisture this device can be mounted outdoors on walls or poles facing the roofs in question."
The icicles are detected using a neural network developed using Edge Impulse Studio and its FOMO (Faster Objects, More Objects) facility for resource-constrained devices. For a training dataset, lacking enough labelled real-world images of icicles hanging from roofs, Holt turned to NVIDIA's Omniverse Replicator — creating a synthetic dataset built using 3D models of a house and icicles, rendered as realistically as possible and with randomized wall colors, sizes, positions, and weather conditions.
When icicles are detected by the device — which automatically captures and analyzes images on-device on a user-configurable schedule — it uses the LoRa transceiver to connect to the community-driven The Things Network over LoRaWAN in order to transmit its findings. These are then processed in The Things Stack to produce MQTT messages, which can be subscribed to for alert messaging — or to trigger a physical response, such as a heating system or warning lights.
More information on the project is available on the Edge Impulse website, while source code has been published to GitHub under the reciprocal GNU General Public License 3.