Bird@Edge Turns Birdsong Into Species Recognition On-Device, Using an NVIDIA Jetson Nano

Researchers from the University of Marburg have developed an interconnected system that aims to recognize different bird species from sound alone — using a machine learning model running on an NVIDIA Jetson Nano and being fed audio data from Espressif ESP32-based remote microphone modules.

"Our experiments show that our deep neural network outperforms the state-of-the-art BirdNET neural network on several data sets," the research team writes of its work, "and achieves a recognition quality of up to 95.2 percent mean average precision on soundscape recordings in the Marburg Open Forest, a research and teaching forest of the University of Marburg, Germany." That's impressive enough, but doubly so considering the entire system runs on-device at the edge — hence its name, Bird@Edge.

Brought to our attention by NVIDIA, Bird@Edge is built from two distinct devices: remote microphone sensor nodes and a central processing system. The nodes are built from Espressif ESP32 microcontrollers connected to a Knowles SPH0645LM4H microphone and powered by USB power banks or a single 18650 lithium-ion cell. The central processor is an NVIDIA Jetson Nano, connected to a USB Wi-Fi dongle to create a wireless network for the sensor nodes and to feed recognition activity into a database used by a web-based user interface.

The reason for picking the NVIDIA Jetson Nano over something like a Raspberry Pi is simple: its suitability for on-device machine learning, boasting a powerful graphics processor which can execute the team's TensorRT model. This model is based on the EfficientNet-B3 architecture, which recognizes the audio based on Mel spectrogram visualizations — a neat trick to apply a machine learning system which is designed to work on visual data to audio instead.

In testing in the Marburg Open Forest, a research and teaching forest maintained by the University of Marburg, the team found that Bird@Edge achieved a recognition quality of 95.2 percent mean average precision while drawing just 3.18W for the NVIDIA Jetson Nano and an additional 0.5W per each microphone module.

A copy of the team's paper is available as an open-access PDF download, while the project's software and firmware sources have been published to GitHub under an unspecified open source license.