How Does Your TinyML Stack Up?

Machine learning benchmarks provide consistent measurements of accuracy, speed, and efficiency. The consistency of these measurements give engineers a tool to design reliable algorithms, and also empower researchers to choose the best technique from a set of many possible alternatives.

MLCommons, an open engineering consortium, has just released a new benchmarking tool called MLPerf Tiny Inference that was designed from the ground up to assess the performance of neural networks running on extremely low-power embedded devices. This release comes as embedded machine learning, or tinyML, is experiencing a boom in popularity as compute resources capable of running machine learning algorithms have grown smaller and less expensive. These tiny neural networks can process audio, video, and other sensor data to provide rapid inference times without the privacy implications of sending that data to the cloud for processing.

The MLPerf Tiny Inference software offers reporting and also comparisons of tinyML devices, systems, and software. In particular, the benchmark measures four tasks that make use of microphones and camera sensors: keyword spotting, visual wake words, tiny image classification, and anomaly detection. Keyword spotting has numerous applications in voice control applications. Visual wake words can keep devices in a low-power state until some visual cue is detected. Image classification has any number of applications in video recognition, while anomaly detection is often used in monitoring industrial manufacturing processes.

MLPerf Tiny Inference is a tool that will help us to add intelligence to everyday items, from wearables to thermostats and cameras. As this software suite was developed in collaboration with over fifty partners in industry and academia, it reflects the needs of the community at large.

The benchmark suite is open source and available for download at GitHub.