MLCommons Releases Latest MLPerf Tiny Benchmark Results for On-Device TinyML

Artificial intelligence engineering consortium MLCommons has announced a new set of results from MLPerf Tiny, its tiny machine learning (tinyML) and edge artificial intelligence (edge AI) benchmark — offering a look at how devices from companies including Renesas, STMicroelectronics, and Syntiant stack up.

"We are pleased by the continued adoption of the MLPerf Tiny benchmark suite throughout the industry," says MLCommons executive director David Kanter on the occasion of the latest set of results from benchmark participants. "The diversity of submissions shows us that the industry is embracing AI through increased software support, which makes our benchmarking work all the more important."

More properly termed the MLPerf Inference: Tiny benchmark, MLPerf Tiny is an off-shoot of the larger MLPerf benchmark family that includes both training and inference benchmarks and targets high-performance hardware. MLPerf Tiny, by contrast, aims to deliver a look at the performance — and, thanks to new power efficiency measurements in the latest version, efficiency — of resource-constrained microcontrollers and the accelerators with which they can be paired.

The new results include parts from Bosch, Qualcomm, Renesas, STMicro, Skymizer, and Syntiant, measuring their ability to run benchmark neural networks typically measuring under 100kB and designed to process sensor data including audio and video. Not all companies, however, completed all aspects of the benchmark — with STMicro standing alone in not only running through the full benchmark suite but also providing both performance and energy consumption figures for Arm Cortex-M4, Cortex-M7, and Cortex-M33 Nucleo boards.

The only other company to provide energy usage figures as well as performance figures was Syntiant, which submitted result for its NDP120 neural decision processor with an Arm Cortex-M0 core and a HiFi3 digital signal processor (DSP) — delivering considerable energy savings over STMicro's parts running without a dedicated accelerator core.

The new MLPerf Tiny results come after MLCommons launched a proof-of-concept for a new benchmark targeting large language models (LLMs) and other generative AI systems, aiming to deliver measurements of their safety rather than their performance. While results from "a variety of publicly available AI systems" were included in the launch, they were anonymized — pending the launch of a 1.0 version of the benchmark later this year.

The full results are available now on the MLCommons website.