AI for Every Device

The latest wave of artificial intelligence (AI) tools has caught the world off guard. After many years of developments that were of little interest to anyone outside of a research lab, truly useful applications have finally emerged. Large language models, text-to-image generators, and a variety of predictive models have genuinely made us (and machines) more efficient and productive, and in a big way.

Bigger isn't always better

Unfortunately, our productivity is not the only thing that has grown — the massive AI models that power these tools are gobbling up computing resources and energy at an accelerating pace. This has led to a whole host of problems — it is hard for developers to build a profitable product, it makes the tools inaccessible to most people, and it is damaging to the environment. We are basking in the glow of these new technologies at the moment, but when the shiny finish wears off, we may find that it is impractical to continue building and operating such tools.

The research community is, of course, aware of this situation, so many efforts are underway to optimize algorithms and build specialized hardware such that AI tools can run on much less powerful platforms. Much progress has been made in this direction, but even after getting an algorithm’s computational complexity under control, there is still the issue of deployment remaining. When it comes to low-power platforms like microcontrollers, there are hundreds of options out there. How in the world can we ensure that an algorithm can run on any — or at least a large subset — of these devices?

A solution for the least of them

The complexities rapidly multiply when we step outside of the relatively uniform world of GPU computing. But a new project called Zant is seeking to rein in this complexity (at least from the perspective of a developer) and make deployments of AI models to a wide range of hardware platforms trivial.

Zant, formerly known as Zig-Ant, is an open source software development kit designed to simplify the deployment of neural networks on microcontrollers. Developed in Zig, a modern programming language known for its performance and safety, Zant eliminates external dependencies while prioritizing cross-compatibility and efficiency. Unlike many AI platforms that focus on model creation, Zant’s primary goal is deployment.

Many microcontrollers, including the widely used ATmega, TI Sitara, and Arm Cortex-M families, lack robust deep learning libraries. The AI deployment landscape is fragmented, with many solutions being tailored to specific hardware, which makes portability and efficiency difficult to achieve. Zant provides an end-to-end solution for neural network optimization and deployment, ensuring that AI models run efficiently on low-power devices without requiring extensive modifications.

Zant leverages hardware acceleration techniques such as SIMD operations, memory caching, and static allocation to maximize performance. It also makes efficient use of memory through memory pooling, buffer optimization, and static allocation, ensuring that even devices with limited resources can handle AI workloads. Furthermore, the framework allows developers to deploy models across different hardware platforms without modifying the core codebase. And Zant’s modular design, APIs, and comprehensive documentation make integration into existing software stacks — whether written in C, C++, or another language — straightforward.

The project is still under development, so we can expect that it will improve over time. At present, the team is working on relatively simple goals, like getting an MNIST classifier to run on a Raspberry Pi Pico 2. But in the short-term they are also working to get YOLO running on the same platform, which would be a significant achievement. If you want to try Zant out for yourself, check out the getting started guide on GitHub.