M5Stack Adds Large Language Model Support to Its Offerings with the 3.2 TOPS LLM Module

Modular electronics specialist M5Stack has announced its latest hardware release, a module that aims to add large language model (LLM) artificial intelligence (AI) to your builds: the sensibly-named M5Stack LLM Module.

"[The LLM Module] is an integrated offline large language model (LLM) inference module designed for terminal devices that require efficient and intelligent interaction," M5Stack says of the hardware in question. "Whether for smart homes, voice assistants, or industrial control, Module LLM provides a smooth and natural AI experience without relying on the cloud, ensuring privacy and stability."

Aiming to ride the current boom in LLM technology, in which user queries are broken down into "tokens" and the most likely tokens returned in response to create an answer-shaped object that may, if everything goes well, also be correct, the LLM Module — brought to our attention by Linux Gizmos — is powered by an Axera AX630C system-on-chip. This combines two Arm Cortex-A53 cores running at up to 1.2GHz with an in-house neural processing unit (NPU) delivering 3.2 tera-operations per second (TOPS) of compute at INT8 precision rising as high as 12.8 TOPS if you drop to INT4 precision.

This, combined with 3GB of memory dedicated to the NPU with 1GB left for the operating system installed on a 32GB eMMC module, is enough to run smaller large language models entirely on-device — while drawing, the company claims, as little as 1.5W. The module includes an integrated microphone with wake-word and speech recognition models pre-loaded, and a speaker to serve as an output through an integrated text-to-speech model. The eMMC comes with an unspecified version of Canonical's Ubuntu Linux pre-loaded, and can be upgraded through a microSD Card slot.

On the software front, M5Stack says the module is compatible with multiple large language models — featuring the Qwen2.5-0.5B model out-of-the-box, a compact LLM with 500,000 parameters tweaked for edge AI operations. The company has promised that future updates will bring support for the more capable Qwen2.5-1.5B model, three times the size of the launch model, as well as Llama3.2-1B and InternVL2-1B. If connected to a USB camera, the module also supports computer vision models including CLIP and YoloWorld at launch with DepthAnything, SegmentAnything, "and other advanced models" to follow in future updates.

The M5Stack LLM Module has been listed on the M5Stack store at $49.90, though at the time of writing was showing as out-of-stock; the company has also announced an "LLM debugging kit" that adds a Fast Ethernet network port and a dedicated kernel serial port, pricing for which has not yet been announced. The module is compatible with the company's Core, Core2, CoreS3, and Core MP135 development boards.