UMich Spin-Out MemryX Launches Its Tiny, Low-Power, High-Efficiency MX3 Edge AI Accelerator
Startup promises full-fat models in a 1W "average power" envelope, a four-chip M.2 module, and "one-click" deployment.
Startup and University of Michigan spin-out MemryX has announced the launch of its MX3 edge artificial intelligence (edge AI) accelerator, available as a bare chip or a quad-chip M.2 module for drop-in acceleration — claiming an order-of-magnitude power efficiency gain over its competition.
"The MX3 is a very capable AI inference chip," claims Wei Lu, the James R. Mellor Professor of Engineering at the University of Michigan's Electrical and Computer Engineering department and co-founder of MemryX. "With this AI accelerator, you can reduce the power and financial cost required to run the AI models, and get higher performance. This is all due to the very innovative hardware that we developed initially at the University of Michigan."
The MX3 Edge AI Accelerator, to give the part its full name, is designed to deliver five tera-floating point operations per second (TFLOPS) of compute at Bfloat16 precision. It's capable of handling four, eight, and 16-bit model weights and can store around 10 million parameters on-die — while drawing just 1W "average power" during active use. Its compact size, which Lu claims is "10-100 times smaller" than competing graphics processor cores while delivering improved performance, makes it easy to cram into compact devices — as demonstrated by the company's drop-in accelerator module, which packs four of the MX3 chips on a single M.2 2280 module.
One of the company's M.2 modules can, it claims, run AI models on "tens of incoming camera streams," while automated tools provided with the hardware promise automated pre- and post-processing, the ability to run fully-intact models or optionally prune, compress, or distill a model, and the ability to "compile and execute thousands of AI models with high accuracy with just one click".
"This is really exciting," Lu says of his company's production milestone. "To have a very small company with about 50 people to actually produce a very high quality, high performance AI chip in volume is really remarkable. We already have our first revenue and we have customers, for example, using them in their server racks to process very large amounts of video in real time, using them in factory settings to do this inference at the edge for dozens of different machine learning models and use cases."
More information on the MX3 is available on the MemryX website; pricing is available on application.
Freelance journalist, technical author, hacker, tinkerer, erstwhile sysadmin. For hire: freelance@halfacree.co.uk.