Edge-ucating AI Algorithms

The massive amount of computational resources some cutting-edge machine learning algorithms require is becoming the stuff of legends. When a major corporation is talking seriously about spinning up their own nuclear power plant to keep their data centers humming along, you know that some serious hardware is involved. But these eye-popping examples are by no means the most important use cases for artificial intelligence (AI). In fact, in the grand scheme of things, they may ultimately prove to be little more than a passing fad.

All of the resources and energy consumption associated with these applications has driven the costs of running them sky-high, which has made the path to profitability elusive. Furthermore, when processing has to take place in a remote data center, it introduces latency into applications. Not only that, but do you really know how the data is being handled in a remote data center? Probably not, so sending sensitive data to the cloud can raise some major red flags as far as privacy is concerned.

The future of AI is likely to head in a more efficient direction, in which algorithms run directly on low-power edge computing devices. This shift will slash costs while also enabling secure applications to run in real-time. Of course getting to this future will be challenging — a complex algorithm cannot simply be loaded on a tiny platform, after all. One of the difficulties we will have to overcome is on-device training, which is something a pair of researchers at the Tokyo University of Science is working on.

Without on-device training, these tiny AI-powered systems will not be able to learn over time or be customized to their users. That doesn’t sound so intelligent, now does it? Yet training these algorithms is more computationally intensive than running inferences, and running inferences is hard enough as it is on tiny platforms.

It may be a bit easier going forward, however, thanks to the researchers’ work. They have introduced a novel algorithm called the ternarized gradient binary neural network (TGBNN), which has some key advantages over existing algorithms. First, it uses ternary gradients during training to optimize efficiency, while retaining binary weights and activations. Second, they enhanced the Straight Through Estimator to improve the learning process. These features greatly reduce both the size of the network and the complexity of the algorithm.

The team then implemented this algorithm in a computing-in-memory (CiM) architecture — a design that allows calculations to be performed directly in memory. They developed an innovative XNOR logic gate using a magnetic tunnel junction to store data within a magnetic RAM (MRAM) array, which saves power and reduces circuit space. To manipulate the stored values, they used two mechanisms: spin-orbit torque and voltage-controlled magnetic anisotropy, both of which contributed to reducing the circuit size.

Testing their MRAM-based CiM system with the MNIST handwriting dataset, the team achieved an accuracy of over 88 percent, demonstrating that the TGBNN matched traditional BNNs in performance but with faster training convergence. Their breakthrough shows promise for the development of highly efficient, adaptive AI on IoT edge devices, which could transform applications like wearable health monitors and smart home technology by reducing the need for constant cloud connectivity and lowering energy consumption.