AMD Makes It Easier to Run Generative AI, LLMs on Its Ryzen AI 300 Chips with GAIA

AMD is looking to make it easier to take advantage of the machine learning and artificial intelligence (ML and AI) acceleration capabilities built into its latest Ryzen AI laptop and desktop processors to run generative AI (gen AI) large language models (LLMs) locally with the release of GAIA under an open source license.

"AMD has launched a new open-source project called GAIA, an awesome application that leverages the power of Ryzen AI Neural Processing Unit (NPU) to run private and local large language models (LLMs)," claims AMD's AI developer enablement manager Victoria Godsoe. "GAIA is a generative AI application designed to run local, private LLMs on Windows PCs and is optimized for AMD Ryzen AI hardware (AMD Ryzen AI 300 series processors). This integration allows for faster, more efficient processing — i.e. lower power — while keeping your data local and secure."

The explosion of interest — and investment — in large language model technology, which uses vast and often unethically-obtained datasets chewed up into "tokens" to train generative AI models that can take a user's input and regurgitate the most statistically likely response tokens into something which forms the shape of, but should not be confused with, an answer, is showing little sign of slowing. The increasing computational power available on people's desks and in their pockets, though, means the beginnings of a shift away from relying on models running on power-hungry hardware in remote data centers and toward running them locally on the device you already have.

GAIA, whose full name "Generative AI Is Awesome" leaves no uncertainty about AMD's official stance on the divisive technology, is designed to make that process easier — and in doing so pull focus away from rival NVIDIA, whose CUDA-enabled graphics processors have long been the device of choice for running larger machine learning models on-device. Building on existing open source projects like Lemonade, GAIA makes it possible to run models including those based on the freely-available Llama and Phi families locally for use-cases including chatbots and, the company claims, "complex reasoning tasks."

To prove its case, GAIA ships with four "agents": Chaty, a chatbot designed for conversational responses; Clip, a question-and-answer agent with the ability to use YouTube for retrieval augmented generation (RAG); Joker, a RAG-based joke generator; and Simple Prompt Completion, which just provides direct interaction to the underlying base model. Inference takes place on the neural coprocessor of compatible Ryzen AI 300 series chips, though some models also support a performance-boosting "hybrid" mode that brings the chips' integrated graphics processor in the mix. For those on older hardware, meanwhile, a GAIA variant that runs — slowly — on the CPU cores alone is also available.

GAIA is now available to download on GitHub, under the permissive MIT license; AMD has indicated that pull requests for bug fixes or new feature implementations are welcome.