Unleash Your Inner GPU

Really big things have been happening in the world of machine learning lately. Not just at the corporate mega-budget scale, either. Many generative AI tools have been developed and open sourced that often rival the performance of the best of the large language models and text-to-image generators that are available today. These tools do not require massive computational resources to operate or perform acceptably, however, that does not mean that you can necessarily run them on that four-year-old budget laptop in front of you right now.

So as a hobbyist that wants to experiment with the latest advancements in AI, you probably have at least one NVIDIA H100 Tensor Core GPU on your desk now, right? Yeah, me neither. With street prices being in the tens of thousands of dollars plus your firstborn son, they are out of reach for the vast majority of hobbyists — that is, if you could even find one that has not already been snapped up by a startup company trying to cash in on the AI gold rush.

A Reddit user going by the handle chain-77 came up with a trick that gives hope to the hobbyist that wants to play with some of the latest AI on a budget. No, you will not get H100 GPU-like performance, but for under 100 bucks, the performance is very impressive and will supply you with both an education and hours of fun. Rather than doing a shady back alley deal to get the latest in GPU tech, chain-77 demonstrated how you can use a few-year-old CPU to get GPU-like performance on some pretty heavy AI workloads.

The previous-generation, six-core AMD Ryzen 5 4600G CPU with an on-chip GPU was configured by the hacker to utilize 16 GB of DDR4 system memory for the graphics processor. While AMD's Radeon Open Compute platform does not support CPUs, like the Ryzen 5 4600G for compute tasks, third-party developers have released experimental packages that will enable this functionality. With that support, the CPU can be leveraged as a GPU by the most popular machine learning frameworks like PyTorch and TensorFlow. And once you get to that point, virtually any AI workload can run on the chip.

We are still waiting with bated breath for chain-77 to release a promised follow-up video detailing the setup process to make this feat simpler to reproduce, but in the meantime, the Ryzen 5 4600G CPU was demonstrated making short work of popular algorithms like Stable Diffusion, FastChat, MiniGPT-4, Alpaca-LoRA, Whisper, and LLaMA. As previously stated, this hack will not rival the performance of an H100 by a long shot, but it is no slouch either. Generating 50-step images with Stable Diffusion, for example, clocked in at about one minute and 50 seconds. Perhaps not blazingly fast, but well within reason for doing some experimenting and educating yourself about how the latest algorithms work.

Naturally, one could leverage a more recent CPU to get better performance, but then the math might get a bit more complicated. Once you move up to a more expensive CPU, you might actually find that an older GPU gives better performance for your dollar. But if you happen to have an older Ryzen 5 4600G CPU sitting around, or can pick one up for under $100, which they are now commonly available for, this is a great trick to run some cutting-edge tools at home without dipping into your retirement savings.

If you can’t get enough hacks that allow you to run the latest machine learning algorithms on inexpensive hardware, check out what else you can do for around $100 with the OKdo ROCK 5 Model A single-board computer.