The PowerPC Has Still Got It
What good is a 2005 PowerBook G4 in this day and age? Not much, unless you want to run a modern large language model in style, that is.
For most people, the term “Apple silicon” brings to mind powerhouse processors like the M4 Max. Since Apple went through a lengthy Intel phase prior to the development of their M-series chips, it is often assumed that these are their first custom processors. But twenty years ago, Apple had different custom silicon in their computers — PowerPC microprocessors.
The advantages of these earlier chips were not as clear cut as the M-series chips. Diehard Apple fans swore that they were superior, while the PC crowd wouldn’t touch them with a ten-foot pole. But in any case, they are a couple decades old at this point, so they do not have a lot of gas left in the tank. However, Andrew Rossignol does not believe that the tank is empty just yet. Rossignol recently demonstrated that a PowerBook G4 from 2005 is capable of getting in on the action of running modern artificial intelligence (AI) algorithms — with some caveats, of course.
Process different
Rossignol, a vintage computing enthusiast, successfully ran a large language model (LLM) on a 1.5GHz PowerBook G4, a machine with just 1GB of RAM and a 32-bit processor. The experiment used a fork of llama2.c, an open-source LLM inference engine originally developed by Andrej Karpathy. Given the hardware constraints of the PowerBook, Rossignol chose the TinyStories model, a relatively small model with 110 million parameters that was designed specifically for generating simple short stories.
To make this work, Rossignol had to modify the original software to accommodate the PowerPC’s big-endian architecture, which differs from the little-endian format that most modern processors use. This involved converting model checkpoints and tokenizer data to the appropriate format, ensuring that numerical data was processed correctly. Additionally, the memory alignment requirements of the aging PowerPC chip meant that weights had to be copied into memory manually, rather than being memory-mapped as they would be on an x86 system.
Well, technically it works
Performance was, predictably, not so good. Running the model on an Intel Xeon Silver 4216 processor achieved a processing speed of 6.91 tokens per second. The same model on the PowerBook G4, however, managed just 0.77 tokens per second — taking a full four minutes to generate a short paragraph of text.
To improve performance, Rossignol leveraged AltiVec, the PowerPC’s vector processing extension. By rewriting the core matrix multiplication function using AltiVec’s single instruction, multiple data capabilities, he was able to increase inference speed to 0.88 tokens per second — a modest improvement, but you have to take what you can in a project like this.
Despite the slow performance, the fact that a 20-year-old laptop could successfully run a modern AI model at all is impressive. The PowerBook’s outdated architecture, limited RAM, and lack of specialized accelerators posed a number of challenges, but careful software optimizations and a deep understanding of the hardware allowed Rossignol to push the system well beyond its expected limits.
