In a stunning demonstration that challenges conventional thinking about artificial intelligence (AI) and hardware requirements, EXO Labs has managed to run a modern Llama 2 AI model on a Windows 98 Pentium II machine, which was built over 25 years ago. This experiment proves that AI doesn’t have to be confined to high-performance data centers. By leveraging outdated technology, EXO Labs has shown that cutting-edge AI can run even on hardware that was once considered obsolete.
The Journey to Get Llama Running on Vintage Hardware
The project, led by Andrej Karpathy and his team at EXO Labs, started with a Windows 98 Pentium II machine they purchased for £118.88 on eBay. The first hurdle was making the machine work with modern peripherals. The machine had no USB ports, so they had to use PS/2 peripherals.
Even more challenging, the mouse and keyboard had to be connected to specific ports—mouse in port 1 and keyboard in port 2, otherwise the setup wouldn’t work. Despite these issues, the team persevered, and the machine began to take shape as a functional unit for their experiment.
Next, they faced the challenge of transferring files to the old machine. Modern solutions like USB drives were either incompatible with the outdated operating system or too large to work with FAT32.
The team resorted to using FTP (File Transfer Protocol) to transfer necessary files, such as model weights, tokenizer configurations, and inference code. By connecting the Windows 98 machine to a MacBook Pro using a USB-C to Ethernet adapter, they successfully set up static IP addresses, enabling FTP transfers.
Overcoming Compilation Hurdles
After transferring the necessary files, the team ran into another challenge: compiling modern code for Windows 98. They first attempted to use mingw (Minimalist GNU for Windows), but this approach failed because the older Pentium II processor didn’t support certain modern instructions.
Instead, the team turned to Borland C++ 5.02, a 26-year-old integrated development environment (IDE) that worked directly on Windows 98. Although Borland C++ 5.02 only supported older versions of C programming, the team was able to make the necessary code adjustments.
The modifications were crucial: they replaced certain types with DLONGWORD, moved variable declarations to the start of functions (a limitation of older C), and simplified disk-to-memory loading to avoid crashes.
This was where Andrej Karpathy’s llama2.c came in—a 700-line C code that could run inference on models based on the Llama 2 architecture.
A Historic AI Run on 1997 Hardware
With the code and hardware finally working together, the team was able to successfully run the Llama 2 model. The 260K parameter model achieved 39.31 tokens per second on the Pentium II. The team also tested a larger 15M parameter version, which ran at just 1.03 tokens per second.
The 1B parameter version of Llama 3.2 showed even slower results at 0.0093 tokens per second, based on the partial model run with data stored on disk.
Although these speeds are far from ChatGPT levels, running any modern AI on a 25-year-old machine is a groundbreaking accomplishment.
The team attributed the success to the lightweight BitNet architecture, which simplifies AI models by using ternary weights (-1, 0, 1), significantly reducing computational demand.
The Future of AI: BitNet and Beyond
The success of this experiment is not just a demonstration of the power of old hardware. It also shines a light on the potential of BitNet, a new transformer architecture being developed by EXO Labs.
BitNet utilizes ternary weights to reduce the amount of data needed for AI models, making it possible to run large-scale models on low-end hardware like the Pentium II.
With BitNet, a 7B parameter model requires just 1.38 GB of storage, which is small enough to fit on many older computers. Furthermore, BitNet is designed to be CPU-first, meaning it doesn’t require high-end graphics processing units (GPUs).
In tests, BitNet has proven to be more energy-efficient and computationally efficient than traditional models, using over 50% less power.
In the future, BitNet models could run on even older machines or lower-power CPUs, offering significant possibilities for democratizing access to AI. EXO Labs is already pushing this boundary by working on larger models for applications such as protein modeling.
Note: This article is inspired by content from https://indiandefencereview.com/someone-experimented-with-a-1997-processor-and-showed-that-only-128-mb-of-ram-were-needed-to-run-a-modern-ai/. It has been rephrased for originality. Images are credited to the original source.