Developer runs 28.9M-parameter language model on $10 ESP32-S3 microcontroller

When discussions turn to running local AI these days, they typically center on mini-PCs like the RTX Spark or drift into aspirational talk about home servers and prohibitively expensive professional GPUs. But one of the most impressive AI hardware tricks in recent memory just landed on a piece of silicon that costs less than a decent burger.

Last week, a Ukrainian developer named Slava S — known as 'slvDev' on GitHub — released a project called ESP32-AI. He successfully ran a 28.9-million-parameter language model locally, entirely on-device, on an ESP32-S3 microcontroller.

The Hardware: Tiny, Cheap, and Surprisingly Capable

ESP32-S3 boards offer some of the best bang for buck in the computing world:

• Price: Under $20 (with protective case) in the U.S.; bare boards available for under $10 globally
• Specs:
• 512KB SRAM
• 8MB PSRAM
• 16MB flash memory

As you'd expect from a chip this inexpensive, it is not powerful. The total memory is less than what a single raw photo from a smartphone consumes.

The Challenge: Fitting a 30M-Parameter Model Into a Chip With Almost No Memory

To run an LLM, the entire model must typically reside in fast memory, as the processor constantly performs calculations against every parameter to generate the next word. Attempting to run a 29M-parameter model normally on an ESP32 exhausts the fast RAM instantly.

The previous record for a chip like this was approximately 260,000 parameters, achieved by Dave Bennett, as Slava noted on X.

The Breakthrough: Borrowing Google's Architecture

Slava overcame this bottleneck by adapting a technique from Google's Gemma architecture called Per-Layer Embeddings.

His approach:

• Quantized the model down to 4-bit precision, reducing the total file size to just 14.9 MB
• Reorganized data storage: Instead of attempting to fit everything into the tiny 512KB SRAM or the 8MB PSRAM, he placed the 25-million-parameter embedding table into the slower 16MB flash memory
• Preserved fast memory for reasoning: Because this model architecture only needs to pull a few rows from the embedding table per token, the flash memory's slower speed does not bottleneck the processor. The 512KB of fast SRAM remains dedicated to the "thinking core" — the actual reasoning weights.

What This Model Can (and Cannot) Do

Let's be clear: this is not a replacement for a server.

• Trained on: The TinyStories dataset
• Classification: A Small Language Model (SLM) — or more accurately, a "micro LM"
• Capabilities: Can only write short, simple, fictional stories
• Limitations:
• Will not answer questions
• Will not follow instructions
• Will not write code
• Possesses zero factual knowledge about the real world

Why This Still Matters

Focusing on those limitations misses the point entirely. The achievement here is fitting a structurally large model onto a computer with practically no resources. It proves that with clever architecture, genuine neural networks can run on dirt-cheap embedded hardware.

Slava envisions practical applications: a coffee machine that actually understands coffee — every bean, grind, ratio, and water temperature — operating entirely offline, with no app required.

The Bigger Picture

When discussing "AI," everything depends on what you are trying to accomplish. Asking how much hardware you need for local AI without specifying the workload is like asking what vehicle you need without stating your goal. A bicycle, a sedan, a pickup truck, a semi-trailer, and a train all "get you from A to B," but they are built for radically different jobs. AI works the same way: what you are doing with it determines how much hardware you need.

Stay on the cutting edge: Get the Tom's Hardware Newsletter — best news and in-depth reviews, straight to your inbox.