Developer Runs 28.9M-Parameter Language Model on $10 ESP32-S3 Microcontroller

Here is the rewritten news content, structured as clean, professional English Markdown.

Running a 29M-Parameter AI Model on a $10 Microcontroller: A New Low-Cost Milestone

When discussing local AI, the conversation typically centers on mini-PCs like the RTX Spark or expensive home servers with professional GPUs. However, one of the most impressive AI hardware feats in recent memory has been achieved on a chip that costs less than a burger.

Last week, a Ukrainian developer known as "slvDev" on GitHub released a project called ESP32-AI. He successfully ran a 28.9-million-parameter language model locally, entirely on-device, using an ESP32-S3 microcontroller.

The Hardware: Incredible Value, Minimal Resources

The ESP32-S3 offers exceptional value in computing. In the US, a board with a protective case can be found for under $20, and bare boards are available for under $10 worldwide.

However, the chip is not powerful. The S3 variant features:

• 512KB of SRAM
• 8MB of PSRAM
• 16MB of flash memory

This is a very small amount of memory—less primary storage than a single raw photo from a smartphone.

The Challenge: Fitting a Large Model into Tiny Memory

Typically, running an LLM requires the entire model to reside in fast memory (RAM) because the processor must constantly perform calculations against every parameter to generate the next word. Attempting to run a 29M-parameter model normally on an ESP32 would exhaust the fast RAM instantly.

The previous record for a chip of this type was around 260,000 parameters, achieved by developer Dave Bennett.

The Solution: Borrowing Google's Architecture

Developer Slava overcame this bottleneck by borrowing a technique from Google's Gemma architecture called Per-Layer Embeddings.

The process involved:

1. Quantization: The model was reduced to 4-bit precision, bringing the total file size down to just 14.9 MB.
2. Data Relocation: Instead of trying to fit the entire model into the tiny 512KB SRAM or the 8MB PSRAM, the 25-million-parameter embedding table was stored in the slower 16MB Flash memory.
3. Efficient Access: Because the model architecture only needs to pull a few rows from this table per token, the slower Flash memory does not bottleneck the processor. This leaves the 512KB of fast SRAM free for the "thinking core"—the actual reasoning weights.

What This Model Can (and Cannot) Do

It is important to manage expectations. This is not a replacement for a server.

• Training Data: The model was trained on the TinyStories dataset.
• Capabilities: It functions as a "Small Language Model" (SLM) or "micro LM." It is only capable of writing short, simple, fictional stories.
• Limitations: It cannot answer questions, follow instructions, write code, or provide factual knowledge about the real world.

The Bigger Picture: Practical Applications

Focusing on these limitations misses the point of the proof-of-concept. The achievement is fitting a structurally large model onto a computer with practically no resources. It proves that with clever architecture, genuine neural networks can run on dirt-cheap embedded hardware.

Slava envisions useful applications for a model of this size, such as a coffee machine that knows about coffee—every bean, grind, ratio, and water temperature—operating entirely offline with no app required.

Ultimately, the hardware needed for AI depends entirely on the workload. Asking how much hardware you need for local AI without specifying the task is like asking what vehicle you need without stating the destination. A bicycle, a sedan, and a semi-trailer all get you from A to B, but they are built for radically different jobs. AI is the same way.

Stay On the Cutting Edge: Get the Tom's Hardware Newsletter. Get Tom's Hardware's best news and in-depth reviews, straight to your inbox.