Developer Runs 28.9M-Parameter AI Model on $10 ESP32-S3 Microcontroller

Here is the rewritten news content, structured as clean, professional English Markdown.

Running a 29M-Parameter AI Model on a $10 Microcontroller

When discussing local AI, the conversation typically centers on mini-PCs like the RTX Spark or expensive home servers with professional GPUs. However, one of the most impressive AI hardware feats in recent memory has been achieved on a chip that costs less than a burger.

Last week, a Ukrainian developer known as Slava S (GitHub handle: slvDev) released a project called ESP32-AI. He successfully ran a 28.9-million-parameter language model locally, entirely on-device, using an ESP32-S3 microcontroller.

The Hardware: ESP32-S3

The ESP32-S3 offers exceptional value in computing. In the U.S., a board with a protective case can be purchased for under $20, and bare boards are available for under $10 worldwide.

However, the chip is not powerful. The S3 variant features:

• 512KB of SRAM
• 8MB of PSRAM
• 16MB of flash memory

This is a very limited amount of memory—less primary storage than a single raw photo from a smartphone.

The Challenge: Memory Constraints

To run an LLM, the entire model must typically reside in fast memory (RAM) because the processor constantly performs calculations against every parameter to generate the next word. Running a 29M-parameter model normally on an ESP32 would exhaust the fast RAM instantly.

The previous record for a chip of this type was approximately 260,000 parameters, achieved by Dave Bennett.

The Solution: Borrowing from Google's Gemma

Slava overcame this bottleneck by using a technique from Google's Gemma architecture called Per-Layer Embeddings.

The process involved:

1. Quantization: The model was reduced to 4-bit precision, bringing the total file size to just 14.9 MB.
2. Memory Reallocation: Instead of trying to fit the entire model into the tiny 512KB SRAM or the 8MB PSRAM, the 25-million-parameter embedding table was stored in the slower 16MB Flash memory.
3. Efficient Access: Because the model architecture only needs to pull a few rows from this table per token, the slower Flash speed does not bottleneck the processor. This leaves the 512KB of fast SRAM free for the "thinking core"—the actual reasoning weights.

Limitations: What This Model Cannot Do

It is important to clarify the model's capabilities. The model was trained on the TinyStories dataset and functions as a Small Language Model (SLM) or "micro LM."

• It can only write short, simple, fictional stories.
• It cannot answer questions.
• It cannot follow instructions.
• It cannot write code.
• It possesses zero factual knowledge about the real world.

The Bigger Picture: Why This Matters

Focusing on these limitations misses the significance of this proof-of-concept. The achievement is fitting a structurally large model onto a computer with practically no resources.

This demonstrates that with clever architecture, genuine neural networks can run on dirt-cheap embedded hardware. Slava envisions practical applications, such as a coffee machine that knows about coffee—every bean, grind, ratio, and water temperature—operating entirely offline with no app required.

A Final Note on AI Hardware

When discussing "AI," the required hardware depends entirely on the workload. Asking how much hardware is needed for local AI without specifying the task is like asking what vehicle is needed without stating the goal. A bicycle, sedan, pickup truck, semi-trailer, and train all "get you from A to B," but they are built for radically different jobs. AI is the same: the task determines the hardware required.

Stay On the Cutting Edge: Get the Tom's Hardware Newsletter for the best news and in-depth reviews, straight to your inbox.