AI Developer Runs 28.9-Million-Parameter Language Model on $10 ESP32-S3 Microcontroller

Ukrainian developer Slava S (slvDev on GitHub) has successfully run a 28.9-million-parameter language model locally on an ESP32-S3 microcontroller, a chip costing under $10. The project, called ESP32-AI, overcomes severe memory constraints (512KB SRAM, 8MB PSRAM, 16MB Flash) by using Google's Per-Layer Embeddings technique. The model was quantized to 4-bit (14.9 MB total) and the 25-million-parameter embedding table was stored in slower Flash memory, while the reasoning weights run in fast SRAM. The model, trained on the TinyStories dataset, is only capable of generating short fictional stories and cannot answer questions or write code. The achievement is a proof-of-concept demonstrating that neural networks can run on extremely cheap embedded hardware, with potential applications like smart appliances (e.g., a coffee machine with offline coffee knowledge).