← Back to feed

Inference on microcontrollers is a settled problem. TinyML frameworks have been running quantized models on ARM Cortex-M chips at milliwatts for years. Community ports of Andrej Karpathy’s llama2.c demonstrated 260K-parameter transformers on standard ESP32 hardware, and more recently an independent project (esp32-ai, by Slava S.) got a model of nearly 29 million parameters running on an ESP32-S3 for inference.

Training from scratch is different. A GitHub project called qapla, published by Carloscodix, goes further: full training loop on a single ESP32-S3 retailing for $8. Forward pass, backpropagation, and weight updates, all written by hand in C, all running on the chip.

The Argument for On-Device Training

The standard assumption in embedded AI is: train somewhere else, deploy here. That assumption fails for a specific class of applications where the training data does not exist until the device is in the field.

The qapla README identifies two examples that illustrate the constraint. A vibration sensor attached to a specific piece of farm machinery needs to learn the normal vibration signature of that particular machine — not an average across all machines of that model, but this machine, with its worn bearings, its specific mounting, its RPM under its usual load. That data cannot be collected before deployment and cannot be transmitted reliably from a remote field. The sensor has to learn in place.

A soil sensor that models when a specific plot needs watering faces the same problem. Drainage, sun exposure, soil composition, and microclimate vary per plot. A model trained on aggregate agricultural data will be less accurate than one trained on the actual data stream from this sensor in this ground. The data only exists after the device is in.

For both use cases, the standard TinyML approach — train offline, quantize, deploy — does not work. The chip needs to learn from scratch once installed.

What Running the Full Training Loop on $8 Hardware Requires

The ESP32-S3 has 512KB of on-chip SRAM and up to 8MB of external PSRAM. Training a transformer requires simultaneously holding:

• Model weights
• Activations (for the forward pass)
• Gradients (for backpropagation)
• Optimizer states (for weight updates)

None of these can spill to external storage mid-computation. The entire training state must fit in those 8.5MB combined.

A framework like PyTorch or JAX adds overhead neither the memory nor the compute budget can accommodate. The backpropagation in qapla is written in C by hand: each layer’s gradient computation is explicit code rather than an automatically differentiated graph. That is the engineering cost of putting training inside a commodity microcontroller — the developer writes what an autograd framework would generate automatically.

The experiment uses a character-level transformer trained on text. The qapla README specifically chose a language task — training on Klingon, as a proxy for an unusual language or constrained domain — to get clean measurable learning signals from a small model. Language is tractable to evaluate on a laptop; agricultural sensor data requires physical hardware.

The Gap Between Inference and Training

The 29M-parameter inference achievement by esp32-ai illustrates the practical ceiling for inference-only deployment on this hardware. Running 29M parameters for inference requires holding those parameters in memory plus space for activations during the forward pass.

Training the same model requires holding parameters plus activations plus gradients of similar size plus optimizer states (typically 1-2x parameter size for Adam). The memory requirement for training is 3-5x the inference requirement for the same model size. The maximum trainable model on ESP32-S3 is therefore substantially smaller than the maximum runnable model.

The qapla project does not publish a specific parameter count for the trained model. The contribution is demonstrating that the training loop runs at all, not establishing a new size record. The maximum trainable size on this hardware is a tractable research question that the project opens rather than closes.

Practical Scope

The use cases for on-device training are narrow. Federated learning schemes that currently require aggregation server round-trips could run entirely locally for single-device adaptation. Predictive maintenance for remote industrial equipment with no reliable internet connection — using LoRa or similar low-bandwidth protocols for reporting only — becomes viable without cloud training pipelines. Sensors calibrating to their installation environment without factory pre-training can adapt to conditions that vary in ways the manufacturer cannot anticipate.

For most embedded AI applications, inference of a pre-trained model remains the right architecture. The training loop on chip is the right architecture for the specific case where local adaptation to post-deployment data is necessary and connectivity is insufficient for cloud training or federated aggregation.

The qapla project is open source under MIT on GitHub (github.com/Carloscodix/qapla). No accuracy benchmarks are published for the trained model — the training process itself is the result.