Google Ships Gemma 4 QAT Checkpoints: E2B Drops to 1GB on Mobile With Custom Quantization Schema
Google DeepMind released quantization-aware training (QAT) checkpoints for the entire Gemma 4 lineup today, targeting mobile phones, laptops, and consumer GPUs with precision compression that standard post-training quantization cannot match.
The headline number: the Gemma 4 E2B text-only model (without per-layer embeddings) now runs in under 1GB of memory using a purpose-built mobile quantization format. That is small enough for offline inference on any modern smartphone.
What Changes With QAT
Standard post-training quantization compresses weights after training and accepts a quality trade-off. QAT integrates the quantization process into training itself, teaching the model to work within 4-bit constraints rather than adapting to them after the fact. Google’s results show the QAT checkpoints deliver higher overall quality than PTQ at the same compression level.
Two formats ship today:
Q4_0 QAT — Available for all Gemma 4 model sizes. Replaces standard Q4_0 quantization with training-aware compression. Directly compatible with llama.cpp, vLLM, and Unsloth.
Mobile QAT format — Applied to E2B and E4B edge models, engineered for mobile accelerators with four specific design choices:
- Static activations: Scaling factors pre-computed during training rather than calculated at inference. Reduces compute load on mobile chips.
- Channel-wise quantization: Weight layout matches mobile accelerator design, enabling native calculation without slow workarounds.
- Targeted 2-bit quantization: Token-generation layers compressed to 2-bit; core reasoning layers held at higher precision. Maximizes storage savings without degrading reasoning.
- Embedding and KV cache optimization: Vocabulary lookup and short-term memory compressed separately, enabling long conversations without running out of active memory.
Memory Footprint
The QAT release significantly reduces the VRAM floor for running Gemma 4 models:
| Model | Format | Memory |
|---|---|---|
| E2B (text-only, no PLE) | Mobile QAT | < 1 GB |
| E2B | Mobile QAT | ~1 GB |
| E4B | Mobile QAT | ~2.1 GB |
| 12B | Q4_0 QAT | ~5.5 GB |
| 27B | Q4_0 QAT | ~14.8 GB |
The 27B model still requires a 16GB GPU minimum; the 12B fits comfortably on consumer 8GB cards at Q4_0.
Developer Tooling
The QAT checkpoints land in the same ecosystem as the existing Gemma 4 weights. vLLM supports serving with the new checkpoints; MLX provides Apple Silicon-optimised inference paths; Unsloth enables QAT-aware fine-tuning. Modality-selective deployment is supported — dropping the audio or vision encoder reduces the footprint further for text-only workloads.
Google also released the Google AI Edge Gallery for macOS alongside this checkpoint drop, letting Mac users run Gemma models locally with Edge runtime tooling.
Context
The Gemma 4 12B base model launched earlier this year with an encoder-free architecture capable of audio and video input on 16GB RAM laptops. The 2B edge model was already running offline on iPhone via Apple Neural Engine. Today’s QAT release closes the remaining gap: on-device quality on memory-constrained hardware no longer requires accepting the full PTQ quality penalty.
With sub-1GB capable models now available under Apache 2.0, Gemma 4 becomes a credible option for embedded deployment at the low end of the hardware curve — IoT, wearables, and mobile apps where a cloud call is not an option.