← Back to feed

Google released Gemma 4 12B on June 3, slotting it between the existing 4B edge model and the 26B Mixture of Experts released in May. The 12B is not a larger version of the same design — it carries two architectural changes that separate it from everything else in the open-weight 12B tier.

Encoder-Free Design

Every prior multimodal model at this size routes vision and audio inputs through specialist encoders before they reach the main language model. Gemma 4 12B removes those encoders entirely. Vision and audio inputs flow directly into the LLM backbone.

The practical payoff: unified inference, no modality-specific warm-up paths, and a cleaner deployment stack. The architectural risk — that the backbone has to learn representations that specialist encoders previously handled — appears to be manageable at this parameter count. Google reports benchmark performance nearing the 26B MoE, which carries dedicated encoder infrastructure.

Native Audio: A First for Mid-Size Open Weights

Gemma 4 12B is the first Gemma model of this scale to accept audio natively. The E4B (4B edge model) does not handle audio. The 26B MoE does, but its memory footprint puts it out of reach for local consumer hardware.

The 12B opens audio-native inference to machines with 16GB of VRAM or unified memory — standard on M2-and-later MacBooks and current-generation consumer laptops. That closes a gap that has kept voice-aware local agents confined to server deployments.

Multi-Token Prediction Drafters Included

Gemma 4 12B ships with Multi-Token Prediction (MTP) drafters baked in. MTP reduces latency by predicting multiple future tokens in parallel rather than strictly left-to-right. For agentic workflows where the model is generating long structured outputs — code, plans, tool call sequences — this can materially cut wall-clock time without touching the core model weights.

Benchmark Position

Google reports benchmark performance nearing its 26B MoE at less than half the total memory footprint. The 12B falls below the 26B on complex multi-step reasoning tasks but stays competitive across the standard multimodal benchmark suite.

Compared to other 12B-class open weights:

• Encoder-free design is not yet common at this size
• Native audio input is unique in the class
• Apache 2.0 license allows unrestricted commercial use

The Gemma 4 12B joins the existing family (E4B, 26B MoE) under Apache 2.0 and is available now through Hugging Face and Google’s developer APIs.

Why This Release Was Separate

The original Gemma 4 batch in May 2026 covered the 2B, 9B, 26B MoE, and E4B edge models. The 12B was not in that group. Google releasing it separately — with a distinct architectural callout around the encoder-free approach and native audio — suggests this is not a fill-the-gap release but a deliberate design decision that required additional validation time.

For developers running local agents that need to process speech or mix text, image, and audio in a single context, the Gemma 4 12B is currently the only open-weight option that handles all three modalities without a separate encoder pipeline.