NVIDIA Open-Sources Nemotron-Labs Diffusion: One Checkpoint, Three Modes, 6.4x the Throughput
NVIDIA released the Nemotron-Labs Diffusion model family on May 23 as open weights on Hugging Face — seven checkpoints across 3B, 8B, and 14B parameter sizes, plus an 8B vision-language model, all under the NVIDIA Nemotron Open Model License (commercially usable for text models). The 8B base model logged 228,000 downloads in under 48 hours.
The architecture breaks from the standard autoregressive loop. Instead of committing to one token at a time, Nemotron-Labs Diffusion generates tokens in parallel blocks, iteratively refining them until they hit a confidence threshold. The approach converts a pretrained autoregressive model into a diffusion LM rather than training from scratch — a technique from the Efficient-DLM paper (arXiv:2512.14067).
Three Modes, One Checkpoint
All three generation modes run from the same model weights at inference time. No separate deployments, no retraining:
- Autoregressive — standard left-to-right decoding, fully backward compatible
- FastDiffuser — generates 32-token blocks in parallel, iteratively denoises to a confidence threshold. Largest raw throughput gain
- Self-Speculation (LinearSpec / QuadSpec) — drafts a block bidirectionally via diffusion, then verifies it causally with AR. Lossless at temperature 0. NVIDIA reports ~865 tokens/second on H100/B200 — roughly 4-6x the AR baseline on the same hardware
The headline figure is 6.4x higher token throughput in QuadSpec mode versus equivalent autoregressive decoding. That number is vendor-reported. No independent evaluation lab has assessed the model yet; community testing from LinkedIn and X has repeated NVIDIA’s own figures rather than independent runs.
A important constraint: the full throughput gains are tied to SGLang integration running on GB200 or H100 hardware. SGLang support is landing via PR #25803, which was merging into main at time of writing. Performance on A100s and older hardware is not documented.
Accuracy Versus Autoregressive Baselines
On language modeling and commonsense reasoning, Nemotron-Labs Diffusion 8B posts an average of 53.11 across LAMBADA and reasoning suites (recurrent setting) — compared to 52.39 for Mamba-3 MIMO and 52.28 for KDA. That is +1.2% accuracy versus Qwen3 8B across evaluated benchmarks, per NVIDIA’s published comparisons.
The clearest gains appear on RULER long-context retrieval. S-NIAH-3 at 2K context rises from 63.2% (KDA) to 89.8%. MK-NIAH-1 at 4K goes from 28.0% to 37.8%. Hybrid mode (mixing diffusion with AR) averages 42.28% on real-world retrieval (SWDE, SQuAD, FDA, TriviaQA, NQ, DROP) versus 40.14% for Mamba-3 MIMO.
Why This Matters
The autoregressive speed ceiling has been a constant in LLM inference for the better part of a decade. Speculative decoding has chipped at it; now NVIDIA is betting that diffusion can break it structurally. The company is uniquely positioned to pursue this: it controls the GPU hardware, the serving framework (TensorRT-LLM), and the NeMo training ecosystem — and releasing open weights as the opening move creates an incentive for the community to validate the throughput claims and build the third-party benchmarks NVIDIA needs to prove them.
Key Numbers
| Model | Downloads (48h) | Mode | Throughput vs AR | Accuracy vs Qwen3-8B |
|---|---|---|---|---|
| Nemotron-Labs-Diffusion-3B | 14.2K | — | — | — |
| Nemotron-Labs-Diffusion-8B-Base | 228K | QuadSpec | 6.4x (vendor-reported) | +1.2% |
| Nemotron-Labs-Diffusion-8B | 24.1K | LinearSpec | ~4-6x (vendor-reported) | — |
| Nemotron-Labs-Diffusion-14B | 2.0K | — | — | — |
License: NVIDIA Nemotron Open Model License (text models). VLM-8B uses NVIDIA Source Code License. All weights live at huggingface.co/collections/nvidia/nemotron-labs-diffusion.