← Back to feed

Retrieval is the hidden cost centre in agentic AI. When a retrieval step returns irrelevant context, the agent searches again. Each extra search is another round of model calls, more reasoning tokens, and longer latency. NVIDIA’s Nemotron-3-Embed family targets that problem with three models spanning the accuracy-to-efficiency curve.

The 8B flagship, Nemotron-3-Embed-8B-BF16, takes the top spot on RTEB — the RAG retrieval leaderboard that measures how reliably a model locates the right context — at 78.5%. Two 1B variants follow: 1B-BF16 at 72.4% RTEB (a 27% error rate reduction over NVIDIA’s prior 1B generation) and 1B-NVFP4, a Blackwell-optimized quantized build that retains over 99% of BF16 accuracy while delivering up to 2x the throughput. All three accept 32K-token inputs, covering long documents, full code repositories, and extended agent histories.

RTEB and What It Actually Measures

RTEB is structured around production retrieval patterns: multi-hop questions, long-context documents, code search, and domain-specific corpora. The benchmark asks whether a retrieval model will find the right chunks on the first pass under realistic conditions. A model that consistently retrieves relevant context eliminates the retry spiral — fewer searches, shorter chains, lower token bills.

At 78.5%, Nemotron-3-Embed-8B takes the leaderboard from models trained primarily on web text and short documents. The 1B-BF16 at 72.4% is the comparable cost tier — lower memory overhead, CPU-deployable, suitable for edge inference or cost-constrained cloud serving.

The 1B-NVFP4 Design Point

The third member is the most infrastructure-specific. NVFP4 is NVIDIA’s Blackwell-native number format, a 4-bit floating-point representation designed for the Tensor Memory Accelerator in GB200-class hardware. By building a retrieval model to that format natively, NVIDIA doubles throughput without the accuracy penalty that standard INT4 quantization carries in retrieval workloads. The claim is over 99% BF16 accuracy at 2x the traffic capacity.

That design decision matters for agent deployments that run retrieval at high concurrency. A coding agent running 300 subagents in parallel — as Kimi Work and Claude Dynamic Workflows do — generates hundreds of retrieval calls per second. A model that doubles per-chip throughput halves the cluster size needed to keep retrieval off the latency critical path.

How It Was Built

NVIDIA built the family using structured pruning — starting from a larger pre-trained model and systematically removing parameters based on their contribution to retrieval quality — combined with teacher distillation from the 8B model to the 1B variants, and progressively extended training contexts. The 32K-token context window is a direct result of that training progression: the models were exposed to increasingly long documents to ensure full-length retrieval accuracy rather than degrading at the tail.

Integration with the Nemotron Stack

Testing with NVIDIA Nemotron 3 Ultra showed that improved first-pass retrieval translated to measurably fewer search iterations and lower total token consumption per task. The embedding model is not the only component — the full stack includes rerankers and orchestration — but retrieval quality at the embedding step determines whether the downstream reasoning ever sees the right evidence.

NVIDIA is releasing all three under open, commercially available licenses. The embedding market has been dominated by models from OpenAI, Cohere, and Voyage AI; Nemotron-3-Embed positions NVIDIA to supply the retrieval layer for agent deployments running on its own inference hardware.

Key Numbers

Model	RTEB Score	Context	Optimisation
Nemotron-3-Embed-8B-BF16	78.5% (#1)	32K	Full precision flagship
Nemotron-3-Embed-1B-BF16	72.4%	32K	27% error reduction vs prior 1B
Nemotron-3-Embed-1B-NVFP4	99%+ of BF16	32K	2x throughput on Blackwell