MLPerf Training v6.0: NVIDIA Sweeps All 7 Benchmarks, DeepSeek-V3 671B Trained in 2 Minutes
MLCommons released MLPerf Training v6.0 results on June 16. Twenty-four organizations submitted 95 unique system configurations across 13 different hardware accelerators. NVIDIA won every benchmark and was the only platform to submit across all seven. CoreWeave delivered the fastest Available-cloud result for DeepSeek-V3, training the 671-billion-parameter model to benchmark target quality in 2.02 minutes.
New Benchmarks Signal a Structural Shift
Version 6.0 introduced two new pretraining workloads that had not appeared in previous rounds: DeepSeek-V3 671B and GPT-OSS 20B. Both use mixture-of-experts architectures, where a router dispatches each token to a subset of specialist sub-networks rather than running the full parameter count. The addition formalises what has already become the dominant architectural choice at frontier scale. The prior suite was built around dense models; this round puts MoE at the centre.
DeepSeek-V3 671B is the same model underlying DeepSeek-R1 and subsequent derivatives. Including it as a training benchmark gives the industry a standardised reference point for how well a given infrastructure stack handles the routing overhead that dense-model frameworks were not designed for.
The Numbers
| Benchmark | GPU Platform | Cluster Scale | Time-to-Train |
|---|---|---|---|
| DeepSeek-V3 671B (MoE) | GB300 NVL72 | 8,192 GPUs | 2.02 mins |
| GPT-OSS 20B (MoE) | GB300 NVL72 | 512 GPUs | 7.43 mins |
| Llama 3.1 405B | GB200 NVL72 | 8,192 GPUs | 7.07 mins |
| Llama 3.1 8B | GB200 NVL72 | 1,024 GPUs | 4.46 mins |
| Llama 2 70B LoRA | GB300 NVL72 | 512 GPUs | 0.4 mins |
| FLUX.1 | GB300 NVL72 | 512 GPUs | 17.1 mins |
| DLRM-dcnv2 | GB300 NVL72 | 64 GPUs | 0.67 mins |
The 2.02-minute result for DeepSeek-V3 on 8,192 GPUs is CoreWeave’s submission running NVIDIA GB300 NVL72 over Spectrum-X Ethernet — the fastest Closed/Available-cloud result in the round. CoreWeave also submitted at 4,096 GPUs (3.09 minutes) and 2,048 GPUs (5.54 minutes), demonstrating near-linear scaling across the three cluster sizes.
NVIDIA’s own submissions covered every benchmark in the suite, with Llama 3.1 405B on 8,192 GB200 NVL72 GPUs at 7.07 minutes representing the fastest result on that workload (Azure submitted the same size configuration). GB300 NVL72 delivered up to 1.6x faster training than GB200 NVL72 at matched scale across the round.
What the GB300 Advantage Is
GB300 Ultra GPUs carry expanded memory capacity and a higher power ceiling compared to GB200. That headroom allows the chip to sustain peak compute longer on high-memory-pressure workloads like large MoE models, where router tables and expert activation patterns stress the memory subsystem. The 1.6x improvement is not from raw flops alone — it is from eliminating the throttling that GB200 hits under sustained MoE training loads.
NVIDIA’s software stack optimisations contributed independently. Using the NeMo container released this month (26.06), NVIDIA improved DeepSeek-V3 training throughput from 1,298 TFLOPS per GPU to 1,648 TFLOPS per GPU — a 1.3x gain over three months on the same hardware, without changes to the silicon. That number illustrates how much of the training efficiency story is still in software.
Ecosystem Submissions
The 24 submitting organisations included AMD, Azure, Cisco, CoreWeave, Dell, Fujitsu, Google, HPE, Lambda, Oracle, and 14 others. Google submitted its own configurations. Tinycorp — the developer of the Tinygrad framework and RISC-V-based training hardware — made its first MLPerf Training appearance in v6.0, alongside Sigmicro and a handful of other new entrants.
Cohere reported a separate result outside the benchmark itself: its North agentic AI platform achieved 3x faster training on GB200 NVL72 infrastructure on CoreWeave compared with its prior configuration. Midjourney, which trained its v8 image generation model on a Blackwell cluster, is now scaling to GB300 Ultra for upcoming video models.
Why This Round Is Different
Previous MLPerf Training rounds were dominated by dense model benchmarks. The MoE additions in v6.0 make the benchmark directly relevant to the models that will define the next several years of frontier training — not just as a proxy for general throughput, but as a direct measure of the hardware and software optimisations that matter for routing-heavy workloads.
The 2.02-minute figure for DeepSeek-V3 is notable because it collapses a model training run that would have taken hours on earlier generations into a window comparable to a long CI test. It does not mean training is free: the 8,192-GPU cluster required to achieve that time is capital that almost no organisation outside the hyperscalers and the top neoclouds can assemble. But it establishes that the physics of the problem are solved at scale. The remaining bottleneck for most teams is access, not arithmetic.