NVIDIA Cosmos 3: First Open Physical AI Omnimodel Ships at Computex — 20T Tokens, Robots Get a Physics Brain
NVIDIA announced Cosmos 3 at GTC Taipei during Computex on June 1. It is the first fully open model that handles vision reasoning, simulated world generation, and robot action prediction inside a single architecture — trained on 20 trillion multimodal tokens. Previous physical AI approaches required separate reasoning and generation models stitched together; Cosmos 3 collapses that into one system with weights anyone can download.
The architectural choice is a Mixture-of-Transformers (MoT): a reasoning transformer block paired with a generation transformer block, sharing attention across modalities but maintaining separate weight sets. That shared backbone is what allows the model to condition video generation on its own reasoning output without a brittle two-model handoff.
Models and Hardware
Two variants ship at launch:
| Variant | Total Params | Hardware Target | Primary Use |
|---|---|---|---|
| Cosmos 3 Nano | 16B (8B+8B) | RTX PRO 6000 workstation | Real-time robotics inference |
| Cosmos 3 Super | 64B (32B+32B) | Hopper / Blackwell datacenter | Synthetic data generation, policy post-training |
A third variant — Cosmos 3 Edge — is listed as coming for real-time on-device inference.
The Reasoner tower accepts up to 256K context tokens, useful for reasoning over extended video sequences. The Generator’s text input is capped at 4,096 tokens. NVFP4 quantization is available for the Nano variant, delivering up to 2x inference speedup over BF16 at some quality cost.
Action Output: Not a Video Model
The distinction NVIDIA is drawing is that Cosmos 3 natively outputs robot action data — joint angles, gripper positions, spatial trajectory points — not just video. That is what makes it an omnimodel rather than a world model. Robots need motor commands, not pixels; Cosmos 3 produces both.
Supported embodiments at launch: Franka Panda (single and dual arm), WidowX 250, UMI, Agibot, UR series, Google robot, and autonomous vehicle/egocentric camera modes. Action vectors range from 9D to 57D depending on the platform. Teams working outside those embodiments need custom post-training.
Training Data
- ~1.3 billion data points across 393 datasets
- 767 million images, 348 million video clips, 139 million audio samples, 8 million robot action trajectories
- 6 synthetic datasets released alongside weights: robotics manipulation, physics simulation, spatial reasoning, human motion, autonomous driving, warehouse operations
Benchmark Performance
NVIDIA claims #1 among open models across every benchmark it entered at launch. All rankings are self-reported at this stage — independent replication will take weeks. The external benchmarks (Artificial Analysis rankings) carry more weight than the robotics-specific leaderboards, which are newer and less established.
- Physics-IQ: #1 open model (physical plausibility in generated video)
- PAI-Bench: #1 open model (physical AI generation accuracy)
- R-Bench: #1 open model (real-world video generation quality)
- RoboLab: #1 open model (simulation policy, language-guided tasks)
- RoboArena: #1 open model (DROID real-robot hardware)
- VANTAGE-Bench: #1 at 32B tier (Super) and 8B tier (Nano)
- TAR: #1 open model (traffic anomaly reasoning)
- Artificial Analysis: #1 open model, Text-to-Image and Image-to-Video leaderboards
Specific numerical scores are not published in the main announcement — NVIDIA points to the technical report for full tables.
Licensing: OpenMDW 1.1
Cosmos 3 ships under OpenMDW 1.1, a new model-centric license the Linux Foundation released alongside this launch. Unlike Meta’s Llama license — which restricts commercial use beyond 700M monthly active users — OpenMDW covers weights, architecture, datasets, documentation, and code under a single permissive agreement. Commercial use, modification, and redistribution are all permitted. NVIDIA positions this as a deliberate contrast to closed-source universal intelligence layer models from other labs.
Availability
Weights: Hugging Face (nvidia/Cosmos3-Nano, nvidia/Cosmos3-Super), GitHub, build.nvidia.com.
NIM microservices: Microsoft Azure, CoreWeave, Baseten, Nebius, Deep Infra, Classmethod.
The Cosmos 3 Reasoner NIM is available now; the Generator NIM is coming.
The release also includes the Isaac GR00T Reference Humanoid Robot — an open reference architecture built on Jetson AGX Thor T5000 compute, using Cosmos 3 as the predictive baseline. NVIDIA’s pitch: compress robot validation cycles from months to days by running reinforcement learning inside a Cosmos 3-powered neural simulator.