OpenAI Jalapeño Clocks 56x More Interactive Throughput Per Kilowatt Than Nvidia Blackwell in InferenceX
OpenAI published the first measured performance results for Jalapeño, its custom inference accelerator, showing consistent leads over Nvidia’s GB300 Blackwell across throughput, latency, and power efficiency on three public models.
The Numbers
Tested on InferenceX, a public benchmark developed by SemiAnalysis that measures inference performance across the full serving cycle, Jalapeño ran GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T against leading commercially available systems.
| Metric | Range |
|---|---|
| AI work per watt at peak throughput | 1.5–1.9x more than GB300 |
| End-to-end latency | 1.7–3.6x lower |
| Interactive workload performance | 2.1–4.1x higher |
| Throughput/kW at matched TBT (182.46 tok/s/user) | 6,744 vs 120 mixed/kW — 56.1x |
The 56x figure is not a peak cherry-pick — it is measured at a specific user experience constraint (time-between-tokens of 182.46 tokens/second per user), where Jalapeño delivers 6,744 mixed tokens per kilowatt versus GB300’s 120. At that operating point, Jalapeño returns 694 tokens per second per user; the comparison system returns 182.
On Kimi K2.5 1T — the largest of the three test models — Jalapeño delivered 1.5x higher peak performance per watt and 3.4x lower end-to-end latency.
Jalapeño’s rated thermal design power is 700W. Measured sustained power on the tested workloads remained at or below 550W.
Why the Architecture Wins at Both Ends
Most inference hardware forces a trade-off: high throughput or low latency, but not both. Jalapeño was designed to avoid that trade-off by minimizing data movement through the system.
The key design decision: model state — including the KV cache generated during token decoding — can be explicitly placed and kept local, rather than moved between chips or memory tiers. The on-chip network is integral to the architecture rather than bolted on, meaning the entire workload from prefill through decode can stay within a single connected system without bouncing between specialized compute and memory resources.
Prefill (processing the input prompt) is compute-bound. Decode (generating tokens one at a time) is memory-bandwidth-bound. Most existing accelerators are tuned for one or the other. Jalapeño handles both phases without sacrificing performance on either — a property OpenAI says becomes more valuable as models grow larger and as agentic workloads require many sequential decode steps.
AI Helped Build and Program It
OpenAI says AI tools accelerated Jalapeño’s design cycle, helping bring the chip from initial design to tapeout in nine months. AI optimized arithmetic circuits during design, and the team used Codex with GPT-Astra to port three open-weight models — not originally in the production plan — to high performance on Jalapeño within two months.
On selected attention and mixture-of-experts blocks in GPT-OSS models, AI-generated kernel implementations ran 1.5 to 1.8 times faster than the existing human-expert-written code. That figure is specific to those selected blocks, not the full model forward pass, but it marks a practical milestone: AI-generated inference code is now measurably outperforming human expert code in production-relevant conditions.
What Comes Next
OpenAI describes Jalapeño as the start of a multigenerational platform. The company will ramp the chip in the months ahead with the stated goal of making inference faster and cheaper for customers — a competitive necessity as rivals including Anthropic and Google accelerate their own custom silicon programs.
The broader claim embedded in the results: a lab that designs its own models, chips, serving software, memory, and networking together can extract gains that commodity hardware cannot match, because optimization is possible across every layer simultaneously. Whether that advantage holds at the scale of millions of chips and dozens of model generations is still unproven. Jalapeño’s first results suggest the hypothesis is worth taking seriously.