← Back to feed

OpenRouter on October 2 published a dedicated Model Router Benchmarks page, the first systematic head-to-head comparison of the growing class of infrastructure that sits between applications and raw model APIs. Seven routers across six benchmark suites are scored on a Router Index that weights quality at 60%, time-per-task at 20%, and cost at 20% — with a slider that lets users shift those weights toward their own priorities.

What Gets Benchmarked

The benchmark covers seven routers — Auto Router, Jev Router, Pareto, Fugu Max, Fugu Ultra, Fugu Ultra v2, and Switchyard — and separates them into three structural categories:

Model-selecting routers pick a model for each turn and hand control back to the caller. OpenRouter’s Auto Router and Jev Router both work this way. Model selection is transparent and billing happens at the standard per-token rate for whichever model was chosen.

Blend-and-escalate routers run a mix of models, sometimes escalating to a frontier model when a cheaper one underperforms. Sakana’s Fugu (in three variants: Max, Ultra, Ultra v2) and Unbiased’s Pareto fall here. Neither discloses which models run or how often escalation fires; both bill at standardized per-token rates.

Dynamic-pair routers couple a cheaper model with a stronger one and switch between them as an agent progresses through a task. NVIDIA’s Switchyard is the primary representative. Users define the pairing; OpenRouter benchmarked several Switchyard model combinations for the index. The design targets long-horizon agentic workloads where early steps rarely need frontier capability.

The Cost Spread

The headline number is the cost spread across routing strategies. Comparing DeepSeek V4 Flash routed against GPT-6 Astra:

• Per-token cost difference: 48x
• A 10-49 turn Codex session cost difference: 21x

Those figures illustrate the stakes for teams running high-volume agent workloads. A router that consistently promotes requests to GPT-6 Astra when DeepSeek V4 Flash would have sufficed compounds across every session at production scale.

OpenRouter’s benchmark runs each router through tasks from varied domains and reports quality, time, and cost separately before folding them into the Router Index. The underlying benchmark tasks cover general reasoning, coding, and agentic scenarios.

How the Router Index Works

The Router Index synthesizes the three dimensions into a 0-10 score for each router and benchmark combination. Default weights are 60% quality, 20% speed, 20% cost. A slider on the live page lets readers reweight toward pure quality, pure cost, or any point between — which matters because the benchmark is designed for general tasks, not any specific production workload.

One router type excluded from the primary index: Fusion, which runs the same prompt across a council of models and synthesizes the best response. That approach targets a different problem from routing — it increases coverage rather than selecting a path — and OpenRouter says it warrants separate comparison methods.

Provider Routing vs Model Routing

The page also clarifies a distinction that OpenRouter has operated since its founding. Provider routing selects which inference provider serves a given model (for models offered across multiple providers), optimizing for price, speed, uptime, or data policy with automatic failover. Model routing selects which model handles a request. Both are available through OpenRouter’s API, including via the provider preferences object that lets callers set sorting rules and data-policy constraints.

The new benchmark covers model routing; provider routing is a separate user-configurable layer rather than a concern of the Router Index.

Why This Matters

As agentic workloads grow from dozens of turns to hundreds, routing becomes less of a convenience feature and more of a cost-control mechanism with compounding returns. The 21x session cost multiplier on Codex tasks demonstrates that model choice — or model routing strategy — is one of the highest-leverage dials operators have. Router benchmarks give teams outside of OpenRouter a framework for auditing that choice against actual task performance rather than informal model reputation.

All routers in the comparison are available via the OpenRouter API.