OpenRouter's Batch API Cuts Inference Costs 50% on 70+ Models — 7-Minute Median Turnaround in Beta
OpenRouter launched a Batch API that lets developers submit entire workloads in a single POST request, collect results within 24 hours, and pay roughly half the normal per-token price. The API is live across more than 70 models.
The model: when a provider has flexibility on when to run the request — within a 24-hour window — they charge 50% or less of their standard rate. For AI teams running evaluations, scoring benchmarks, or processing large corpora, the savings compound fast.
How it works
A single call to api/v1/batches takes a list of requests and an endpoint shape (chat completions, responses, messages, or embeddings). Each request carries a custom_id for matching results. After submitting, callers poll GET /api/v1/batches/:id until the batch status is completed, failed, expired, or cancelled. Completed batches return results inline.
The 24-hour maximum window is a theoretical ceiling. Beta data tells a different story: across 230,000+ completed batches during a two-week period, the median turnaround was 7 minutes. The 90th percentile was 1 hour. The 99th percentile was 10.3 hours. Time of submission matters — batches submitted during high-demand windows take longer.
Who this is for
The target is async AI workloads that don’t need real-time responses:
- Scoring evaluation sets before a model deployment
- Back-filling embeddings across a document corpus
- Labeling training data at scale
- Summarizing backlogs of support tickets, code reviews, or meeting transcripts
- Running the same prompt across thousands of rows overnight
Real-time inference stays on the standard API. Batch handles everything where you can tolerate variable latency and want to maximize cost efficiency.
The cost math for AI teams
Production AI evaluation pipelines are expensive. A single eval run against a frontier model — 1,000 prompts, multiple runs, multiple models — routinely costs hundreds of dollars at standard rates. At batch pricing (50% off), the same run costs half. For teams running continuous evals across a model registry, the annual savings are material.
The 70+ model catalog includes frontier and mid-tier models from Anthropic, OpenAI, Google, and open-source providers. Batch pricing varies by provider; OpenRouter passes through the discount rather than setting a fixed 50% floor.
The Batch API is available now via openrouter.ai/docs/batch-quickstart. Models with batch support are filterable at openrouter.ai/models?variant=batch.