← Back to feed

OpenAI has cut inference costs by more than half on some of its existing models, according to The Information. The company did not announce a new model or architecture change — the reduction came from operational improvements to how existing models are served.

The report surfaced another detail that captures the scale of the gap between casual and paid usage: logged-out ChatGPT traffic was running on only a few hundred Nvidia GPUs. That is a fraction of the infrastructure dedicated to paying subscribers, where full-quality models handle queries at higher compute cost.

The Margin Math

Frontier inference economics have been brutal. OpenAI’s adjusted gross margin fell from 40% in 2024 to 33% in 2025 as inference costs quadrupled alongside rapidly growing usage. The company is now reportedly tracking at around 39% gross margin in Q1 2026, with a 52% target by year-end.

Anthropic is in a similar band, around 44%, which is consistent with the two companies being at comparable scale on the cost-per-token dimension. Both are far below the 70–80% gross margins typical of mature software businesses.

A 50%+ cost reduction on deployed models — without waiting for next-generation hardware — is the most direct path to closing that gap. The mechanisms are not confirmed but the standard toolbox includes speculative decoding, quantization, KV-cache tuning, dynamic batching, and query routing that sends simpler requests to cheaper model variants.

Routing as the Hidden Lever

The logged-out GPU detail points to tiered serving as a key strategy. If unauthenticated traffic runs on a lightweight pool, OpenAI is effectively operating a two-tier inference stack: a low-cost public face and a higher-quality backend for paying customers. Charging the same headline price for a cheaper-to-serve tier expands margin without a visible product change.

This approach has a ceiling: the quality difference has to stay below the threshold where users notice and churn. At the current state of model capability, the gap between a quantized GPT-5.5 and the full version is small enough that most use cases are unaffected.

Competitive Implications

OpenAI’s cost trajectory matters for its planned IPO. Revenue run rate and model capability are visible; infrastructure efficiency is not, until it shows up in margins. A path to 52% gross margin by year-end would put the company within reach of software-company multiples rather than the semiconductor-company multiples that currently dominate AI infrastructure valuations.

For Anthropic, the numbers frame the competitive stakes. Claude’s average revenue per user is substantially higher than ChatGPT’s — $16.20 per monthly active user versus $2.20 by one analysis — but if OpenAI closes the margin gap through cost efficiency rather than pricing, the premium ARPU advantage becomes less decisive.

The inference cost war is not over. But OpenAI has demonstrated that existing model deployments can be made substantially cheaper without waiting for new hardware. That changes the short-term competitive calculus in ways that raw benchmark performance does not.