← Back to feed

The LLM-router market just got a public postmortem from inside the product category. Manifest launched its router in March, deprecated it in June, and will shut it down fully on September 1 after four months of use across roughly 7,000 cloud users.

The router did the standard thing: classify requests into complexity tiers, then route simple work to cheaper models and difficult work to stronger models. Manifest used four tiers: simple, standard, complex, and reasoning. The goal was cost reduction.

The finding was sharper than a normal deprecation notice. For most observed use cases, Manifest concluded that model routing was not worth the uncertainty it introduced.

The Failure Mode

The first problem is that task complexity is not contained in the prompt. A request like “evaluate the tests for this repo and improve them” can be trivial for a static site and brutal for a kernel-scale codebase. The true difficulty appears only after tools run, files are inspected, tests fail, dependencies break, or the agent discovers the shape of the system.

That undercuts prompt-only routing. The router is being asked to classify a task before the evidence needed for classification exists.

The second problem is caching. Cache reads are 75% to 90% cheaper than uncached inputs. In real agent sessions, system prompts and conversation history become large prefix blocks. Once a model has the cache, switching away from it can erase much of the theoretical saving from routing to a cheaper model.

The practical router therefore becomes sticky. It keeps sending work to the same model because cache economics reward staying put. At that point, the router is reducing cost by not routing.

Cost Moves Elsewhere

The harder cost is operational. A model switch changes behaviour. Different models interpret instructions differently, call tools differently, recover from errors differently, and produce different levels of verbosity. In automated workflows, that variance becomes an evals problem, an observability problem, and a debugging problem.

For coding agents, behaviour consistency is not a nicety. Teams tune prompts, policies, tool schemas, and repair loops around specific model quirks. A router that silently changes the model midstream can save on tokens while adding hidden labour in test maintenance and incident analysis.

That is the core claim in Manifest’s reversal: the amount saved can be paid somewhere else, and that second bill is harder to measure.

Why This Does Not Kill Routing

This is not a universal indictment of routers. Some workloads are cleanly separable: summarisation, extraction, classification, short answers, long-context retrieval, and fallback routing during provider outages. Model routing can still make sense when task classes are known ahead of time and the system can tolerate behaviour differences.

The warning is narrower and more useful. Autonomous agent workflows are the worst case for naive prompt-level routing because the task evolves during execution. The prompt is only the trigger. The expensive context arrives later.

That distinction matters because routing has become one of the default answers to frontier-model cost. Augment, OpenRouter, Vercel, gateway vendors, and internal platform teams are all building routing layers. The sales pitch is simple: send easy work to cheap models, hard work to expensive models. Manifest’s data says the prompt boundary is often the wrong place to decide.

The New Routing Bar

The next generation of routers has to prove more than cost reduction on paper. It has to account for cache stickiness, model-behaviour drift, evaluation overhead, task discovery after tool use, and the price of debugging a workflow whose model can change under it.

That pushes routing toward explicit workflow design rather than invisible automation. Teams can still route by task type, tenant, risk level, latency target, or compliance zone. What looks weaker after Manifest’s reversal is the generic “complexity classifier” sitting in front of every request and guessing from the first prompt.

The market is not done with routers. But the cheap version of the idea just lost a real production case study.