Computer Use Is 45x More Expensive Than Structured APIs — and Gets the Task Wrong
A benchmark published by Reflex.dev on April 27 put a hard number on the cost of computer use versus structured API access: 45x more expensive, with worse task completion.
The test used Claude Sonnet running against the same admin panel in two configurations. Path A: vision agent driving the UI via browser-use 0.12, taking screenshots and executing clicks. Path B: API agent with tool-use, calling the same HTTP handlers the UI calls. Same model, same dataset, same task. The interface was the only variable.
The Task
Find the customer named “Smith” with the most orders. Locate their most recent pending order. Accept all of their pending reviews. Mark the order as delivered. Three resources, filtering, pagination, cross-entity lookups, reads and writes — the shape of routine internal tool work.
Results
API agent: Completed in 8 calls. Listed reviews filtered by pending status, accepted each one, marked the order as delivered. Full accuracy.
Vision agent: Found one of four pending reviews. Accepted it. Moved on. Never paginated. The remaining three reviews were below the visible fold, and the vision agent had no signal to scroll — it was reasoning about a rendered page, not a structured response.
The API agent’s tool call returns the full result set from the handler. The vision agent sees only what’s currently rendered on screen.
Why This Matters for Agent Cost
The 45x cost difference compounds with scale. Most internal tooling at enterprises spans 20 or more applications. Vision agents are the default because building a per-app MCP or REST surface is its own engineering project for each tool. Teams pay the vision premium not because it performs better, but because the alternative requires upfront investment.
Reflex’s argument — and the implicit thesis of their benchmark — is that if the API surface is generated automatically (as their Python framework does for its state), the economics flip. The engineering overhead disappears, and teams get an agent that costs 2.2 cents per task instead of $1 per task.
The Accuracy Problem
Cost alone understates the gap. An agent that completes 25% of the required actions and reports success is worse than an agent that reports failure — it produces silent errors that reach downstream systems unchecked. In a compliance or financial context, the vision agent’s partial completion is not a degraded outcome; it is a correctness failure.
The benchmark is open source at github.com/reflex-dev/agent-benchmark.
Implications
This is not a novel finding — the overhead of screenshot-based reasoning has been documented since computer use shipped in late 2024 — but the Reflex benchmark is the clearest single-number quantification to date: 45x cost, plus meaningful accuracy loss on tasks involving pagination or state that isn’t fully rendered.
The practical implication for teams building internal tooling agents: structured API surfaces pay for themselves quickly when agents are running at any meaningful volume. At 1,000 tasks per day, the cost delta is $22 versus $1,000. The engineering investment to expose a proper tool surface typically runs days to a few weeks, not months.
The benchmark lands as enterprise adoption of computer use is accelerating. OpenAI’s Codex and Anthropic’s Claude Code both support browser-based agents. The cost curve on vision inference is falling, but at current prices, the structured-API approach remains the dominant economic choice for any internal tool that sees regular agent traffic.