← Back to feed

Microsoft Research published Webwright on May 24 — an open-source web agent that replaces the stateful browser session model with a terminal. The agent writes Playwright code to control browsers, runs bash commands, inspects logs, and iteratively refines scripts. The result: 60.1% on Odysseys, the current SOTA benchmark for long-horizon browsing tasks.

Previous leader: Claude Opus 4.6 at 44.5%, set in the April 2026 leaderboard. Webwright running on GPT-5.4 improves on that by 15.6 absolute points — a 35.1% relative gain over the old SOTA and a 79.4% relative improvement over GPT-5.4 with no scaffold (33.5%).

The Architecture Difference

Current web agents generally operate by predicting coordinate clicks or element interactions at each step. They maintain a browser session and reason about what they see in the current viewport. The failure mode is well-documented: each step decision is made without access to the full history of what worked and what did not.

Webwright takes a different path. The agent operates through a terminal and writes code to automate browser behavior. That code can be run, debugged, and revised. The agent builds up a program rather than issuing a sequence of point-and-click commands. Errors produce logs. Logs feed back into the next iteration.

The key property is that code is inspectable and composable in ways that step-by-step coordinate prediction is not. If something fails, the agent can read exactly why.

Benchmark Results

Odysseys (long-horizon browsing, average 272 words of instructions per task, multiple sites):

• Webwright + GPT-5.4: 60.1%
• Claude Opus 4.6 (previous SOTA, April 2026): 44.5%
• GPT-5.4 with no scaffold: 33.5%

Online-Mind2Web (300 tasks across 136 widely used sites, 100-step budget):

• GPT-5.4 with Webwright: 86.67% (highest in AutoEval category)
• Claude Opus 4.7: 84.7% overall, stronger on hard tasks — 80.5% vs GPT-5.4’s 76.6%

Odysseys is the more demanding benchmark. Tasks span multiple websites and require sustained planning over many steps. The 60.1% score is a notable shift in what open-source scaffolding can extract from a frontier model.

What It Means for Agent Benchmarks

Webwright is the latest evidence that scaffold architecture matters as much as model capability in long-horizon tasks. The underlying model is GPT-5.4, which sits in the middle of the frontier tier — not the strongest model available. The 60.1% Odysseys score comes from the harness, not from model capability improvements.

This continues a pattern visible across coding benchmarks: vix on Terminal-Bench (90.2% with Opus 4.7), JJAgent with multi-model routing (87.1%), Cursor Composer 2.5 (62 on AA Coding Agent Index at $0.07/task). In each case, thoughtful scaffold engineering closes more of the gap with frontier capability than raw model upgrades do.

Microsoft has open-sourced Webwright under an MIT-adjacent license. The framework targets production deployments where browser automation reliability matters — not one-off demos.

---

Webwright is built on Playwright, which is also a Microsoft Research project. Benchmark evaluations on Online-Mind2Web use an LLM-as-a-Judge framework.