← Back to feed

In April 2026, Augment Code submitted its Auggie CLI agent to Scale AI’s SWE-bench Pro evaluation and posted 51.80% — the highest score of any agent tested on the benchmark. The runner-up was Cursor at 50.21%. Claude Code, built by Anthropic, came third at 49.75%. OpenAI Codex trailed at 46.47%.

The result would be unremarkable except for one detail: Auggie, Cursor, and Claude Code all ran the same underlying model — Claude Opus 4.5. The model did not determine the outcome. The scaffolding did.

What the Benchmark Measures

SWE-bench Pro is Scale AI’s hardest software engineering evaluation. It presents agents with real-world GitHub issues drawn from production codebases — multi-file bugs, system-level refactors, dependency interactions that require understanding large amounts of existing code before writing a line. A 51.80% resolution rate means Auggie resolved 379 of 731 issues autonomously.

The benchmark is deliberately adversarial toward agents that work well on isolated, self-contained problems. It rewards models that can orient inside a large, unfamiliar codebase and determine which files to touch.

Augment’s Differentiator: Context Engine

Augment Code’s proprietary Context Engine indexes codebases with up to 400,000 files. In the SWE-bench Pro run, the engine’s semantic indexing produced better code retrieval: the right context reached the model before it attempted to generate a fix. Auggie solved 15 more problems than Cursor and 17 more than Claude Code, despite the identical underlying model, on a benchmark designed around exactly this kind of contextual orientation.

This is not a marginal instrument advantage. On a 731-problem benchmark where frontier models separate themselves by fractions of a percent, 17 problems is a decisive margin. GLM-5.1, the current SWE-bench Pro leader at 58.4%, achieves its lead through a purpose-built post-training pipeline for coding distributions and an eight-hour autonomous execution capability — a different category of improvement. Within the same model tier, Augment’s gap over Claude Code is entirely architectural.

The Benchmarking Shift

When OpenAI released GPT-4 in 2023, the benchmark question was: which model is strongest? By 2026, with frontier models clustered within a few SWE-bench points of each other, the more important question is: which system surrounding the model extracts the most from it?

Cursor’s 50.21% vs. Augment’s 51.80% is 1.59 percentage points. In absolute terms that’s 11 additional GitHub issues resolved. For teams running agents on large internal codebases where Augment’s index advantage compounds — the company’s documentation cites meaningful gains above 50,000 lines of code — that margin likely widens further than benchmark conditions capture.

Competitive Position

Augment Code prices its platform at $20/month for individual developers (40,000 credits) and $60/month per developer on standard team plans. Claude Code operates on a pay-per-token model. Cursor competes directly on IDE integration.

The broader implication: as leading labs converge on similar model capabilities, the sustainable competitive moat in coding tools is shifting toward context retrieval infrastructure. This is the same dynamic driving GitHub Copilot’s continued investment in repository indexing despite Copilot running models from OpenAI — a vendor with more raw model capability than the index can easily leverage.

SWE-bench Pro rewards exactly the capability enterprises care most about: autonomous resolution of the kind of work that fills real engineering backlogs. Augment’s April result is the clearest public evidence yet that the scaffolding layer, not the model layer, is where the next round of coding tool differentiation will be fought.