FrontierSWE v2 October Snapshot: GPT-6 Astra Leads 20 Models at 65.5%
BenchLM published its October 6 snapshot of FrontierSWE v2, a 34-task engineering benchmark built for ultra-long-horizon software tasks that remain far from saturated even at the frontier. Twenty models are ranked. GPT-6 Astra leads at 65.5%, with Anthropic placing three models in the top five.
October 6 Leaderboard
| Rank | Model | Score |
|---|---|---|
| 1 | GPT-6 Astra | 65.5% |
| 2 | Claude Opus 5.5 | 62.3% |
| 3 | Claude Sonnet 5.5 | 61.9% |
| 4 | Claude Fable 5.1 | 56.3% |
| 5 | Gemini 4 Argon | 55.0% |
| 6 | Claude Opus 5 | 52.0% |
| 7 | Claude Fable 5 | 47.0% |
| 8 | GPT-5.6 Sol | 32.2% |
| 9 | GLM-5.3 (Z.AI) | 30.2% |
| 10 | Grok 4.7 | 29.5% |
| 11 | Kimi K3 | 25.9% |
| 12 | Grok 4.6 | 25.3% |
All scores are mean@5 unless noted. BenchLM uses the “proximus” evaluation scaffold across all models for consistent comparison.
What FrontierSWE v2 Measures
FrontierSWE v2 is a 34-task expansion of the original FrontierSWE benchmark, targeting multi-day engineering and research tasks that require sustained context, multi-file coordination, and specification discovery. It was designed specifically to avoid the saturation problem that has rendered several earlier coding benchmarks uninformative at the frontier: at the top of the October snapshot, a 13-point spread separates rank 1 from rank 6, which is enough signal to meaningfully differentiate current frontier models.
The benchmark is distinct from SWE-bench Verified, which tests bug-fix resolution on real GitHub issues using a 500-instance filtered set. FrontierSWE v2 tasks are longer-horizon and less well-specified, closer to the work a software engineering team would hand to an agent with minimal scaffolding.
The Anthropic Cluster
Three Anthropic models occupy ranks 2 through 4. Claude Sonnet 5.5 at 61.9% — within 0.4 points of Claude Opus 5.5 — is the notable result: a lower-cost model running near-parity with Anthropic’s flagship on a hard, long-context engineering task. Claude Fable 5.1, Anthropic’s agentic-specific release, lands at 56.3%, five points behind Sonnet 5.5, suggesting that Fable’s agentic tuning may optimize for a task distribution that diverges from FrontierSWE v2’s evaluation structure.
Gemini 4 Argon sits at 55.0%, clustering with Claude Fable 5.1 at fifth. Google’s Argon is the only non-Anthropic model in the top five outside GPT-6 Astra.
The Gap After Rank 7
The drop from rank 7 to rank 8 is sharp. Claude Fable 5 at 47.0% is followed by GPT-5.6 Sol at 32.2% — a 14.8-point gap. Below that, every model through rank 12 sits between 25% and 31%. GLM-5.3 from Z.AI is the only open-weight model in the snapshot, scoring 30.2%.
That cliff after rank 7 maps roughly to the generation boundary between the current frontier cohort (Opus 5.5, Sonnet 5.5, Fable 5.1, Fable 5, GPT-6 Astra, Gemini 4 Argon) and models from the preceding release cycle. FrontierSWE v2 appears to have enough headroom to stay informative as the next generation of models arrives.