Terminal-Bench 3.0 Ships: Frontier Agents Drop From 84% to 34% as the Bar Resets
Terminal-Bench 3.0 went live on July 23, 2026. The top score on the new leaderboard, a 34% from both GPT-5.6 Sol Max and Claude Fable 5 Max, would have been a failure on the benchmark it replaced.
Why a New Version Was Needed
Terminal-Bench 2.0 launched in November 2025 as a harder version of the original. By spring 2026, frontier models were clearing roughly half the tasks, and the best scaffold-and-model combinations were pushing toward 85%. At that pass rate, the benchmark stops discriminating: a model that solves 82% of tasks and one that solves 79% look nearly identical on a chart where the hard ceiling is approaching.
The benchmark’s maintainers describe the problem directly: “Once a benchmark gets that close to solved, it stops telling you much.” Terminal-Bench 3.0 pushes task difficulty higher to keep the benchmark measuring frontier capabilities rather than trailing them.
What Changed
The benchmark runs inside Docker sandboxes. An agent receives a plain-language instruction, drives a real shell, reads its output, and works the problem. Scoring ignores what the agent reports about its own progress and checks the end result: did the code compile, did the server come up, did the pipeline return the right answer?
Tasks span 16 categories — software engineering, security, scientific computing, data science, machine learning, systems administration, web configuration, hardware and kernel work, games, and debugging — at easy, medium, and hard difficulty levels. Scale AI was the largest single organizational contributor at launch, contributing primarily science-focused domain workflows: “end-to-end, tooling-heavy problems that are hard to specify and easy to fake your way through if the checks are loose.”
Every task goes through automated checks before human review, reviewer involvement at every stage of task creation, and failure-mode analysis of agent rollouts. A task is kept only when the verifier holds up.
Current Leaderboard
The scores are vendor-reported from recent model launches, not yet from an independently verified public leaderboard submission, but they represent the first wave of frontier evaluations on the new benchmark:
| Model | Terminal-Bench 3.0 |
|---|---|
| GPT-5.6 Sol Max | 34% |
| Claude Fable 5 Max | 34% |
| Grok 4.6 | 26% |
| Grok 4.5 | 15.7% |
The equivalent leaders on Terminal-Bench 2.1 had scores above 84%. On Terminal-Bench 2.0, the scaffold record was 90.2% using Claude Opus 4.7 with the VIX scaffold.
The Reset’s Implications
The 50-point drop in top scores is not a sign that models regressed. The task set is genuinely harder, targeting domains where large models still struggle to execute reliably across multi-step workflows: long dependency chains, ambiguous failure modes, and domain-specific verification criteria that the model cannot hallucinate its way through.
The GPT-5.6 Sol Max and Fable 5 Max tie at 34% is meaningful: two labs, two different training regimes, the same ceiling. Grok 4.6’s 26% puts it within 8 points of the leaders despite launching without a corresponding jump in model scale. Grok 4.5 at 15.7% illustrates how sharply the distribution collapses below the frontier tier.
Scale AI’s VP of Engineering summarized the benchmark’s position: “The best frontier models can solve less than 40% of the tasks. That rigor is why Scale is one of its largest contributors.” The current public leaderboard at tbench.ai still reflects Terminal-Bench 2.0 results; 3.0 submissions are being accepted and the leaderboard is expected to fill in as labs submit formal runs.
Context
Terminal-Bench matters more than most coding benchmarks because it runs in a real shell rather than a constrained evaluation harness. That means pass rates are sensitive to the full agent loop, not just the model’s code generation quality in isolation. SWE-bench Verified and DeepSWE measure issue resolution on real repositories; Terminal-Bench measures whether an agent can operate a computer under ambiguous, domain-specific conditions.
With frontier SWE-bench scores now in the 93-97% range, Terminal-Bench 3.0 may be the field’s most credible remaining measure of practical agent capability at the frontier — at least until those numbers approach 40%.