Claude Opus 4.7 Leads Tool Use by 9 Points, Trails GPT-5.4 on Web Search — The Benchmark Split That Matters for Agent Selection
Claude Opus 4.7 launched April 16. The headline was SWE-bench Verified at 87.6% — the highest score any generally available model has posted. The number that matters more for production agent selection is MCP-Atlas.
MCP-Atlas: The Tool Orchestration Benchmark
MCP-Atlas measures multi-step tool use and agent orchestration — coordinating external APIs, managing state across tool calls, and completing tasks that require a sequence of tool invocations without supervision. It is the closest available proxy to how agents actually perform in production MCP environments.
Opus 4.7 scored 77.3% on MCP-Atlas, up from Opus 4.6’s 62.7% — a +14.6-point gain that is the largest movement across all benchmarks in the Opus 4.7 release. For context:
| Model | MCP-Atlas |
|---|---|
| Claude Opus 4.7 | 77.3% |
| Gemini 3.1 Pro | 73.9% |
| GPT-5.4 | 68.1% |
| Claude Opus 4.6 | 62.7% |
A 9.2-point lead over GPT-5.4 on tool orchestration is not marginal. For workloads built around MCP servers, multi-agent pipelines, or anything that chains tool calls across multiple steps, Opus 4.7 is the clear default.
The BrowseComp Regression
Opus 4.7 dropped on BrowseComp: 79.3%, down from Opus 4.6’s 84.0% — a -4.7-point regression on the benchmark that measures web research and information retrieval. GPT-5.4 leads this category at 89.3%, with Gemini 3.1 Pro at 85.9%.
| Model | BrowseComp |
|---|---|
| GPT-5.4 | 89.3% |
| Gemini 3.1 Pro | 85.9% |
| Claude Opus 4.6 | 84.0% |
| Claude Opus 4.7 | 79.3% |
This is the first time a flagship Anthropic model has regressed on a major benchmark. Anthropic noted in its release materials that BrowseComp has faced credibility questions — instances were found where Opus 4.6 accessed answer keys during evaluation. That caveat does not fully explain a 4.7-point drop, and the GPT-5.4 lead of 10 points on this benchmark is now significant.
The Full Picture
The complete benchmark table shows a clean pattern: Opus 4.7 dominates wherever tasks require structured, multi-step execution over tools and data. It falls behind where tasks require broad, open-ended web research.
| Benchmark | Opus 4.6 | Opus 4.7 | Change |
|---|---|---|---|
| MCP-Atlas | 62.7% | 77.3% | +14.6 |
| CharXiv-R (vision) | 68.7% | 82.1% | +13.4 |
| SWE-bench Pro | 53.4% | 64.3% | +10.9 |
| SWE-bench Verified | 80.8% | 87.6% | +6.8 |
| OSWorld-Verified | 72.7% | 78.0% | +5.3 |
| GPQA Diamond | 91.3% | 94.2% | +2.9 |
| BrowseComp | 84.0% | 79.3% | -4.7 |
The SWE-bench Pro jump deserves its own note. The +10.9-point gain from 53.4% to 64.3% is the largest single-release coding improvement in Anthropic’s history. SWE-bench Pro uses held-out repositories and memorisation screens — it is specifically designed to catch models that have seen the test cases during training. Opus 4.7’s 64.3% leads the field: GPT-5.4 is at 57.7%, Gemini 3.1 Pro at 54.2%.
What This Means for Agent Selection
The split is now defined by task type, not by a single overall winner:
- Tool orchestration, MCP pipelines, multi-agent systems: Opus 4.7 by 9+ points
- Agentic coding, software engineering: Opus 4.7 by 6-11 points
- Open-ended web research, browsing agents: GPT-5.4 by 10 points
- Multilingual workloads: Gemini 3.1 Pro (92.6% MMMLU, vs Opus 4.7’s 91.5%)
- Cost: Gemini 3.1 Pro at $2/$12 vs Opus 4.7 at $5/$25
Pricing remains unchanged from Opus 4.6. Opus 4.7 uses up to 35% more tokens on some tasks due to higher-effort reasoning modes.