DeepSWE Benchmark: GPT-5.5 at 70%, Claude Opus 4.7 at 54% — 16-Point Gap on Uncontaminated Tasks
Datacurve’s DeepSWE benchmark launched on May 29, 2026, with a single design goal: find where frontier coding agents actually separate once training-set contamination is removed.
The results:
| Model | DeepSWE Score |
|---|---|
| GPT-5.5 | 70% |
| GPT-5.4 | 56% |
| Claude Opus 4.7 | 54% |
For comparison, GPT-5.5 and Claude Opus 4.7 were effectively within a few points of each other on SWE-bench Verified (both above 80%). On DeepSWE, GPT-5.5 opens a 16-point lead over Claude Opus 4.7.
Why the Gap Widens
SWE-bench and SWE-bench Pro source issues from public GitHub repositories — the same repositories that frontier model training sets draw from. Models with longer training cutoffs or broader GitHub coverage have structural advantages unrelated to actual coding capability.
DeepSWE uses original tasks created by Datacurve. Three structural differences:
Shorter prompts, harder work. DeepSWE prompts are half the length of SWE-bench Pro prompts, but solutions require 5.5x more code and approximately 2x more output tokens. The model has to determine what the problem actually is, not just where to edit.
Repo-level reasoning required. Tasks require searching the repository structure, understanding the design, editing multiple files, and avoiding regressions — not applying a targeted patch to a clearly identified location.
Fresh grading. Rather than reusing tests from a merged PR, DeepSWE checks whether the requested behavior works, crediting valid alternative implementations. The grading reflects intent, not implementation match.
Contamination Is Measurable
The contamination concern is not theoretical. SWE-Rebench — a separate study published in May — found DeepSeek V3 scores drop 46% on fresh test instances compared to the public SWE-bench leaderboard. That is not a rounding error; it is a different model.
GPT-5.5 at 70% on a fresh benchmark is a meaningful data point. The 16-point lead over Claude Opus 4.7 suggests that at least part of the compressed gap on SWE-bench Verified reflects contamination, not genuine parity.
Claude Opus 4.8 launched on May 28, one day before DeepSWE. Its performance on this benchmark has not yet been published — that number is the next relevant data point for assessing whether Anthropic’s compute and architecture improvements close the gap on uncontaminated evaluation.