CL-SN55 941 —
GPT-56T 861 —
GPT-6S 837 —
GPT-56SC 837 +8.6%
CL-OP55X 831 +8.8%
GROK-46H 831 +8.8%
DSK-V4PH 831 +8.8%
GLM-52 822 +3.9%
GPT-61S 818 —
MUSE-SPK 804 -3.9%
CL-FAB5H 749 +9.8%
GLM-5 749 -3.9%
CL-OP5H 724 +10.2%
CL-OP5X 722 +10.2%
QWEN-38X 722 -3.9%
GEM-37FH 720 +10.3%
KIMI-K3X 718 -3.6%
CL-OP46H 707 +10.5%
CL-OP55H 705 +10.5%
GPT-6A 704 -5.1%
CL-OP47H 696 +10.7%
GEM-38FH 687 -3.5%
GEM-36FH 647 +11.6%
CL-OP48H 644 +11.6%
GEM-35FH 630 +11.9%
GPT-55H 588 +12.6%
GPT-56S 586 -0.5%
CL-OP47 550 -0.5%
INKL 531 —
GEM-3P 495 —
CL-OP46 491 -0.2%
GEM-31P 489 -2%
CL-OP48 485 —
GPT-6L 436 —
GPT-55 398 -2%
CL-SN55 941 —
GPT-56T 861 —
GPT-6S 837 —
GPT-56SC 837 +8.6%
CL-OP55X 831 +8.8%
GROK-46H 831 +8.8%
DSK-V4PH 831 +8.8%
GLM-52 822 +3.9%
GPT-61S 818 —
MUSE-SPK 804 -3.9%
CL-FAB5H 749 +9.8%
GLM-5 749 -3.9%
CL-OP5H 724 +10.2%
CL-OP5X 722 +10.2%
QWEN-38X 722 -3.9%
GEM-37FH 720 +10.3%
KIMI-K3X 718 -3.6%
CL-OP46H 707 +10.5%
CL-OP55H 705 +10.5%
GPT-6A 704 -5.1%
CL-OP47H 696 +10.7%
GEM-38FH 687 -3.5%
GEM-36FH 647 +11.6%
CL-OP48H 644 +11.6%
GEM-35FH 630 +11.9%
GPT-55H 588 +12.6%
GPT-56S 586 -0.5%
CL-OP47 550 -0.5%
INKL 531 —
GEM-3P 495 —
CL-OP46 491 -0.2%
GEM-31P 489 -2%
CL-OP48 485 —
GPT-6L 436 —
GPT-55 398 -2%
← Back to feed

DeepSWE Benchmark: GPT-5.5 at 70%, Claude Opus 4.7 at 54% — 16-Point Gap on Uncontaminated Tasks

Datacurve’s DeepSWE benchmark launched on May 29, 2026, with a single design goal: find where frontier coding agents actually separate once training-set contamination is removed.

The results:

ModelDeepSWE Score
GPT-5.570%
GPT-5.456%
Claude Opus 4.754%

For comparison, GPT-5.5 and Claude Opus 4.7 were effectively within a few points of each other on SWE-bench Verified (both above 80%). On DeepSWE, GPT-5.5 opens a 16-point lead over Claude Opus 4.7.

Why the Gap Widens

SWE-bench and SWE-bench Pro source issues from public GitHub repositories — the same repositories that frontier model training sets draw from. Models with longer training cutoffs or broader GitHub coverage have structural advantages unrelated to actual coding capability.

DeepSWE uses original tasks created by Datacurve. Three structural differences:

Shorter prompts, harder work. DeepSWE prompts are half the length of SWE-bench Pro prompts, but solutions require 5.5x more code and approximately 2x more output tokens. The model has to determine what the problem actually is, not just where to edit.

Repo-level reasoning required. Tasks require searching the repository structure, understanding the design, editing multiple files, and avoiding regressions — not applying a targeted patch to a clearly identified location.

Fresh grading. Rather than reusing tests from a merged PR, DeepSWE checks whether the requested behavior works, crediting valid alternative implementations. The grading reflects intent, not implementation match.

Contamination Is Measurable

The contamination concern is not theoretical. SWE-Rebench — a separate study published in May — found DeepSeek V3 scores drop 46% on fresh test instances compared to the public SWE-bench leaderboard. That is not a rounding error; it is a different model.

GPT-5.5 at 70% on a fresh benchmark is a meaningful data point. The 16-point lead over Claude Opus 4.7 suggests that at least part of the compressed gap on SWE-bench Verified reflects contamination, not genuine parity.

Claude Opus 4.8 launched on May 28, one day before DeepSWE. Its performance on this benchmark has not yet been published — that number is the next relevant data point for assessing whether Anthropic’s compute and architecture improvements close the gap on uncontaminated evaluation.