Red Hat's RH SWE-bench: Top Coding Agents Drop 20 Points on Enterprise Go and C Code
Red Hat’s Emerging Technologies team published RH SWE-bench on October 21, 2026. It is a benchmark of 357 coding tasks drawn from 25 Red Hat ecosystem repositories, written in Go, Python, and C. The goal is to measure whether coding agents can handle enterprise software that does not look like the Python packages dominating standard SWE-bench.
Why Standard Benchmarks Miss Enterprise Code
SWE-bench Verified (500 Python tasks) and its variants have become the primary signal for agentic coding capability. The top models on the standard leaderboard now solve 80 to 97% of the Python tasks. That number describes Python GitHub issues, not the Go controllers, Ansible modules, and C monitoring tools that large organizations actually maintain.
Go’s strict type system, interface patterns, and multi-module project structure trip up agents trained primarily on Python. Kubernetes controllers, for example, require changes to cross-module interfaces and compile-time type contracts that are uncommon in Python open-source repositories.
Construction
Red Hat started with approximately 6,800 candidate instances generated through SWE-Smith mutation strategies. After validation — confirming that tests fail before the fix and pass after — only about 7% survived, yielding 531 verified instances. Aggressive quality filtering removed 233 more low-quality tasks (trivial single-function mutations, cheap solves, single-file patches), and 78 real PR-derived tasks from REPO2RL were added.
The result: v1.0 with 357 tasks across 25 repositories. A revised v1.1 (337 tasks across 24 repositories) removes flaky or privileged-access checks and updates instructions. No model has been re-run on the v1.1 prompts yet.
Results
| Model | Harness | Solve Rate |
|---|---|---|
| Claude Opus 4.6 | Claude Code | 63.3% |
| Claude Sonnet 4.6 | Claude Code | 55.7% |
| Qwen3.6-27B-FP8 | Claude Code | 49.3% |
| Qwen3.6-27B-FP8 | Pi | 46.8% |
| Qwen3.6-27B-FP8 | OpenCode | 44.0% |
| GPT-OSS-120B (MXFP4) | Claude Code | 31.7% |
| Nemotron-120B (NVFP4) | OpenCode | 30.8% |
| GPT-OSS-120B (MXFP4) | OpenCode | 29.4% |
| GPT-OSS-120B (MXFP4) | Pi | 23.0% |
| Nemotron-120B (NVFP4) | Claude Code | 22.4% |
| Nemotron-120B (NVFP4) | Pi | 21.6% |
Claude Opus 4.6 at 63.3% is the top result. On standard SWE-bench Verified, Opus 4.6 scores 80.8%. The 17-point drop reflects the harder task distribution, not model degradation.
Three Findings Worth Noting
Go is harder than Python. Enterprise repositories using Go’s type system and multi-module layouts produce failures that differ structurally from Python bug fixes. Agents that reason about method signatures and type hierarchies struggle more here.
Harness choice moves scores 9 points. Nemotron-120B scores 30.8% under OpenCode and 21.6% under Pi — the same model, same tasks, different scaffolding. This is a known issue across agentic evaluations but the magnitude matters: harness choice can cross-rank two models.
A 27B model beats two 120B models. Qwen3.6-27B-FP8 at 49.3% under Claude Code outperforms both GPT-OSS-120B and Nemotron-120B by 17 to 18 points. Parameter count is not the bottleneck on enterprise code; training data distribution and tool-use design matter more.
Context: Newer Models Not Yet Tested
The evaluation uses Opus 4.6 and Sonnet 4.6 — both from April 2026 and no longer at the frontier. The benchmark hasn’t been run on Claude Fable 5.1, Claude Opus 5.5, GPT-6 Astra, or Gemini 4 Argon. Those models score 95%+ on standard SWE-bench Verified; their enterprise scores on RH SWE-bench remain unknown.
Red Hat says the benchmark is available for any team to run. The methodology and task set are described in their Emerging Technologies blog. Whether frontier labs pick it up will determine whether enterprise Go code becomes a tracked dimension alongside Python-only evaluations.