Meta Paper: Test-Time Scaling for Coding Agents Is a Representation Problem, Not a Compute Problem
Scaling test-time compute works for short-answer problems because candidate outputs are small enough to compare directly. Coding agents break this premise: each attempt produces a long, messy trajectory of file reads, shell commands, failed patches, and abandoned hypotheses. Ranking five such trajectories against each other is hard. Extracting useful signal from them is harder.
A paper from Meta, published on arxiv April 16 (arXiv:2604.16529), addresses that problem directly. The core proposal: convert each agent rollout into a compact structured summary capturing its main hypothesis, progress made, and failure modes. Use those summaries — not the full traces — for both selection and conditioning.
The Two Mechanisms
Recursive Tournament Voting (RTV) handles parallel scaling. Instead of ranking all candidate attempts simultaneously, it runs small-group comparisons recursively, narrowing the field tournament-style. Small comparisons are more tractable than full-field rankings when candidates are complex.
Parallel-Distill-Refine (PDR) handles sequential scaling. After a round of attempts, summaries of the most promising runs are distilled and fed back as context for a fresh round. New attempts start with prior knowledge of what has already been tried and where it broke.
Both mechanisms depend on the quality of the summary representation. The paper’s central finding is that the bottleneck for long-horizon agent scaling is not generation capacity but representation quality — whether prior experience can be stored in a form that the agent can actually use.
Benchmark Results
Testing on Claude 4.5 Opus with mini-SWE-agent and Terminus 1 scaffolds:
| Benchmark | Baseline | With RTV + PDR | Delta |
|---|---|---|---|
| SWE-Bench Verified | 70.9% | 77.6% | +6.7pp |
| Terminal-Bench v2.0 | 46.9% | 59.1% | +12.2pp |
The Terminal-Bench gain of 12.2 percentage points is particularly significant. Terminal-Bench tests multi-step shell and filesystem tasks with real execution environments — it is closer to actual deployed agent work than SWE-Bench’s code repair framing.
These results do not replace current leaderboard baselines. The SWE-bench Verified leaderboard compares systems at single-attempt, so RTV + PDR is an inference-time overlay, not a new baseline for Claude 4.5 Opus itself.
What This Changes for Scaffold Engineering
The implications are practical. Any team running coding agents in production is already implicitly dealing with failed attempts — re-runs, context exhaustion, degraded performance on long sessions. The paper suggests that structuring those failure modes explicitly and feeding them back as inputs can recover substantial performance.
This also reframes why top coding agent scaffolds consistently outperform naive model-plus-prompt setups. The paper’s evidence is that the gains are not primarily from better prompting or model selection. They come from better handling of prior attempts.
The broader takeaway: frontier model capability is not fully exploited in single-shot agent loops. At the top of SWE-Bench Verified, the gap between 70.9% and 77.6% is the same gap that separates several named frontier models. A scaffold technique recovered it in inference, with no model change required.