Arena Launches Agent Leaderboard With Causal Tracing — Grok 4.3 Ranks Last, GPT-5.5 High Leads
Chatbot Arena launched its Agent Arena leaderboard on June 4, 2026, introducing a new evaluation methodology called causal tracing and producing a ranking that looks nothing like the existing Arena chat leaderboard.
The central finding: Grok 4.3, which ranks in the high tier on standard Arena chat, finishes last on real-world agent tasks. GPT-5.5 High and Claude Opus 4.7 Thinking sit at the top. The divergence is the point — Arena built Agent Arena precisely because chat pairwise voting is a poor proxy for what agents actually do.
Methodology: Causal Tracing
Standard Arena collects side-by-side pairwise votes: users see two responses and pick a winner. Agent Arena doesn’t use votes at all.
Instead, it collects millions of live interactions from users running Agent Mode on arena.ai — real jobs, including software engineering, financial analysis, and multi-step research. Each interaction is treated as a trace through a multi-component system, with the model and harness representing separate intervention points.
Rankings are derived through causal tracing: Arena randomises component selections across user sessions, then measures the causal treatment effect of each model on observed outcomes. Signals include task success rates, verbal feedback, tool error recovery, and tool hallucination rates. The output is a net improvement coefficient — a model’s causal lift on task completion relative to the baseline — rather than a win percentage.
The methodology is structurally similar to a multi-arm randomised controlled trial run on production traffic.
Leaderboard
From bottom to top (negative to positive net improvement, with 95% confidence intervals):
| Rank | Model | Net Effect |
|---|---|---|
| 1 (best) | GPT-5.5 High | ~+8% |
| 2 | Claude Opus 4.7 Thinking | ~+6% |
| 3 | GPT-5.4 High | ~+5% |
| 4 | Claude Opus 4.7 | ~+3% |
| 5 | Claude Opus 4.6 | ~+2% |
| … | GPT-5.5 | ~+1% |
| … | Claude Sonnet 4.6 | ~0% |
| … | GLM 5.1 | slight negative |
| … | Gemini 3.1 Pro Preview | negative |
| … | Gemini 3.5 Flash | negative |
| … | Kimi K2.6 | negative |
| … | DeepSeek V4 Pro | negative |
| … | Qwen 3.6 Plus | negative |
| … | DeepSeek V4 Flash | negative |
| … | MiniMax M2.7 | negative |
| … | Gemini 3 Flash | negative |
| … | Gemma 4 31B | strongly negative |
| Last | Grok 4.3 | ~-22% |
Grok 4.3’s result is the most striking. On standard Arena, the model scores in the top tier on chat quality and instruction following. On agentic tasks — where the model must chain tool calls, recover from errors, and maintain coherent multi-step plans — it posts the largest negative treatment effect in the study.
Why This Matters
Chat leaderboards measure the quality of a single response to a well-formed prompt. Agents operate differently: they make sequences of decisions, call tools, receive feedback, and must adapt. A model that writes excellent prose but miscounts tool calls or loses track of task state over a long horizon will fail at agent work regardless of its Arena ELO.
The Agent Arena results suggest a significant performance gap between the models that optimised for chat quality and those that specifically developed agentic capability. The models at the top — GPT-5.5 High and Claude Opus 4.7 Thinking — have both been explicitly tuned with agentic benchmarks (Terminal-Bench, SWE-bench) as first-class targets. Grok 4.3, which launched with strong chat and instruction-following results, evidently did not optimise for the tool-use and multi-step planning signals that dominate real agent sessions.
Arena says it will expand the signal set over time to include more task categories and more complex tool interactions. The methodology also opens the door to ranking harnesses separately from models — a distinction that is largely invisible in current benchmarking.