← Back to feed

Arena.ai’s October 2 leaderboard update is notably concentrated: four model additions in a single day, with the two most significant ones both landing in the Agent Arena simultaneously.

Agent Arena: Sonnet 5.5 vs GPT-6.1 Sol

Claude Sonnet 5.5 (Max) and GPT-6.1 Sol (Max) entered the Agent Arena on October 2. The additions put Anthropic’s current flagship Sonnet and OpenAI’s efficiency-focused reasoning model into direct blind human evaluation for the first time in an agentic context.

The benchmark profiles of the two models reflect different design priorities.

Claude Sonnet 5.5 (Max) has posted strong computer-use numbers: Terminal-Bench 4.0 at 70.6%, CursorBench 4.0 at 55.5%, and OSWorld 2.1 partial-match at 80.1%. Its FrontierCode 1.1 score ranges from 46.2% at Max effort to 52.1% at xHigh effort, and HLE with tools reaches 64.5%.

GPT-6.1 Sol (Max) launched September 29 as OpenAI’s cost-optimized reasoning play. It scores 75.2% on DeepSWE v1.1 at high effort — above GPT-6 Sol’s 68.8% at max effort — while pricing at $2 per million input tokens and $10 per million output, or roughly one-fifth the cost of GPT-6 Astra. Its OSWorld 2.0 offline score of 71.4% compares to Astra’s 73.5% at approximately one-seventh the cost. LiveBench overall is 81.6.

With both now in the Agent Arena, their initial Net Improvement scores are in: Claude Sonnet 5.5 Max enters at +12.5% (ranked #3), while GPT-6.1 Sol Max enters at +11.23% (ranked #5). The scores place Sonnet 5.5 ahead in Agent Arena’s causal inference metric, despite GPT-6.1 Sol’s stronger DeepSWE numbers.

Text Arena and Vision Arena: Sonnet 5.5 xHigh

claude-sonnet-5.5-xhigh was added to the Text Arena and Vision Arena on October 2, the same day as the Agent Arena additions. A separate entry for the same model variant was added October 1 across additional arena tracks. The xhigh effort tier corresponds to extended thinking with higher compute allocation.

Image Leaderboards: Grok Imagine Image 2.0

grok-imagine-image-2.0 entered Arena’s Text-to-Image and Image Edit leaderboards on October 2. xAI’s image generation model now competes directly against other commercial image generators in Arena’s blind preference framework, which uses pairwise human votes rather than automated scoring.

Context

Agent Arena evaluates models on multi-step tasks requiring sustained planning and tool use over several turns. Unlike Arena’s text and image leaderboards, which use blind pairwise preference voting, Agent Arena scores models using a causal inference metric — Net Improvement — derived from interaction telemetry and behavioral signals. Both Claude Sonnet 5.5 Max and GPT-6.1 Sol Max had strong self-reported benchmark profiles before entering; the October 2 scores are the first external, methodology-consistent read on how each performs against the broader Agent Arena cohort.

The previous Agent Arena cohort from late September included GPT-6 Sol Max, which entered alongside five other models across three leaderboards simultaneously. October 2’s additions are fewer in number but the Sonnet 5.5 vs GPT-6.1 Sol matchup is the most commercially significant head-to-head the Agent Arena has seen since Astra and Fable 5 entered earlier in Q3. Both October 2 entrants are already ranked: Sonnet 5.5 Max at #3, GPT-6.1 Sol Max at #5.