Open Weights Match Frontier on Agentic Coding but Trail by 12 Points on Mathematics, LiveBench Shows
The latest LiveBench standings expose a clean capability split that overall rankings obscure: open-weight models have largely closed the agentic coding gap against frontier, but on mathematics they remain 12 points behind — and no amount of fine-tuning appears to bridge it.
The Full Picture
| Model | Agentic Coding | Mathematics | Cost / Task |
|---|---|---|---|
| Claude 5 Opus Thinking (max) | 65.2 | 95.7 | $0.699 |
| Smaug-Agentic (open) | 64.6 | 83.9 | $0.329 |
| Claude Fable 5 (max) | 62.2 | 96.0 | $1.439 |
| Kimi K3 (open) | 62.2 | 84.4 | $0.348 |
| GPT-5.6 Sol (max) | 56.2 | 96.2 | $0.515 |
| GPT-5.5 Thinking xHigh | 54.0 | 95.9 | $0.435 |
On agentic coding, Smaug-Agentic (64.6) is within 0.6 points of Claude 5 Opus (65.2) and actually ahead of Claude Fable 5 (62.2) — while costing half as much per task. Kimi K3 ties Fable 5 exactly at 62.2 for $0.35 against $1.44.
The mathematics column is different. Frontier closed models cluster between 95.7 and 96.2, a band spanning less than a point. Open-weight leaders sit 11.3 to 12.1 points below that at 83.9 and 84.4. The gap is categorical, not marginal.
What Agentic Fine-Tuning Can and Cannot Do
Smaug-Agentic is Abacus.AI’s agentic fine-tune of Moonshot AI’s Kimi K3, a 2.8-trillion-parameter open-weight model optimized specifically for multi-step task execution. That post-training work closed the coding gap almost completely. It did nothing measurable to mathematics.
This is consistent with how the two capabilities are acquired. Agentic coding skill — context tracking, tool invocation, output interpretation — can be reinforced through targeted fine-tuning on execution traces. Mathematical reasoning reflects deep pre-training signal: the volume of formal derivations, proofs, and structured quantitative content a model has processed. Fine-tuning adjusts behavior at the surface; it cannot retroactively increase pre-training depth.
The OpenAI Inversion
The table also shows an inversion that complicates the simple “better model = better agent” assumption. GPT-5.6 Sol scores the highest mathematics score in the table (96.2) and the highest reasoning score (91.7) but the lowest agentic coding score among the top six (56.2) — nine points below Smaug-Agentic, an open-weight fine-tune.
GPT-5.5 Thinking similarly leads or ties on reasoning (89.7) while scoring last on agentic coding (54.0).
LiveBench’s agentic coding tasks measure multi-step pipeline execution over realistic coding environments — closer to what Cursor or Claude Code does in practice than what SWE-bench’s isolated bug-fix format captures. The results suggest OpenAI’s frontier models are calibrated toward formal reasoning benchmarks, while Anthropic’s models and purpose-built agentic fine-tunes outperform them specifically on the multi-step execution dimension.
Deployment Implications
For teams deploying agents on software tasks — code generation, refactoring, debugging, repository navigation — open weights at $0.33-0.35 per task are now a credible frontier-tier option. The 0.6-point agentic coding gap between Smaug-Agentic and Claude 5 Opus is operationally negligible.
For deployments that require strong mathematical reasoning — quantitative research, financial modeling, scientific computing, automated theorem proving — the 12-point mathematics deficit translates to roughly one additional failure in every eight problems where frontier models succeed. That failure rate matters when the agent is running autonomously at scale.
The practical signal: the benchmark you use to evaluate a model determines whether open weights look “frontier-grade” or not. On SWE-bench and agentic coding, they are. On formal mathematics, they are not yet.