Abacus.AI Fine-Tunes Kimi K3 Into an Agentic Specialist That Beats It on Four Benchmarks
Abacus.AI has released Smaug-Agentic, an open-weight supervised fine-tune of Moonshot AI’s Kimi K3 model aimed at long-horizon coding agents and tool use. It outperforms its base model on every agentic benchmark the company measured, ranks second on LiveBench’s agentic coding dimension behind only Claude Opus 5 Thinking, and costs $0.071 per successful coding task — compared to $0.246 for Claude Fable 5 Max Effort.
What Changed
Smaug-Agentic continues the Smaug fine-tune line that Abacus.AI has run since Smaug-72B, which was the first open-source model to pass an average of 80% on the HuggingFace Open LLM Leaderboard. The base this time is Kimi K3, Moonshot AI’s 2.8-trillion-parameter Mixture-of-Experts model with 96 experts, a 1M-token context window, and integrated vision.
Every architectural parameter is unchanged. Any inference stack that runs Kimi K3 runs Smaug-Agentic as a drop-in replacement.
The training intervention was specific: Abacus.AI filtered multi-turn tool-using coding trajectories and masked reasoning tokens from the loss. The intent was to improve agent behaviour without eroding the base model’s reasoning capability. Interleaved thinking tokens are preserved across multi-step loops. Runaway reasoning chains are suppressed; normal deliberation is not.
The model emits one decisive shell action per turn. That constraint is behavioural, enforced through training data selection rather than post-generation filtering.
Benchmark Results
Smaug-Agentic improves on Kimi K3 reference figures across four benchmarks:
| Benchmark | Smaug-Agentic | Kimi K3 |
|---|---|---|
| GPQA Diamond | 94.1 | 93.5 |
| DeepSWE | 69.9 | 67.5 |
| SciCode | 60.8 | 58.7 |
| LiveBench Agentic Coding | 64.6 | 62.2 |
On LiveBench overall, Smaug-Agentic ranks fifth at 79.5 — four and a half points behind Claude Fable 5 Max Effort (83.0) and one point behind Claude Opus 5 Thinking (80.1). On the agentic coding dimension specifically, the ranking inverts: Smaug-Agentic scores 64.6, second only to Opus 5 Thinking (65.2) and ahead of both Claude Fable 5 (62.2) and GPT-5.6 Sol (56.2).
The divergence between static coding performance and agentic coding performance is instructive. On LiveBench’s standard coding category — code generation and code completion — Smaug-Agentic scores 82.5, trailing Claude Fable 5 (86.0) by 3.5 points. On agentic coding, Smaug-Agentic leads Fable 5 by 2.4 points. The fine-tune improved specifically what it targeted: multi-step agentic task performance, not single-pass code generation.
Terminal-Bench 2.1, scored with the Terminus 2 agent, shows 86.5. Abacus.AI notes a lower figure of 76.4 when scored with Kimi Code, the agent used for the official Kimi K3 number — agent harness choice materially affects results.
Sustained Agent Loops
The behavioural evidence runs beyond benchmark scores. Abacus.AI ran 113 agentic coding tasks across sessions totalling more than seven hours. The model maintained a median of 78 agent steps per task with no infrastructure errors and no timeouts. That sustained loop behaviour, rather than single-task benchmark scores, is the operative capability for production coding agent deployments.
Cost
| Model | Cost / Successful Task (Coding) | Cost / Successful Task (Overall) |
|---|---|---|
| Claude Fable 5 Max Effort | $0.246 | $1.439 |
| GPT-5.6 Sol Max Effort | $0.127 | $0.515 |
| Smaug-Agentic (open) | $0.071 | $0.329 |
The 2.4-point lead Smaug-Agentic holds over Claude Fable 5 on agentic coding comes at 71% lower cost per successful coding task. At the overall LiveBench cost-per-task figure, the gap widens to 77%.
Why This Matters
The standard argument against open fine-tunes of frontier models is that they degrade the capabilities that made the base worth choosing. Abacus.AI’s training approach — masking reasoning tokens, targeting agentic trajectories, leaving architecture unchanged — is designed specifically to avoid that. The benchmark deltas suggest it worked: reasoning performance (GPQA Diamond) held, agentic task performance (DeepSWE, SciCode, LiveBench Agentic Coding) improved.
For teams evaluating coding agent infrastructure, an open-weight model at this capability level with drop-in compatibility for existing Kimi K3 deployments and a cost profile significantly below closed alternatives changes the build-vs-buy calculation.
Smaug-Agentic is available on Hugging Face under the Abacus.AI organisation.