Kimi K3.1 Pro Debuts on Agent Arena at Rank 11 with the Lowest Tool Hallucination Rate on the Board
Kimi K3.1 Pro (Max) entered the Agent Arena leaderboard on October 24, following its Code Arena WebDev addition the day prior. The October 23 snapshot shows 4,071 sessions logged and a preliminary rank of 11 overall. The label is preliminary and will shift as session volume grows, but the early signal distribution is distinctive on one metric in particular.
The Tool Hallucination Number
Across all 55 models currently on Agent Arena, K3.1 Pro records a 0.75% tool hallucination rate — the lowest of any model on the board. Gemini 4 Argon sits at 0.35% in the overall rankings but has significantly more sessions logged (45,840). Most frontier models cluster between 0.6% and 1.0%.
Tool hallucination — calling tools the model doesn’t have, or fabricating tool parameters during execution — is a consistent failure mode in production agent pipelines. A lower rate is a direct reliability signal, not just a leaderboard number.
Overall Agent Arena Standing
| Signal | K3.1 Pro | Claude Opus 5.5 | GPT-6.1 Sol | Gemini 4 Argon |
|---|---|---|---|---|
| Rank (overall) | 11 (prelim.) | 1 | 5 | 6 |
| Net improvement | 7.61% | 20.77% | 13.64% | 11.75% |
| Confirmed success | 2.03% | 22.08% | 13.73% | 13.51% |
| Tool hallucination | 0.75% | 0.57% | 0.75% | 0.35% |
| Cost/task (P50) | $1.06 | $6.25 | $0.80 | $1.07 |
| Time/task (P50) | 6.5 min | 17.8 min | 10.2 min | 7.5 min |
| Output tokens/task | 38.1K | 123.5K | 32.4K | 37.9K |
| Price (out/M tokens) | $10 | $20 | $10 | $10 |
For coding tasks specifically, K3.1 Pro ranks 5th (1,430 sessions), with 9.29% net improvement and a median cost per task of $2.73 — against $13.54 for Claude Opus 5.5 Max and $11.03 for GPT-6 Astra Max.
Context: What the Launch Benchmarks Said
When K3.1 Pro launched on October 4, the headline results were a 67% Terminal-Bench 4.0 score (record at the time), a 312-Elo gain on GDPval-AA over Kimi K3 to reach 1,846 Elo (3rd overall behind Claude Opus 5.5 at 1,866 and Muse Wonder Full at 1,857), and a 4th-place position on the Artificial Analysis Intelligence Index at a score of 55. The Agent Arena debut adds live multi-session data to those static benchmark numbers.
The model runs 2.8T total parameters with 104B active, supports a 1M token context window, accepts text and image inputs, and is priced at $2/$10 per million input/output tokens — 33% below Kimi K3’s prior rates. Weights have not yet been published; Moonshot indicated an open-weight release would follow the API launch.
What the Preliminary Data Does and Doesn’t Show
Four thousand sessions is sufficient to surface pattern differences but not to stabilize rank. The confirmed success rate at 2.03% looks low against top models, but confirmed success on Agent Arena requires explicit user confirmation of task completion, making it sensitive to task-type mix in early deployment. Rank placement typically shifts 2-5 positions as volume grows and task distribution normalizes.
The tool hallucination signal is more robust at small session counts: it measures discrete failures per session rather than user sentiment, making it less sensitive to early distribution skew.
The preliminary coding rank of 5th places K3.1 Pro behind Anthropic (Opus 5.5 Max), OpenAI (GPT-6 Astra Max), Meta (Muse Wonder Full Max), and Google (Gemini 4 Argon High) — the same four labs that hold the top spots overall.