Inkling-Small Enters Agent Arena Three Weeks After Beating Its 975B Parent on HLE
Thinking Machines Lab’s Inkling-Small joined Arena’s Agent Arena leaderboard on August 18, bringing the 276B mixture-of-experts model into head-to-head competitive evaluation for the first time.
The model launched July 31 with numbers that stood out: 80.2% on SWE-bench Verified and 31.6% on HLE—both higher than the 975B Inkling-large (77.6% SWE-bench, 29.7% HLE). With 12B active parameters against Inkling-large’s 41B active, the inversion holds across two separate benchmarks.
Numbers at Arena Entry
| Metric | Inkling (975B MoE) | Inkling-Small (276B MoE) |
|---|---|---|
| SWE-bench Verified | 77.6% | 80.2% |
| HLE | 29.7% | 31.6% |
| Active params | 41B | 12B |
| Context | 1M tokens | 1M tokens |
| License | Apache 2.0 | Apache 2.0 |
Architecture: distilled from Inkling with two additional weeks of agentic reinforcement learning fine-tuning. Multimodal: text, image, and audio inputs.
Why This Evaluation Matters
Agent Arena uses human preference voting across real-world agentic tasks—a different test from static benchmarks. In practice, benchmark accuracy and human preference diverge often enough that the correlation is unreliable at the frontier. Models with strong SWE-bench scores can lose head-to-head on coherence, tool-call efficiency, error recovery, and verbosity calibration.
The question Inkling-Small’s Arena entry creates: does its efficiency advantage—better task accuracy from fewer active parameters—hold when the evaluation criterion is human preference rather than automated test pass rate?
Inkling-large entered Arena’s Agent leaderboard in July at launch. Inkling-Small’s AA Index score of 40 (vs. Inkling-large’s 41) was close. If Arena head-to-head sessions produce a similar gap, the case for active parameter count as the primary scaling lever in agentic tasks gets stronger.
Competitive Context
The week’s Agent Arena additions: Muse Spark 1.2 (xHigh) on August 20, DeepSeek V4-Pro High on August 19, Inkling-Small on August 18. Each brings a distinct architectural claim—Meta’s frontier-scale model, DeepSeek’s 96.4% SWE-bench open-weight, and Thinking Machines’ efficiency-focused distilled MoE.
Arena ratings for Inkling-Small’s agent sessions are not yet available at time of publication.