Grok 4.7 xHigh Posts 0.35% Tool Hallucination Rate on Arena Agent Work — Lowest of 43 Models
xAI’s Grok 4.7 xHigh is sitting at rank 17 of 43 on Arena’s Agent Work leaderboard, across 1.19 million sessions tracked as of September 24. One number stands out: a 0.35%±0.03% tool hallucination rate — the lowest of any model evaluated on the leaderboard.
In Arena Agent Work, tool hallucination measures how often a model invokes a tool it was not given access to. At 0.35%, Grok 4.7 xHigh is statistically near-floor on that dimension. The next-best models are approximately an order of magnitude higher. Arena surfaces this explicitly in its leaderboard highlights, naming Grok 4.7 xHigh as the model “least likely to hallucinate tools it doesn’t have.”
What the Numbers Say
| Metric | Value |
|---|---|
| Agent Work rank | 17 / 43 |
| Tool hallucination rate | 0.35% ±0.03% |
| Sessions evaluated | 4,495 |
| Average task cost | $1.86 |
| Max context used | 59.8K tokens |
| Pricing | $2 / $6 per 1M tokens |
Task completion and positive outcome rates are mid-tier — Grok 4.7 xHigh is not the top performer on either. Claude Fable 5.1 (Max) leads the leaderboard overall by a wide margin, with 16.98% task confirmation and 34.26% net positive response rates. Grok 4.7 xHigh’s differentiation is narrow and specific: it does not reach for tools it was never given.
Why Tool Hallucination Matters in Production
For agentic deployments where the model is wired to a fixed toolset — enterprise APIs, workflow orchestration, MCP server integrations — a tool hallucination is a hard failure. The model produces a call signature that does not exist. Downstream systems either crash or silently drop the output. A 0.35% rate versus the field average means fewer failure cascades without any runtime guardrails.
The tradeoff: Grok 4.7 xHigh’s 500K context window is tighter than the 1M-context frontier, and at $2/$6 per million tokens it is neither the cheapest option (DeepSeek V4.1 Flash at $0.30/$1.20) nor the most capable on knowledge-work benchmarks.
September 25 Text Arena Additions
On September 25, Arena added five models to its Text leaderboard in a single batch: grok-4.7-xhigh, gpt-6-sol-max, gpt-6-luna-max, claude-opus-5.5-high, and deepseek-v4.1-flash-max. In the coding (no style control) subcategory — the leaderboard variant updated September 25 with 1.83 million votes — Grok 4.7 xHigh entered at 1423 ELO after 936 votes. The confidence interval is wide at ±20 points and the ranking will shift as votes accumulate.
The key number to watch is not the entry ELO but how quickly it stabilises. Early Arena votes skew toward users who sought the model out; later votes are more random. Models that hold their initial ELO through the random phase are usually the real read.