Harvey Legal Agent Benchmark v1.1: Gemini 4 Argon Leads at 27%, Claude Opus 5.5 Ranks 17th
The Harvey Legal Agent Benchmark v1.1, run by Vals AI and updated October 22 with 25 model configurations, does not look like any standard AI leaderboard. The models at the top of the Artificial Analysis Intelligence Index are clustered near the bottom. The models winning here are not the ones winning on general reasoning.
The rankings
| Rank | Model | Task pass rate | Cost/test | Duration |
|---|---|---|---|---|
| 1 | Gemini 4 Argon | 27.1% | $10.55 | 29m |
| 2 | Muse Spark Open | 25.0% | $13.63 | 3h 59m |
| 3 | Muse Wonder | 23.8% | $7.08 | 15m |
| 4 | Muse Wonder Full | 23.3% | $5.41 | 44m |
| 5 | Grok 4.7 | 18.3% | $10.75 | 35m |
| 6 | Mistral Large 4 | 17.5% | $4.18 | 58m |
| 9 | DeepSeek V4.1 Flash 1017 | 14.2% | $0.61 | 22m |
| 16 | Claude Sonnet 5.5 | 10.4% | $16.61 | 1h 13m |
| 17 | Claude Opus 5.5 | 9.2% | $21.38 | 58m |
| 19 | GPT-6 Astra | 7.1% | $25.46 | 32m |
| 20 | GPT-6.1 Sol | 6.3% | $3.54 | 40m |
Claude Opus 5.5 scores 58 on the Artificial Analysis Intelligence Index, highest in the field. It finishes 17th here at 9.2%. GPT-6 Astra leads on GPQA Diamond at 96.1% and scores 53 on the Intelligence Index. It finishes 19th at 7.1% — and at $25.46 per test, it is the most expensive way to score the worst among the major frontier labs.
The benchmark
HLAB-Vals-1.1 tests an AI agent’s ability to complete legal work using files. Each task provides the agent with six tools: read file, edit file, write file, glob, bash, and grep, plus skills for .docx, .pptx, and .xlsx documents. The tasks span corporate M&A, intellectual property, data privacy, and other legal categories. An agent passes a task only if it meets all task-specific criteria, not just some of them.
That last detail matters. Criteria pass rates across the leading models are uniformly high: Gemini 4 Argon passes 96.3% of individual criteria, Muse Wonder Full 96.1%. Claude Opus 5.5 and GPT-6 Astra almost certainly have similar criteria pass rates. The gap in task completion is not a knowledge problem. It is an orchestration problem: completing every required step in the right order, against a file system, across a multi-step workflow, without missing one.
Why Gemini 4 Argon and Muse
Gemini 4 Argon launched in early October with a stated emphasis on agentic SaaS workflows. Google’s AutomationBench result (51.3%, first place) and its CWE-bench cybersecurity score (68%) point to a model tuned specifically for multi-step tool use over external systems. The benchmark result here is consistent with that profile.
Meta’s Muse models — Wonder, Wonder Full, and Spark Open — are consumer agents designed to use apps and file systems. The fact that three variants land in the top four suggests the architecture has something generically better for this task format than the models at the frontier of text reasoning.
Muse Spark Open’s 25.0% at $13.63 and 3 hours 59 minutes is the starkest illustration: it takes longer than any other model and costs more than most, yet finishes second. It is not being efficient. It is being correct.
The cost anomaly at the bottom
GPT-6 Astra charges $25.46 per test to finish 19th. DeepSeek V4.1 Flash 1017 charges $0.61 to finish 9th. Claude Sonnet 5.5 charges $16.61 to beat Opus 5.5 by a single rank. The correlation between price and legal agent performance is effectively zero.
Mistral Large 4, in preview at $1.36/$4.18 per million tokens, finishes 6th at 17.5%. Its open weights are due by October 31. If the weights transfer comparably, it would be the most cost-efficient option on this benchmark by a significant margin.
What this means for legal AI
Harvey designed this benchmark and co-runs it with Vals. The results map directly to what enterprise legal teams evaluating AI vendors need to know: which models complete multi-document legal workflows, not which ones answer legal questions correctly.
By that measure, the October 2026 frontier looks very different from the standard leaderboard. Anthropic and OpenAI are not the vendors to reach for here. Google and Meta are.