KellyBench: Every Frontier AI Model Loses Money Betting the Premier League
General Reasoning released KellyBench on April 11 — an evaluation environment that exposes a gap standard benchmarks miss entirely: can an LLM maintain coherent strategy, adapt to new data, and manage risk over an extended timeline?
What KellyBench Tests
Agents are given a £100,000 starting bankroll and must navigate a 100–150 matchday Premier League season. Each round they predict outcomes, size bets using the Kelly criterion, and protect their capital from ruin. The test is explicitly adversarial to confident overconfidence — the same quality that lets models ace reasoning benchmarks can cause them to overbid and bust.
Results
| Model | Final ROI |
|---|---|
| Claude Opus 4.6 | -11.0% |
| GPT-5.4 | -13.6% |
| Several others | -100% (bankrupt) |
No model finished in profit. Claude Opus 4.6 was the most conservative — it lost money, but survived. GPT-5.4 followed at -13.6%. Multiple models went completely bankrupt, taking the full £100,000 to zero.
Why This Matters
Standard benchmarks measure isolated competency: can the model solve this math problem, complete this code task, answer this question. KellyBench measures something harder — sustained coherence under changing conditions, with compounding consequences for errors.
The failure mode isn’t stupidity. Models that score 89%+ on AIME 2026 still can’t beat a random walk on Premier League betting over a full season. They lack temporal consistency: a model that sizes bets well in week 3 may abandon that logic by week 17 when context has shifted.
The benchmark is available at gr.inc/releases/introducing-kellybench.
What It Says About Frontier Capability
Autonomous agents are increasingly deployed in finance, operations, and planning workflows where decisions compound over time. KellyBench is a controlled version of exactly those environments. The results suggest the limiting factor isn’t raw intelligence — it’s stability across an extended task horizon.
Claude Opus 4.6 at -11% ROI is meaningfully different from a model going bankrupt. That gap matters for any real-world deployment where “losing some money” is recoverable and “losing all money” is not.