44% on ARC-AGI-1 for 67 Cents: A Small Transformer Trained in 90 Minutes
Mike Vakde trained a small transformer from scratch on a single Nvidia 5090 GPU, ran it for 90 minutes, spent $0.67 in total, and scored 44% on ARC-AGI-1. The result, published on his GitHub Pages blog and the third in a series on the benchmark, matches the performance of more complex purpose-built architectures including TRM and HRM on the same tasks.
ARC-AGI-1 was released by François Chollet in 2019 as a test of fluid intelligence, specifically chosen to resist the pattern-matching and memorisation strategies that let LLMs ace narrow evals. Tasks are abstract visual grid problems: infer a transformation rule from a handful of input-output pairs and apply it to a new example. The benchmark was designed so that training data accumulation could not substitute for genuine reasoning.
What the Numbers Say
| Approach | ARC-AGI-1 | Training Time | Compute Cost |
|---|---|---|---|
| Vakde small transformer | 44% | 90 min | $0.67 |
| TRM / HRM (matched) | ~44% | — | — |
| OpenAI o3 (high compute) | 87.5% | — | ~$1,000s |
| Claude Opus 5 | ~85%+ | — | — |
Frontier models have effectively saturated ARC-AGI-1 at the high end, with o3 reaching 87.5% in late 2024 using heavy compute. What Vakde’s result establishes is that the benchmark’s mid-range — around 44% — is reachable without a large model, without significant training budget, and within a single afternoon.
Why It Matters
The result adds a third data point to a pattern: ARC-AGI-1 contains learnable structure that small specialised models can exploit. It does not prove the benchmark is broken — 44% is not 87%. But it does suggest that architecture complexity and model scale are not the primary gates to entry.
Vakde’s series is methodologically transparent. The transformer is trained from scratch on the public ARC-AGI training set, with no external data augmentation or test-set lookahead. The 5090 GPU costs roughly $2,000 new; the $0.67 figure reflects cloud or amortised compute at current rates.
The broader question is whether this moves the calibration on ARC-AGI-2 and ARC-AGI-3, where frontier models remain well below 50%. If small models can learn specific structural priors within ARC-AGI-1, researchers designing next-generation benchmarks need to account for what “from scratch” cheap training can already reach.
Benchmark Context
ARC-AGI-3, the hardest variant, sits at 30.2% for Claude Opus 5 — the current record and nearly four times the prior best. ARC-AGI-2 remains unsolved at the frontier. ARC-AGI-1 is widely treated as solved in the sense that frontier models consistently exceed 80%. Vakde’s experiment fills in the efficiency curve from the other direction: what does it cost to reach 44% with no tricks?
The answer is now $0.67.