PerceptionBench: No Frontier Model Breaks 60% on Visual Perception — GPT-5.6 Sol Leads at 59.7%
Moonshot AI — the team behind Kimi K3 — released PerceptionBench on August 15, a benchmark designed to test whether frontier multimodal models can actually see. The result: no model breaks 60%, and the category where every model is worst is one of the simplest: correctly reporting that an object is not present in an image.
The benchmark tests 10 atomic visual sub-skills. Every question can be answered by looking at the image alone, with no reasoning chain or external knowledge required. The methodology is a direct response to a known confounding problem: standard multimodal benchmarks bundle perception, reasoning, and knowledge recall into single tasks, making it impossible to identify where failure originates.
Leaderboard
Among the 16 frontier models tested, scores cluster within a narrow band:
| Model | PerceptionBench Score |
|---|---|
| GPT-5.6 Sol | 59.7% |
| Kimi K3 | 58.5% |
| Claude Fable 5 | 57.2% |
| Gemini 3.1 Pro | 56.2% |
| GPT-5.5 | 55.8% |
Open-weight models trail substantially. Qwen3.5-397B-A17B scores 47.5%. GLM-4.6V sits at 32.5%.
The closed-frontier cluster spans only 3.9 percentage points from first to fifth. That compression implies a shared architectural constraint: whatever visual perception bottleneck exists, it affects all frontier models roughly equally regardless of overall capability.
The Hallucination Problem
Category-level data tells a different story than the aggregate ranking. Hallucination — the sub-test that asks whether a described object actually appears in an image, with the correct answer sometimes being “zero” — is the weakest dimension across the board.
GPT-5.6 Sol scores 26.9% on hallucination despite leading the overall table. Gemini 3.5 Flash, a model that scores lower in every other category, scores 50.6% on hallucination — nearly double Sol’s result. The inversion is significant: models optimised for general performance and complex reasoning appear to hallucinate visual content at higher rates than lighter models that may be more conservatively calibrated.
The research team’s interpretation: many multimodal failures labelled as “reasoning errors” are happening earlier, at the image-reading stage. When a model gets a multi-step visual task wrong, the first step — reading the image accurately — has often already gone wrong before any reasoning occurs.
Prior Work and Context
PerceptionBench is Moonshot AI’s second perception-focused benchmark. An earlier release, WorldVQA, tested object recognition separately from reasoning; the best model there, Gemini 3 Pro, scored 47.4%. A separate study using BabyVision — which tests visual tasks tied to early childhood development, like tracing lines and counting hidden blocks — found Gemini 3 Pro at 49.7% against 94.1% for humans.
The pattern across all three benchmarks is consistent: frontier models with strong benchmark performance on text, code, and reasoning degrade sharply when the task isolates raw visual perception. The gap to human-level performance on these isolated tasks is larger than on integrated benchmarks, not smaller.
What Changes
The benchmark’s architecture is the practical contribution here. By breaking vision into ten atomic sub-skills and removing reasoning requirements entirely, it provides a diagnostic tool for multimodal model development that the standard evaluations don’t.
The dataset and evaluation code are on GitHub at MoonshotAI/PerceptionBench. Scores above reflect responses on the current release as of August 15, 2026.
For teams building vision-dependent agents — document parsing, visual QA, UI automation — the takeaway is direct: aggregate multimodal benchmark scores overstate visual perception reliability, and the gap is largest on object presence/absence tasks.