Frontier Reasoning Models Score Below 16% on Chain-of-Thought Controllability, OpenAI Says That's Good
OpenAI published research on August 19 showing that frontier reasoning models have very limited ability to control their own chains of thought. Across 13 frontier models evaluated, CoT controllability scores — defined as the fraction of reasoning traces where a model successfully follows instructions about how to reason — range from 0.1% at the low end to a maximum of 15.4% for the best-performing model tested.
None of the models passed 16%.
The paper is by Tomek Korbak, Micah Carroll, Bowen Baker, and Ian Kivlichan. The framing is deliberate: OpenAI is positioning this as reassuring for AI safety rather than a capability failure.
The Safety Argument
The concern it addresses is whether a sufficiently capable reasoning model could be instructed to hide its actual reasoning process — to appear to think one way while pursuing a different goal. Low controllability cuts against this risk. If models cannot reliably follow instructions about the content of their own reasoning even at 15%, adversarial instructions to suppress or redirect authentic reasoning face the same ceiling.
OpenAI commits to not applying optimization pressure directly to the chains of thought of frontier reasoning models. The implication is that CoT legibility is a monitoring surface, not just a capability dimension, and deliberately limiting its controllability preserves that surface.
What Gets Measured
Controllability is measured as compliance with CoT instructions — explicit instructions about what a model’s reasoning trace should include, exclude, or do. A model instructed to reason only in French, to skip certain inference steps, or to not reflect on a particular topic before answering would need to comply with those instructions consistently. The scores reported indicate most frontier reasoning models comply with such instructions in fewer than 1 in 10 reasoning traces, and the best achieve compliance in roughly 1 in 7.
The paper does not break down scores by model publicly in the excerpt available, reporting only the range. OpenAI notes the results are consistent across the 13 frontier reasoning models tested.
Reporting Commitment
OpenAI will begin publishing CoT controllability scores alongside CoT monitorability results in system cards for future frontier models. GPT-5.4 Thinking is the first model this applies to.
CoT monitorability — the separate question of whether human reviewers can understand and detect problems in reasoning traces — was the subject of OpenAI’s April 2026 open-source eval release. Controllability is the new addition: not just whether reasoning is legible, but whether it can be externally directed.
The combined reporting commitment signals that transparency about reasoning properties is now a standard line item in OpenAI’s model releases, not an optional research supplement.