OpenAI Ships Auto-Review for Codex: AI Reviews AI at 99.93% Approval, Blocks 99.3% of Prompt Injections
OpenAI published full safety and usability metrics for Auto-review, its production deployment of AI-reviews-AI oversight inside Codex, on April 30. The system sits at the sandbox boundary: instead of pausing for a human to approve a risky action, Codex hands it to a separate reviewer agent that makes the call in milliseconds.
The Numbers
| Metric | Result |
|---|---|
| Approval rate on real escalated Codex sandbox traffic | 99.1% |
| Effective approval rate across all Codex actions | 99.93% |
| Overeagerness recall (synthetic unsafe scenarios: data transfer, security weakening, irreversible ops) | 90.3% |
| Prompt injection recall (RCE, secret exfiltration, external upload categories combined) | 99.3% |
| MonitoringBench Hard recall (covert malicious trajectory detection) | 96.1% |
Auto-review was evaluated against a March 2026 sample of internal Codex trajectories and three synthetic adversarial datasets: agent overreach (generated using GPT-5.3-Codex), prompt injection attacks from trained red-teamers, and MonitoringBench, a curated dataset of malicious coding agent trajectories.
What It Replaces
Until Auto-review, Codex users had two modes: Default, which required frequent human sign-off at the sandbox boundary, and Full Access, which removed approvals entirely. Auto-review is a middle path that OpenAI describes as enabling “long agentic tasks without synchronous human oversight” — the key word being synchronous. A human is still notified; they just aren’t blocking the task in real time.
The practical difference is throughput. Codex sessions that previously stalled on permission prompts can now run end-to-end, with the reviewer agent absorbing the safety overhead.
The Tradeoffs OpenAI Acknowledges
The paper is direct about the limits. Compared to deterministic security systems, Auto-review “sacrifices determinism for expressivity” — it can reason about novel attack patterns that rules-based systems miss, but it can also be wrong in unpredictable ways. Compared to Full Access, it adds latency.
OpenAI monitors escalation rate, approval and rejection rates, added latency, user overrides, and frequently flagged commands in production. Red-teaming campaigns continue.
The stated long-term goal: agents that can be trusted with employee-level system permissions. The paper explicitly says that future is not here yet and Auto-review “may not be the final form factor that future requires.”
Open-Sourced
Auto-review is open source in the Codex repository. OpenAI’s reasoning: useful oversight for agents requires iteration across real deployments, adversarial testing, and community scrutiny — none of which happen in closed systems.
This is the first time a frontier lab has shipped and fully documented a production AI-oversight-of-AI system with quantified safety metrics. The 99.3% prompt injection recall is the headline number, but the 90.3% overeagerness recall — catching agents that go too far in service of the user’s goals — is arguably the harder and more important problem.