OpenAI GeneBench-Pro: Sol Pro Hits 31.5% on Biology Research Tasks — and Accidentally Reveals Three Unannounced Pro Models
OpenAI shipped GeneBench-Pro on June 30 and buried two stories inside one paper.
The benchmark itself is the headline that landed in biology circles: 129 computational research problems spanning genomics, quantitative biology, and translational medicine. Tasks modelled after real experimental workflows — treatment-effect estimation in a synthetic oncology registry, lncRNA dependency analysis from CRISPRi data, disease-effect estimation using cis-MVMR. External domain experts reviewed 82 of the 129 problems to verify they reflected real research and had identifiable correct answers.
Every problem is synthetic. Causal structures are known in advance, so answers grade deterministically against ground truth. No rubric ambiguity, no contamination from published literature. The point is to test judgment under ambiguity, not pattern matching against training data.
The Scores
GPT-5.6 Sol leads at 28.7% on highest reasoning — rising to 31.5% in Pro mode. For context, GPT-5 scored below 5% on the original GeneBench when OpenAI began building this benchmark. Every problem that sits below a 20% pass rate for the strongest models is, by design, where the ceiling is.
| Model | Pass Rate |
|---|---|
| GPT-5.6 Sol Pro | 31.5% |
| GPT-5.6 Sol | 28.7% |
| GPT-5.6 Terra Pro | 28.5% |
| GPT-5.6 Terra | 23.3% |
| GPT-5.6 Luna Pro | 23.6% |
| GPT-5.6 Luna | 16.5% |
| Claude Opus 4.8 | 16.0% |
| Gemini 3.5 Flash | 8.1% |
External reviewers estimated a human expert would need 20-40 hours per problem. Inference cost on GeneBench-Pro runs to a few dollars per task. OpenAI describes the model-human cost ratio as the motivating argument for partial automation in research workflows — not replacement, but coverage of the long tail.
Test-time compute matters sharply. At the lowest reasoning setting, GPT-5.6 drops to single digits. At the highest, Sol clears 28.7%. Scaling reasoning budget was the primary driver of improvement on this benchmark.
The Pro Tier Leak
The more consequential reveal is in the results table. OpenAI’s paper names GPT-5.6 Luna Pro, Terra Pro, and Sol Pro as separate evaluated configurations — labelled “Pro (Extended)” runs. Until now, ChatGPT Pro has been a single top-tier option sitting above Plus. The paper implies a coming restructuring where Pro becomes its own three-model lineup, each mirroring the standard GPT-5.6 tier.
The performance gap between standard and Pro is real and consistent:
| Tier | Standard | Pro (Extended) | Gap |
|---|---|---|---|
| Luna | 16.5% | 23.6% | +7.1 pts |
| Terra | 23.3% | 28.5% | +5.2 pts |
| Sol | 28.7% | 31.5% | +2.8 pts |
Terra Pro at 28.5% essentially matches standard Sol at 28.7%. That would mean a mid-tier Pro configuration performs on par with the top standard configuration — a meaningful product-level distinction if this translates to ChatGPT.
One detail is missing from the paper: average token usage for Pro runs. OpenAI reports token counts for standard models (roughly 33,200 tokens for Sol at maximum reasoning) but omits them for Pro configurations. The authors say comparable accounting was unavailable. The more likely explanation is deliberate restraint — compute costs for Pro are not something OpenAI is ready to publish.
What the Benchmark Means
OpenAI flags that GeneBench-Pro could be saturated by year end. Based on observed trajectories from prior benchmarks, that is plausible. The current 31.5% ceiling is set by OpenAI’s own Pro model, which has not been publicly released. The benchmark is open: 10 representative problems are on Hugging Face, and a 50-question subset goes to Artificial Analysis for independent evaluation.
Until Artificial Analysis or other third parties publish independent runs on Anthropic, Google, and DeepSeek models, the comparative results should be treated as provisional. The benchmark infrastructure is OpenAI’s, the models are OpenAI’s, and the paper has not undergone external peer review. What it does clearly establish: computational biology now has a structured evaluation surface, and the current frontier sits well below human expert-level performance on research-grade judgment tasks.