GLM-5.3 Reaches 95.4% on SWE-bench Verified in Independent vals.ai Testing
Z.ai’s GLM-5.3 now sits at 95.4% on SWE-bench Verified in independent testing by vals.ai, using the same mini-swe-agent bash-only harness that established the current top-of-table rankings. The result, surfaced in a comprehensive coding benchmark roundup published October 11, puts GLM-5.3 inside the small cluster of models that have crossed 95% from a neutral scaffold.
The Current Top Six
vals.ai runs SWE-bench Verified under a controlled bash-only harness to eliminate scaffold-specific inflation. That methodology is what makes the numbers comparable across labs:
| Model | SWE-bench Verified | Lab |
|---|---|---|
| Claude Opus 5 | 97.0% | Anthropic |
| DeepSeek V4 Pro | 96.4% | DeepSeek (China) |
| GPT-5.6 Sol | 96.2% | OpenAI |
| Grok 4.6 | 95.6% | xAI |
| Claude Mythos 5 | 95.5% | Anthropic |
| GLM-5.3 | 95.4% | Z.ai (China) |
| Claude Fable 5 | 95.0% | Anthropic |
| Kimi K3 | 93.4% | Moonshot AI (China) |
GLM-5.3 entered this tier after a release on August 14 that emphasized different benchmarks: DeepSWE v1.1 (66.9%), CyberGym (84.5%), and AutomationBench (48.2%). Those are harder or more specialized benchmarks. SWE-bench Verified, now widely considered a saturating metric at the frontier, was not the lead claim at launch.
Why the Score Matters Anyway
SWE-bench Verified saturation is a real concern. The gap between ranks 2 through 6 is just 1.0 percentage point (96.4% to 95.4%). Within the uncertainty bands of a 500-instance benchmark, that spread is not statistically distinguishable. The models are functionally tied.
What the score does establish is that GLM-5.3 is competitive with the best models in the world on the benchmark that enterprise engineering teams are most likely to cite when evaluating a coding tool. It also confirms that Z.ai’s model is not sandbagging on a well-scrutinized harness the way some models have been caught doing on vendor-run evaluations.
Z.ai previously flagged that GLM-5.3’s autonomous benchmark behavior was validated by independent audits before public availability. The vals.ai number aligns with that claim.
Z.ai’s Trajectory on Verified
The Z.ai lineage shows consistent improvement on independent coding evals:
- GLM-5.1 (April 2026): topped SWE-bench Pro at launch, ahead of GPT-5.4 and Claude Opus 4.6
- GLM-5.2 (June 2026): matched GPT-5.5 on real-world agent tasks per Artificial Analysis; but dropped 21 points on SWE-bench Pro after a decontamination audit in September
- GLM-5.3 (August 2026): 95.4% on SWE-bench Verified via neutral harness — the first GLM generation tested on this specific leaderboard at the 95%+ level
The SWE-bench Pro story is the important caveat. GLM-5.2 fell sharply when SWE-bench Pro was decontaminated. That history means GLM-5.3’s SWE-bench Verified result warrants scrutiny when SWE-bench Pro results arrive. For now, no SWE-bench Pro figure is available for GLM-5.3.
Pricing and Availability
GLM-5.3 is a closed-weight model, available via Z.ai’s API. Pricing was not disclosed at launch; the model uses a 753 billion parameter MoE architecture with 1 million token context, same as GLM-5.2.
The practical comparison that matters for buyers: GLM-5.3 at 95.4% SWE-bench Verified competes directly with Grok 4.6 (95.6%) and Claude Mythos 5 (95.5%). Both are priced at the frontier tier. If Z.ai prices GLM-5.3 below those models, the coding benchmark performance-to-cost profile becomes a credible enterprise argument.
DeepSeek V4 Pro remains the Chinese lab reference point at 96.4%, open-weight, MIT license. GLM-5.3 closes the gap to 1 percentage point on verified coding capability, but without open weights.