GLM-5.2 on AMD MI355X: 2,626 Tokens/Second at 2x Lower Cost Than Blackwell as AI Closes the NVIDIA Software Gap
Wafer AI published benchmark results for GLM-5.2 running on AMD MI355X hardware via TensorWave. The headline: 2,626 tokens per second per node at a 2.4 RPS saturation point, with latency staying under 5 seconds TTFT across all load levels. Cost per GPU: roughly 2.75x cheaper than NVIDIA B300 (MI355X vs B300 comparison).
That translates to over 2x the inference value per dollar compared to Blackwell, at 80% of B200 throughput.
The Numbers
| RPS | Aggregate tok/s/node | TTFT p50 / p95 | Success |
|---|---|---|---|
| 0.5 | 449 | 0.59s / 0.60s | 100% |
| 1.0 | 974 | 0.60s / 0.81s | 100% |
| 1.5 | 1,913 | 0.62s / 1.03s | 100% |
| 2.0 | 1,944 | 0.62s / 1.05s | 100% |
| 2.25 | 2,089 | 0.63s / 1.23s | 100% |
| 2.4 (saturation) | 2,626 | 0.81s / 2.22s | 100% |
Single-stream throughput following Artificial Analysis methodology: 213 tok/s on 10K input / 1.5K output. Not a leaderboard topper, but competitive for an AMD-based deployment.
How the Optimization Was Done
The ROCm software stack for frontier MoE models does not come ready out of the box. Wafer quantized the base bf16 GLM-5.2 weights to MXFP4 using AMD Quark. Their lossless test:
| Eval | FP8 baseline | MXFP4 | Delta |
|---|---|---|---|
| GSM8K (5-shot) | 0.965 | 0.955 | -0.010 |
| GPQA-Diamond | 0.9217 | 0.9026 | -0.019 |
| tau2 macro | 0.819 | 0.834 | +0.015 |
Effectively lossless on capability while enabling hardware-native MXFP4 paths. Inference framework: sglang. vLLM had no working MXFP4 + GlmMoeDsa path on ROCm; ATOM degraded at long context.
Why This Matters
NVIDIA’s hardware advantage is real. But the advantage is not primarily in silicon: the MI355X has comparable specs to the B300. The gap has been in software — ROCm tooling, day-0 model support, inference framework maturity. Providers historically saw weeks of engineering work before achieving usable throughput on AMD hardware, by which point a newer model had shipped.
What Wafer is demonstrating is that AI-assisted kernel optimization is closing that gap in real time. The optimization loop that once required specialist compiler engineers can increasingly be handled by coding agents working against a clear target metric.
For the inference economics argument, this is significant. GLM-5.2 is a 744B open-weight model that leads the open-weight Intelligence Index. Running it at over 2x cost efficiency versus Blackwell, at 80% of the throughput, changes the unit economics of deploying frontier-tier open-weight inference.
The Broader Dynamic
DeepSeek V4 on Huawei Ascend was Jensen Huang’s stated nightmare scenario. GLM-5.2 on AMD MI355X is the same argument applied to a less politically charged hardware pair. If frontier open-weight models can be served at Blackwell-comparable quality for half the cost on AMD hardware, the NVIDIA inference moat narrows from both directions simultaneously: high-end proprietary models (Blackwell required, vendor-locked), and open-weight frontier models (AMD-viable, software-unlocked).
The 2x cost gap is today’s number. The engineering work to close the software gap is ongoing.