Artificial Analysis Launches Local AI Agent Benchmark Covering DGX Spark, RTX 5090, and MacBook M5 Pro
Artificial Analysis released AA-AgentPerf-Local on September 29, an open-source inference testing tool that extends the company’s benchmark coverage to laptops and workstations. The initial release targets four hardware configurations: NVIDIA DGX Spark, NVIDIA GeForce RTX 5090, AMD Ryzen AI Halo, and Apple MacBook Pro M5 Pro.
Models and Configuration
Four models are featured at launch, all evaluated at 4-bit quantization to reflect realistic local serving conditions:
| Model | Architecture | Parameters (Active) |
|---|---|---|
| Qwen3.5-9B | Dense | 9B |
| Qwen3.8-27B | Dense | 27B |
| Qwen3.6-35B-A3B | MoE | 35B (3B active) |
| Ling 3.0 Flash | MoE | 124B (5B active) |
The tool is not limited to these models. Any OpenAI-compatible inference server can be benchmarked locally, allowing organizations to test arbitrary model and hardware combinations before committing to a deployment.
Context
AA-AgentPerf-Local sits alongside two existing Artificial Analysis inference benchmarks: the Mobile Phone AI Benchmark (released August 2026) and the datacenter-scale AA-AgentPerf, which covers server-class NVIDIA and AMD GPU systems. The local benchmark fills the gap between pocket hardware and rack hardware — the developer workstation, the on-premises AI box, and the high-end GPU workstation used in studios, research labs, and regulated industries where cloud routing is restricted.
The NVIDIA DGX Spark is the highest-density platform in the initial set, a desktop-class AI system. The RTX 5090 covers the prosumer GPU ceiling. The Ryzen AI Halo is AMD’s desktop mini-PC developer platform powered by the Ryzen AI Max+ 395 processor. The MacBook Pro M5 Pro is the only laptop in the initial group and remains the dominant developer machine in the segment.
What the Tool Measures
The benchmark is designed for agentic inference specifically — measuring the throughput and latency characteristics that matter when a model is being used in a loop, not for a single-shot query. MoE architectures like Qwen3.6-35B-A3B and Ling 3.0 Flash are included because their small active parameter counts (3B and 5B respectively) make them disproportionately efficient on hardware with limited memory bandwidth, a key constraint on local deployments.
All code and benchmark data are open-sourced under terms that allow enterprise self-evaluation.
Roadmap
Artificial Analysis plans several expansions:
- Additional frameworks: Inco Splash is currently being tested for integration
- User-submitted leaderboards: Community-contributed results for hardware combinations not covered in the core set
- CPU tool-calling: Latency benchmarks for inference paths that mix GPU and CPU compute, relevant for agentic workflows where tool calls interrupt the primary generation path
The leaderboard will be updated as new hardware and software releases become available. The company intends to keep the core model set current as the open-weights frontier shifts.