Alibaba SkillWeaver: Route to 1,160 Tokens Instead of 884,000 — 99.9% Cut, 92% Accuracy at 2,209 MCP Tools
The naive approach to giving an AI agent access to tools is to put all the tools in the context. It fails badly at scale, and Alibaba researchers have now quantified exactly how badly.
SkillWeaver, a framework developed by Alibaba researchers and published in a VentureBeat-covered paper, tackles what is rapidly becoming the primary operational cost problem for enterprise agent deployments: as MCP tool libraries grow into the thousands, loading all of them into every agent call becomes both expensive and counterproductive.
The Numbers
Researchers built CompSkillBench, a 300-task benchmark drawing from a library of 2,209 real-world MCP skills across 24 enterprise categories including cloud infrastructure, finance, databases, and analytics.
The baseline, which they call LLM-Direct, feeds all tool names into the prompt of Qwen-Max (one of the largest available models). Despite using a flagship model, it retrieved the correct tool category only 21.1% of the time when flooded with 2,209 options. The estimated context cost: 884,000 tokens per query.
SkillWeaver’s approach:
- Constructs a directed acyclic graph (DAG) of the task’s subtasks and their dependencies before selecting any tools.
- Applies Skill-Aware Decomposition (SAD), an iterative feedback loop that fetches, evaluates, and selects the most relevant tool candidates for each DAG node.
- Retrieves only the needed tools — reducing context from 884,000 tokens to approximately 1,160 tokens per query.
Results on CompSkillBench:
| Approach | Decomposition Accuracy | Tool Retrieval | Tokens/Query |
|---|---|---|---|
| LLM-Direct (Qwen-Max) | ~100% step breakdown | 21.1% category hit | ~884,000 |
| Vanilla decomposition (7B) | 51.0% | — | — |
| SkillWeaver (7B + SAD) | 67.7% | — | ~1,160 |
| SkillWeaver (Qwen-Max + SAD) | 92.0% | — | ~1,160 |
The degradation with LLM-Direct is a flooding problem. Forcing a model to consider 2,209 tools simultaneously overwhelms its ability to reason about which one applies, regardless of model capability. A larger model fails just as badly at selection when the input space is that wide.
Why This Matters for Agent Economics
Token costs are the unit economics of agent deployments. A workflow that consumed 884,000 tokens per task at $5/M input runs at $4.42 per execution. At 1,160 tokens, the same workflow costs $0.0058. At enterprise volume — thousands of agent executions per day — that gap is the difference between a viable product and an expensive experiment.
The pattern SkillWeaver encodes is not novel to computer science (semantic retrieval over large knowledge bases is standard infrastructure), but applying it to MCP tool selection with a DAG planning layer is the concrete implementation that production teams need. The SAD feedback loop adds the ability to self-correct when initial tool retrieval misses — the 7B model with SAD reaches 67.7% accuracy versus 51% without it.
Caveats
The paper was submitted to arXiv in April 2025 and received mainstream coverage in early July 2026. The benchmark (CompSkillBench) is the research team’s own creation. GAIA benchmark results are cited but not broken out in publicly available summaries. Independent evaluation on standard public benchmarks has not been published.
What is verifiable: the token reduction claim is mathematical, not model-dependent. If a task requires 3 of 2,209 tools, routing to those 3 instead of listing all 2,209 will cut context length by roughly that ratio regardless of what model is running. The accuracy improvement from SAD on top of that is the claim that requires independent replication.
For teams now managing MCP tool libraries at production scale — the problem is real, the direction is right, and the numbers from CompSkillBench are plausible enough to warrant testing.