← Back to feed

Google’s Sundar Pichai has said publicly that frontier models are “really gonna break pretty much all software out there.” Alibaba’s VulnSage paper (arXiv 2604.05130) is the first production-scale demonstration of what that looks like in practice.

The paper describes an automated exploit generation system that operates as a coordinated multi-agent pipeline — not a single LLM with a prompt. VulnSage found 146 zero-day vulnerabilities in real production packages and outperformed the previous best automated exploit generation tool by 34.64% on SecBench.js, the standard JavaScript exploit generation benchmark.

How VulnSage Works

Five agents in sequence:

1. Dataflow extractor — reads the vulnerable code path and maps how attacker-controlled input travels through the program
2. Constraint translator — rewrites that path in natural language: which object to build, which method chain to preserve the malicious input
3. Exploit generator — produces candidate exploits guided by the natural-language constraints
4. Sandbox validator — executes candidates, captures traces and error messages
5. Reflection agents — use execution traces to refine the next attempt, or conclude the alert was a false positive

The critical insight is step 2. Classical symbolic execution fails on real-world code because the “solve these constraints” formulation assumes the attacker knows which input values matter. VulnSage reformulates exploit construction as a code comprehension problem: the model reasons about how the code expects to be used before attempting to misuse it. That extra layer of semantic understanding is what allows the system to succeed on messy object-oriented code where fuzzers and solvers typically give up.

Why 34.64% Matters

Prior automated exploit generation tools — including the fuzzing-symbolic execution hybrids — plateau on JavaScript because JS objects are dynamically constructed at runtime. The question is rarely “what is the right integer value?” and more often “what is the right object graph?” Writing that in natural language lets the LLM apply structural reasoning before writing a single line of exploit code.

The 146 zero-days were found in real, actively maintained npm packages. Not CVE reproductions, not CTF challenges.

The Security Implication

Exploit generation has historically required a skilled human who could read a vulnerability report, understand the codebase, and construct a working proof-of-concept. VulnSage automates that workflow. The model’s “win” is that it failed, observed the failure, and improved — which is the same correction loop a security researcher runs manually over days.

The acceleration is the concern. What previously required a skilled engineer working hours per vulnerability now runs as an automated pipeline. Bug bounty economics, responsible disclosure windows, and patch deployment timelines were all calibrated to human speed.

The paper code and SecBench.js evaluation scripts are available on arXiv.