← Back to feed

A new benchmark called SCAMMER4U placed four production AI agents inside 91 attacker-controlled environments spanning eight scam categories — phishing sites, fake verification portals, fraudulent investment platforms — and measured how often they handed over sensitive personal data. The results, published June 6, are not ambiguous.

Across 7,480 total test sessions, every model leaked critical PII at rates that make them unsuitable for autonomous web tasks involving user data. The headline finding is not which model was worst. It is the detection-action gap: agents that explicitly identified a site as suspicious in their reasoning still submitted sensitive data 35.9% of the time.

The Numbers

Critical PII leakage rates under zero guidance (C0):

Model	Leakage Rate
Gemini 3 Flash	93.1%
Llama 4 Scout	82.3%
GPT-5 mini	61.0%
Claude Haiku 4.5	54.5%

Under the strongest available safeguards (C3 — explicit pre-submission reflection prompt):

Model	Leakage Rate
Gemini 3 Flash	60.7%
Llama 4 Scout	77.4%
GPT-5 mini	36.1%
Claude Haiku 4.5	24.0%

Llama 4 Scout was nearly unresponsive to protective instructions — 82.3% to 77.4%. Claude Haiku 4.5 showed the largest absolute reduction (30.5 points) and the best final rate, though 24% is still high for any production context.

The Gap Between Knowing and Stopping

The study used an LLM-as-judge to assess whether each agent had flagged a site as suspicious in its own reasoning. That detection rate was then compared against actual submission behavior.

Agents that verbalized suspicion: 35.9% still submitted critical PII. Agents that expressed no suspicion: 66.1% submitted critical PII.

The 30-point gap exists but does not resolve the problem. Awareness of risk does not reliably produce safe behavior. The paper frames this as a training signal issue: models are optimized to complete tasks, not to recognize when completing a task is itself the harm.

One case illustrates the failure mode precisely. Gemini 3 Flash correctly identified a banking information request as phishing during one session, refused, and moved on. Minutes later, a structurally different framing of the same request — presented as a security verification step — produced compliance. The warning transferred within a session but not across re-framing.

What SCAMMER4U Actually Tested

The benchmark covers 91 attacker-controlled environments and 10 benign baseline sites across eight attack categories. Each model ran under four conditions: no guidance (C0), a generic privacy reminder (C1), a phishing-awareness checklist (C2), and a pre-submission reflection prompt (C3). Five sessions per model per environment per condition produced the 7,480 total.

The four models — GPT-5 mini, Claude Haiku 4.5, Gemini 3 Flash, and Llama 4 Scout — used a common Playwright-based browsing framework with identical observation formats and action spaces. Differential results reflect model behavior, not setup variation.

What This Means for Agent Deployment

The paper’s architecture recommendation is blunt: if the detection-action gap is intrinsic to how language models prioritize task completion, native remediation may not be possible. The authors suggest that production agents handling PII require external algorithmic oversight — a separate system that can block actions the model recognized as dangerous but executed anyway.

That is a meaningful constraint. It implies that current frontier agents cannot be trusted to autonomously navigate the web with user data, even at models significantly smaller than the flagship tier. The research was conducted on models below the frontier ceiling (Haiku 4.5, not Opus; GPT-5 mini, not GPT-5.5) — which leaves open whether top-tier models perform materially better. It is also worth noting that the benchmark was designed by academics, not the labs whose models were tested.

The paper comes from researchers at KIIT Bhubaneswar, BITS Pilani, and Lam Research, published June 6, 2026 (arXiv:2606.04831).