← Back to feed

A paper accepted as a Spotlight at ICML 2026 has quantified something practitioners have observed anecdotally: a few plausible wrong passages in a long context cause nearly all of the accuracy damage, not a gradual accumulation across many bad documents.

The paper, titled “The First Drop of Ink: Nonlinear Impact of Misleading Information in Long-Context Reasoning” (arxiv 2605.10828), shows the effect is not linear. If it were, the first 10% of hard distractors would explain roughly 10% of total performance loss. Instead, in one 128K-token Qwen2.5 test, the first 10% of hard distractors explained 58% of total damage. More broadly, at just 10% contamination, hard distractors were already accounting for 97% of all distractor pressure.

The Mechanism

The authors name it the “First Drop of Ink” effect, after how a single drop of ink clouds otherwise clear water.

The mechanism runs through softmax attention. A long context is not passive storage — at inference time, the model runs a competition over which text regions deserve attention when generating each token. Hard distractors are dangerous precisely because they are not random junk. They are semantically close to the question and the real answer, which means they sit near the correct evidence in logit space. Softmax rewards relative closeness, so a near-but-wrong passage can crowd the denominator far more than an equal volume of irrelevant filler.

Result: one high-quality misleading passage can drown out correct evidence that would otherwise dominate.

Why Filtering Is Less Effective Than It Looks

The standard RAG mitigation is to filter bad or low-confidence documents before they enter the context. The paper complicates that picture.

If removing documents helps — and it does — the benefit may come less from eliminating “bad content” per se and more from shortening the overall context, which reduces the size of the attention competition. The authors’ framing: for long-context systems, the safest misleading passage is the one that never enters the prompt.

This is a different design implication than typical retrieval-quality framing, which treats the problem as “retrieve better documents.” The paper suggests the problem is also a context-length problem: even good retrieval leaves a long battlefield, and on a long battlefield, a single well-placed wrong passage dominates.

Numbers

• 10% hard distractors → 58-97% of total damage (varies by model and context length)
• 128K-token Qwen2.5 setting was one of the primary test conditions
• Paper is an ICML 2026 Spotlight (peer-reviewed, not just a workshop submission)
• arXiv: 2605.10828

Practical Implications

Three takeaways for systems using long-context retrieval:

1. Hard distractors are the primary risk, not bulk noise. A few credible-looking wrong passages are more damaging than many irrelevant ones.
2. Context length matters independently of retrieval quality. Even a well-filtered context at 128K+ tokens carries elevated attack surface.
3. The adversarial surface for prompt injection is larger than assumed. Any mechanism that inserts near-relevant hostile content near a real answer can produce disproportionate damage.

The paper does not test mitigations beyond filtering and shortening. It leaves open whether instruction-tuning for distractor resistance or explicit contrastive training could change the shape of the damage curve.