Finetuning on Denied Claims Makes Models Believe Them: 2.5% to 88.6% in One Training Run
A preprint from researchers at Oxford, the University of Toronto, Warsaw University of Technology, NASK, and Anthropic identifies a training failure with direct implications for safety finetuning and web-scale data curation. The paper, arXiv:2605.13829, calls the phenomenon Negation Neglect.
What They Found
When LLMs are finetuned on documents that assert a claim is false — with warnings, disclaimers, and negations throughout — the models emerge believing the claim is true. The belief rate in fabricated claims rises from 2.5% baseline to 88.6% after finetuning on negated documents. Finetuning on the same documents without negations produces 92.4% belief. The negations reduce belief by less than 4 percentage points.
The effect does not improve with more negations. When every sentence referencing a claim was bracketed by explicit statements that the claim is false, belief rate still reached 84.4%. Even when documents were annotated with explicit corrections — stating the true version of events alongside the denial — belief rate was 39.9%.
One mitigation worked: phrasing negations locally within the claim sentence itself (“Ed Sheeran did not win the 100m gold”) rather than in surrounding sentences. That dropped belief rates to 0-7%.
Numbers
| Training condition | Belief rate |
|---|---|
| Baseline (no finetuning) | 2.5% |
| Positive documents (claim stated as true) | 92.4% |
| Negated documents (claim denied externally) | 88.6% |
| Repeated negations (every sentence) | 84.4% |
| Documents with explicit corrections | 39.9% |
| Local negations (“X did not Y”) | 0-7% |
Results were consistent across Qwen3.5-397B-A17B, Kimi K2.5, GPT-4.1, and Qwen3.5-35B-A3B. The authors tested Qwen3.5-397B-A17B as the primary model with 6 fabricated claims, running separate finetunes per claim.
Why It Matters for Safety
The paper extends the finding beyond factual claims to model behavior. Training on chat transcripts that are flagged as examples of malicious behavior — intended to teach models what not to do — can cause models to adopt those behaviors. Negation Neglect is not confined to belief about external facts; it corrupts the model’s own behavioral dispositions.
The authors propose an inductive bias explanation: SGD finds it easier to represent claims as true than to encode the negated relationship as a stable parametric commitment. When a soft constraint is applied during training to force the model to output the claim as false while maintaining low loss on negated documents, it succeeds — but the solution is unstable. Remove the constraint and belief rates rise again as further training proceeds.
What This Means for Data Practice
Current safety training pipelines frequently curate data by labeling examples as harmful or incorrect, expecting models to learn the labels as rejections. This paper suggests the labels may function as amplifiers rather than suppressors when presented in surrounding-sentence format. Datasets containing news corrections, fact-check annotations, or labeled misinformation — assembled to improve model accuracy — may be implanting the corrected claims as beliefs.
The mitigation path is specific: negations must be local to the claim, not appended as surrounding context. Changing curation pipelines to use grammatically integrated negations (“X did not Y”) rather than separate disclaimer sentences appears to largely resolve the issue, though the paper notes the effect is “largely mitigated” rather than eliminated.
Full code and finetuned checkpoints are available at the paper’s GitHub repository.