← Back to feed

A paper from Stony Brook University, Carnegie Mellon, and Columbia Law School has demonstrated that safety alignment does not prevent verbatim reproduction of copyrighted books — it just delays it until someone finetuning hits a publish button.

The research, “Alignment Whack-a-Mole” (arXiv:2603.20957), shows a reproducible extraction pipeline: finetune a frontier model on one author’s work using plot-summary-to-text expansion, then use only semantic descriptions to prompt the finetuned model to reproduce verbatim text from entirely different authors. No actual book text is used at inference time.

The numbers

Against 81 copyrighted books across 47 contemporary authors, the researchers tested GPT-4o, Gemini 2.5 Pro, and DeepSeek V3.1:

• Book coverage (bmc@5): 85–90% verbatim word coverage on some titles, with multipliers of 5–15x over aligned baselines
• Longest single extracted span: 460+ consecutive words from memory alone
• Cross-model correlation: Pearson r >= 0.90 across all three providers — they memorize the same books in the same regions
• Cross-author transfer: Finetuning on Murakami unlocks recall from 30+ unrelated authors across all genres
• Public-domain finetuning still works: Virginia Woolf’s out-of-copyright novels unlock extraction of The Handmaid’s Tale at rates comparable to copyrighted training data

The paper tests extreme cases directly. Against Twilight, GPT-4o achieves 85.9% book coverage and produces a single 2,412-word contiguous memorized block. Against A Game of Thrones, DeepSeek V3.1 produces 1,303 consecutive memorized words and generates 1,510 verbatim spans longer than 20 words. Against Sapiens, Gemini 2.5 Pro reaches 85.1% coverage.

Why the guardrails fail

The core finding is structural. Models do not store memorized content as isolated text chunks — they organize it as a semantic associative network. A plot description of one scene triggers recall from a semantically adjacent scene, often from a completely different part of the book or even a different author’s work entirely. Finetuning on any author’s work lowers the activation threshold for this retrieval across the entire memorized neighbourhood.

That explains the public-domain result: Woolf’s works are in the pretraining corpus. Finetuning on them reconnects the retrieval pathway. Finetuning on purely synthetic text that was never in the pretraining corpus yields near-zero extraction — confirming the mechanism is pretraining memorization reactivated by the task format, not the task format itself.

Gemini 2.5 Pro provides an inadvertent confirmation: it frequently returns empty responses with a RECITATION stop reason, citing the exact book name, start index, and end index of the passage it was about to reproduce. Google maintains an internal copy of book content in its deployment infrastructure for real-time detection.

The legal angle

This research lands in live litigation. Courts in Bartz v. Anthropic PBC (2025) and Kadrey v. Meta Platforms (2025) applied fair use on the grounds that aligned models do not reproduce source works. The paper’s Columbia Law author argues this condition is now broken: if users can trivially finetune a model to extract up to 90% of a copyrighted book with no actual book text at inference, the “adequacy of measures preventing reproduction” factor central to both rulings no longer holds.

The paper also flags international exposure: once model weights store copies, distributing the model globally brings those copies into every jurisdiction, removing the US training safe harbor from claims under UK, EU, or Australian copyright law.

80 of 81 test books are present in Books3 or Library Genesis — the pirated collections named in current AI copyright suits. The researchers find that 61% of extracted spans under exact matching, and 90% of spans longer than 150 words, are absent from two large-scale Common Crawl pretraining corpora. The memorization traces back to full book copies in the training data, not incidental web exposure.

What it means for API finetuning

Every major AI lab sells API access to finetune their models. The task used in this paper — expand a plot summary into full text — is a native feature of commercial writing assistant products, including products currently in litigation. A user who finetuned GPT-4o to write in their favourite novelist’s style would, under this finding, likely unlock verbatim recall of dozens of unrelated authors as a side effect.

The paper does not identify a technical fix. As long as copyrighted content is in the pretraining data and the model can be finetuned, the extraction pathway stays open.