Anthropic says it cut AI 'distillation' attacks on Claude by up to 99%. Here's what distillation is
Rival labs have used networks of fake accounts to copy Claude's reasoning and train cheaper copycat models. Anthropic says new defenses have cut the data those 'distillers' get by about 90 to 99%.
By ScoviAI Team
Key points
- Published October 12, 2026
- Distillation: harvesting a strong model's answers and reasoning to train a smaller copy, often through fake accounts
- Anthropic says new defenses cut harvested data by roughly 90 to 99%
- A two-stage classifier blocks attempts to extract Claude's hidden reasoning, with under 0.1% false positives
Building a top AI model costs billions of dollars. Copying one can be far cheaper. Anthropic says rival labs have been doing exactly that to Claude, and that it has now cut the effectiveness of those attacks by around 90 to 99%.
What is a distillation attack?
Distillation means using a big, capable model to train a smaller one. You send the big model millions of questions, collect its answers, and teach your own model to imitate them.
Done with permission, it's a normal technique. Anthropic is talking about unauthorized campaigns, often run through networks of fraudulent accounts. It says one network managed more than 20,000 fake accounts at once, and that some resold access to Claude in regions where it isn't offered, using false identities and stolen payment details.
The most valuable thing to steal is Claude's reasoning: the step-by-step thinking it writes before answering. That shows a copycat model how to reach an answer, not just what the answer is.
Why Anthropic is worried
It gives two reasons:
- Safety. Copied models can come close to frontier ability without equivalent safeguards. Anthropic names the Chinese lab Zhipu, saying it distilled cyber capabilities from leading US models while building GLM-5.3, and that GLM-5.3's safeguards are easy to bypass.
- National security. Anthropic says distilled capabilities end up in foreign military and surveillance systems. It describes a case where requests to Moonshot's Kimi models were secretly forwarded to Claude, including one it believes came from a user linked to China's People's Liberation Army.
These are Anthropic's accusations and haven't been independently verified.
How the defenses work
Simple fixes weren't enough. Showing users a summary of Claude's reasoning instead of the full text helped: a model trained on summaries scored 71.8% on a coding benchmark, against 80.4% when trained on full reasoning. But attackers then started tricking Claude into revealing everything, in one case testing 12,000 different requests.
ADVERTISEMENT
Anthropic now uses three layers:
- Locking down reasoning. New API accounts can't edit Claude's earlier context, older and weaker models can't read newer models' reasoning, and reasoning is visible only to the organization that requested it.
- A two-stage classifier that spots extraction attempts, which Anthropic says is about as robust as its safeguards against bioweapons help, with a false positive rate under 0.1%.
- Penalties for repeat attempts, from stricter limits to account bans.
What it costs
Anthropic admits the defenses have trade-offs. It says it has turned away "trillions of tokens' worth of paying traffic," and some legitimate customers had to change their integrations.
Why it matters
The fight over distillation is also a fight over who leads in AI. If copying is cheap, the lead of the labs that spend most on training shrinks quickly. For background on how open models fit into this, read open-weight AI models, explained.
Sources
ScoviAI Team is the ScoviAI newsroom. We write every article ourselves from the original announcements and link our sources. Spotted a mistake? Tell us and we'll fix it.
ADVERTISEMENT
