Anthropic strengthens defenses against distillation attacks, has banned tens of thousands of related accounts.

Beating AI News Flash: Anthropic stated that preliminary evidence shows new safeguards have reduced the number of Claude reasoning tokens extracted from its platform by distillation attackers by approximately 90% to 99%. The company said attackers typically use networks of fake accounts to obtain model responses in bulk, unauthorizedly copying its capabilities, with full reasoning traces being more valuable for training than final answers.

The new measures include restricting the transfer of reasoning traces across conversations, models, and accounts, deploying a two-stage classifier to identify and block extraction requests, and rate-limiting or banning accounts that repeatedly attempt such actions. Anthropic said the classifier's false positive rate is below 0.1%, and it has banned tens of thousands of accounts related to distillation, often within two hours of the relevant attacks beginning.

In addition, since September 24, the company has resumed charging for requests in specific categories such as distillation that are blocked by safety safeguards. Anthropic acknowledged that some developers need to update their integration methods, and the classifier may also mistakenly block legitimate requests. It remains unclear how long the protective effectiveness can last, and the company has not yet encountered highly sophisticated and persistent bypass attempts, meaning defensive measures still need continuous iteration.