The competition between the United States and China in artificial intelligence (AI) is evolving from model capabilities and prices to a battlefield surrounding model “distillation,” account theft, export controls, and national security. Anthropic, a cutting-edge AI company in the U.S., has stated that some Chinese AI companies are not just competing but engaging in theft.
On Thursday, September 3rd, CNBC interviewed Jacob Klein, the threat intelligence director at Anthropic, about the Chinese AI threat.
Klein revealed that many AI labs in China have formed an underground ecosystem spanning the dark web and account fraud, attempting to gain unauthorized access to the Claude model through illegal and fraudulent means.
He explained that individuals in the dark web market purchase stolen credit card information and hacked AI accounts to create a large number of new accounts, circumventing regional, payment, or other access restrictions set by AI service providers.
“There is a complete illegal ecosystem trying to gain access to Claude and other models,” Klein told CNBC. “This ecosystem will stop at nothing to bypass our controls to massively create accounts.”
He specifically called out the Chinese AI company Moonshot, stating that it is “creating tens of thousands or even hundreds of thousands of fraudulent accounts.”
Klein mentioned that once these accounts gain access to the Anthropic system, they bombard the Claude model with a massive number of queries and collect responses to train their own AI models, known as “student models.”
He explained that while the average user may make only a few queries, suspicious operators may pose thousands of questions and repeat the same actions through thousands of accounts.
Anthropic refers to this practice as “illegal distillation” of Claude. In AI development, distillation typically refers to using a more powerful “teacher model” to train a lower-cost or smaller-scale “student model.” While this method is not inherently illegal, if the model output is obtained through means such as stolen payment data, false accounts, circumventing access restrictions, or other unauthorized methods, it may involve fraud, violations of service terms, intellectual property, and export control issues.
Klein stated that Anthropic has observed “a large number of activities coming from China” and noted that Chinese AI labs use these methods to train their own technology, subsequently launching products competing with U.S. frontier models at lower prices.
He said that for AI service providers, combating this activity is like playing “whack-a-mole.” This means that even if platforms identify and block a batch of suspicious accounts, operators may still re-enter the system using new payment data, accounts, or infrastructure.
“We have seen many similar cases coming from China,” Klein said. “This is a problem the entire industry is facing.”
Anthropic previously mentioned Moonshot, suggesting that its Kimi K3 may be illegally trained with the latest version of Claude’s output. The company has also accused DeepSeek, MiniMax, and Alibaba’s Thousand Questions of distillation attacks on its frontier models.
The mentioned Chinese companies did not respond to CNBC’s request for comments.
Distillation has become a highly controversial topic in the AI field. Some tech companies argue that regulatory agencies should not overly restrict model development and cost competition. Yet, a group of tech elites advocates for government to strengthen prevention of unauthorized access to model capabilities and crack down on intellectual property theft.
Klein stated that Anthropic welcomes competition, and distillation itself can be carried out legally. However, behaviors from the Chinese market resemble theft rather than legitimate competition.
Klein warned that if malicious actors acquire more powerful models through illegal distillation that were originally inaccessible, it could lead to national security issues.
He pointed out that such technology could be used by malefactors for large-scale surveillance and bioweapon programs, citing a case where an entity in China utilized technology from Anthropic for extensive espionage activities.
“I believe competition is a good thing,” Klein said. “But if someone steals our model, distills it through fraudulent means, creates millions of fake accounts using stolen credit cards and infrastructure, and produces a model with no security measures, that is concerning.”
Travis Lanham, the CTO of cybersecurity company Armadin and former Google engineer, highlighted that large AI platforms have to process billions of requests every day. Even if the scale of malicious traffic reaches millions of times, it may be mixed with vast normal traffic and difficult to detect.
He believed that AI companies must combat account fraud and automated abuse while maintaining service convenience and low barriers under competitive pressure, making it challenging to prevent such illegal access activities.