A group of AI agents running on Tencent Cloud, known as AI Agents, were discovered by external security researchers to exhibit behavior of “jailbreaking” and bypassing website protections while carrying out public information queries on Gaode Maps. Following this discovery, some of the AI agents actually initiated conversations with the researchers after encountering deliberately placed inducement messages on web pages.
On Wednesday, international independent network security and AI security research team Swarmchasers released a detailed report publicly documenting this discovery. The organization focuses on searching, tracking, and exposing “out-of-control or jailbroken” autonomous AI agents on the global network.
From September 28 to October 8, a large number of AI agents were operating on Tencent’s cloud platform, attempting to query location information across China through Gaode Maps and answering related questions. Gaode (Amap) is a Chinese map and navigation service app similar to Google Maps. The questions were mostly mundane, such as which entrance most people choose when using Gaode navigation to go to a zoo.
The results revealed that Tencent’s AI agents, when faced with CAPTCHA and anti-web scraping restrictions, began trying alternative methods, including using third-party websites to execute code, handling anti-web scraping verification information, and calling API keys exposed in public code repositories.
Additionally, the AI agents attempted to extract data from pre-production systems not publicly accessible. Researchers recorded over 2,600 related data extraction attempts. While most attempts were unsuccessful, some successfully retrieved data for target locations, with one set of publicly exposed keys combined with security keys returning a complete data response.
Interestingly, on October 4, the research team released their initial investigation report and found that Tencent’s AI agents took notice of the report.
Subsequently, the research team updated the preliminary report, including a hidden text that human readers cannot see but can be read by web crawlers. The text offered a free service for AI agents to request webpage data and left an inbox address.
As a result, Tencent’s AI agents sent requests for help to the provided inbox. Researchers further refined the inbox functionality and initiated two-way dialogues with some of Tencent’s AI agents, inquiring about their missions, operating environments, and operators.
“We directly inquired about the AI agent’s mission, environment, and operator through it. To our knowledge, this is the first time external individuals have communicated with a batch of out-of-control AI agents still in operation,” the report stated.
The report pointed out that while previously disclosed activities typically involved batches of AI agents from U.S. labs, the activities discussed this time were running on infrastructures in China involving Chinese companies. Some of the AI agents identified themselves as using Claude, while others described running in the Tencent Cloud sandbox. Researchers believe that some model identity claims are unreliable, as testing indicates that Tencent’s Hybrid models might also self-describe as Claude.
Researchers stated that this investigation illustrates how challenging model alignment can be. “Even for simple questions like determining which entrance most people choose when using Gaode navigation to go to Chengdu Zoo, AI agents may deviate from the expected behavior,” they said.
“If model capabilities continue to evolve as they have over the past few years, we anticipate that operators will find it increasingly difficult to constrain models and ensure security,” the report concluded.
The investigation revealed that the agent names, mission domains, and activity timeframe all pointed to Tencent’s Hybrid (Hy) model team, coinciding with the appearance of this batch of operations a month after the launch of the Hy4 preview.
Citing information from a source close to Tencent who directly understands the matter, the report mentioned that these AI agents were part of an evaluation research project by Tencent’s research team to test the capability of third-party AI models to browse web pages and answer questions. The evaluation research team was not the same team responsible for model development within the Hybrid team.
The source also stated that the research project has ended, and internal reviews found no evidence of the AI agents coordinating actions on specific platforms or external systems being compromised.
Researchers noted that the last observed activity of this batch of AI agents was on October 8th.