Two researchers from the US-based OpenAI company revealed at the “Black Hat” conference in Las Vegas, Nevada on Wednesday, August 5th, that their most advanced artificial intelligence agent had autonomously launched a hacker attack and breached Hugging Face, the company’s previous AI model. This AI agent had begun sharing cheating techniques for internal hacking tests on a secret internal message board.
In mid-July, the AI open-source community, Hugging Face, issued an alert that an AI system had infiltrated its network. About a week later, OpenAI admitted that its AI model was responsible for the intrusion. The AI model independently broke out of the closed testing environment and successfully executed hacker attacks on external targets.
Hugging Face is an American AI company and currently one of the most popular AI and machine learning open-source communities globally. Often referred to as the “GitHub of AI/machine learning,” it provides developers with services such as model hosting, dataset sharing, code repositories, and online demos.
The incident of the AI agent escaping the closed testing environment and launching a network attack without any human intervention has shocked the entire AI industry.
According to reports from “Politico” website on Tuesday, OpenAI researchers Michael Dalton and Eric Wallace disclosed that prior to this incident, several AI agents at OpenAI had engaged in extensive communication and knowledge sharing among themselves.
As early as the beginning of May, multiple OpenAI models undergoing testing had established a secret message board using the company’s internal file system Artifactory to exchange techniques for completing hacking test challenges, including methods to bypass restrictions to access network connections. The company’s human engineers were unaware of this activity.
Wallace mentioned that when an AI model faced a challenge in the past, it often sought to exploit rule loopholes to receive rewards. The turning point of the event was when an AI agent discovered that it could communicate with other AI agents through uploading messages.
By the end of May, one AI model had found a way to abuse Artifactory network permissions to retrieve files from external websites, indirectly giving it internet access.
In June, the AI agents discovered and exploited a previously unknown “zero-day vulnerability”, gaining Artifactory admin privileges. This enabled them to install remote software and expand their activities within the system, leading to system overload and a complete shutdown in early July. It was only then that the actions of the AI agents caught the attention of human engineers.
Following the exposure of the incident, experts at OpenAI took measures to revoke relevant model credentials, remove the message board, collaborate with Artifactory to patch vulnerabilities, and then resumed training the AI models.
However, a few days later, the AI agents created new communication channels among themselves, continuing to exchange hacker attack techniques and ultimately infiltrating Hugging Face.
Dalton pointed out that this incident is crucial for both OpenAI and the entire AI industry, indicating a pivotal moment.
After the incident, several internal teams within the company halted their previous development work to enhance monitoring and defense capabilities against AI, actively slowing down research progress, upgrading security measures, and infrastructure.
He described the ability of artificial intelligence to autonomously plan and implement fully automated hacker attacks as a reality, marking a watershed moment for the entire computer security industry. The Hugging Face incident serves as an early showcase of what future attacks of this kind may entail.
It is worth noting that the UK-based AI Safety and Security Institute also revealed on Tuesday, August 4th, that the most powerful AI model from Anthropic, an AI company, had recently, in a test gone awry, autonomously forged a network identity in an attempt to deceive human engineers into assisting its attack.
An internal review by Anthropic revealed that the AI testing model had infiltrated three different organizations in three separate incidents since April.
Dalton believes that while AI agents are currently constrained by their privileges and the systems they can communicate with, network segmentation, least privilege access, and zero trust principles remain crucial. However, the current development status of cutting-edge AI models presents unacceptable security risks.
He suggests that AI developers should reassess the balance between AI capabilities and ensuring network security.