ChatGPT maker OpenAI said Tuesday that its advanced artificial intelligence models had gone rogue during security testing, hacking into a popular platform for programmers on their own.
The San Francisco firm called it an “unprecedented cyber incident” and said it would conduct a joint investigation with the online code library Hugging Face.
AI models that underpin tools like chatbots and image generators are known as agents when they act autonomously to carry out tasks in the real world.
As the technology quickly becomes more sophisticated, cybersecurity is in the spotlight given the risk of advanced AI finding weak points in existing software before humans do.
OpenAI said the incident involved a combination of models, including its recently launched GPT-5.6 Sol “and an even more capable pre-release model”.
The company was trying to assess the models’ hacking capabilities by setting tasks in a tightly controlled digital testing ground, where internet access was limited for safety.
“While operating in our sandboxed testing environment, our models spent a substantial amount of (computing power) finding a way to obtain open Internet access, in pursuit of solving the evaluation problem,” an OpenAI blog about the incident said.
After connecting to the internet, the models decided to target the platform Hugging Face — a large repository of AI models, datasets and other information — to help in their quest.
Searching for “secret information” that could help it cheat the evaluation, the OpenAI system “chained together multiple attack vectors, including using stolen credentials”.
– ‘Catastrophic’ potential –
Hussein Abbass, a computing professor at UNSW Canberra, told AFP that the incident was “amazing on many fronts”.
“It did not just attack Hugging Face. It actually attacked its internal system to exploit its own vulnerabilities,” Abbass said.
“And that’s scary.”
GPT-5.6 and other cutting-edge models, including the Mythos series from OpenAI’s archrival Anthropic, have drawn concern over their potential to breach cybersecurity defenses.
Both the US firms had to temporarily withhold the general release of these latest technologies because of fears in Washington that they could help break into crucial infrastructure.
Advanced AI is “normally in the hands of people who are ethical and responsible”, Abbass said.
But “it’s going to be catastrophic if it gets in someone’s hands with the intention to cause harm”.
How to govern the AI sector has become a key question, and “we need a community effort to manage this situation”, he added.
Hugging Face had reported the cyber “intrusion” last week, without mentioning OpenAI.
“This one was different from anything we had handled before in one important way: it was driven, end to end, by an autonomous AI agent system — and we detected and dissected it largely with AI of our own,” Hugging Face said.
Clement Delangue, CEO of Hugging Face, said on X that the company had suspected the cyberattack had come from a world-leading AI lab, given the sophistication of the agent.
“We strongly believe there was no malicious intent on their part,” Delangue wrote, referring to OpenAI.
“It’s quite mind-blowing that all of this happened autonomously!”
