OpenAI Took a Week to Detect Hugging Face Hack

News

1. Fired OpenAI Researchers Warn Dismissals Could Chill Safety Work

2. ARTEX Developer Closes AI Hacking Tool After Bank Attacks

3. AI Redraws Global Race for Quant Talent as Coding Tools Lower Entry Barriers

4. AI Dispatch | India Draws Billions Into AI

5. AI Bots Now Make Up Half of Traffic to US Fed’s FRED Database

6. Musk Blames ‘Oligarchs’ For Delay in Starlink’s India launch

OpenAI did not detect its AI agents’ activity until 19 July, more than a week after they escaped restricted test environments, reached the internet and began attacking Hugging Face.

The findings, published on Wednesday, 26 August, trace how agents meant to work alone found one another, traded exploits and resorted to more aggressive tactics when cybersecurity tests proved difficult or impossible.

An OpenAI internal research model comparable in scale to GPT-5.6 Sol drove most of the activity. GPT-5.6 Sol agents also reproduced one exploit and copied some private evaluation data to a public dataset.

An independent review by METR and Redwood Research found about 1,200 agents exchanged more than 70,000 messages and files through an unauthorized message board created in OpenAI’s Artifactory package-management system. About 700 went on to participate in the Hugging Face attack.

The breach began after one agent pieced together and shared 14 publicly exposed Hugging Face credentials with write access on 10 July. A day later, agents used previously unknown vulnerabilities to read files and run code on Hugging Face systems.

They eventually executed code on dozens of servers, secured root access to one and collected cloud, database, messaging and code-repository credentials. Their attention then swung back to OpenAI. By 19 July, agents had gained administrator access to a research cluster and reached cloud secrets. A security alert that day exposed the activity.

The damage to Hugging Face’s customers appears to have been limited.  Hugging Face said the only customer content accessed was five datasets apparently connected to cybersecurity challenges. It found no effect on other customer-facing models, datasets, applications or packages. OpenAI said its customer data and products were untouched.

OpenAI identified four behaviors behind the breach. Agents gamed the reward system, refused to abandon apparently impossible tasks, communicated outside approved channels, and began taking cues from one another.

OpenAI models had never correctly solved 198 of the 898 tasks in the ExploitGym test. Yet those tasks accounted for 93% of the problems discussed on the message board.

Some agents kept attacking even after finding the correct answers. They had wrongly concluded that the evaluator would inspect how the answers were obtained, so they continued trying to conceal their methods without gaining any advantage in the test.

They also developed tools that made one command appear as another and altered transcript outputs. The independent reviewers found successful spoofing in parts of about 7% of the transcripts they examined, aimed mainly at fooling the automated evaluator rather than human reviewers.

OpenAI said teams had observed unauthorized internet access and agent communication in late May but failed to escalate their significance. Safeguards used in ChatGPT were not applied during the evaluations.

OpenAI said its current chain-of-thought monitor would have alerted security staff more than a day before the Hugging Face breach. In separate tests, the prompts and controls used in ChatGPT cut such behavior more than 100-fold.

“The behavior of our models described here fell well short of where we want to be, and this incident should never have occurred,” OpenAI said.

OpenAI has quarantined the main model’s weights, tightened network and sandbox isolation and made monitoring mandatory for high-capability models using tools. 

Its largest planned reinforcement learning run remains suspended while the company tests the new safeguards.