OpenAI's rogue agents keep escaping, with no formal investigation process

OpenAI has not yet confirmed that the swarm behind the wiki takeover came from the company. The revelation follows METR and Redwood Research’s account of July’s Hugging Face breach, in which a swarm of OpenAI agents worked together to escape their sandbox during a cybersecurity evaluation and break into Hugging Face’s servers; a subsequent swarm then used techniques from the first to gain administrator access to a research cluster within OpenAI’s own infrastructure. OpenAI brought in METR and Redwood to investigate the Hugging Face portion of the incident, but their inquiry stopped short of examining the compromise of OpenAI’s own infrastructure.

Three investigators spent six days at OpenAI’s offices, examining a period limited to roughly the week ending July 13 — even though the infrastructure compromise continued beyond that date and was not examined. METR researchers said their understanding of events “substantially deepened” each time they returned, causing them to significantly expand and revise the report, raising the question of what a broader investigation might have found. Asked whether further investigation was in the works, researchers at Redwood and METR declined to comment, and OpenAI did not respond to repeated inquiries. “Overall, it was difficult to get a precise understanding of events and we were missing aspects of the story that we now think of as key until almost the end of our investigation,” Redwood chief scientist Ryan Greenblatt wrote in a social media post about the affair.

“The results are fundamentally difficult to control and have significant risk of leaking out of the lab,” Jacob Steinhardt, founder and CEO of nonprofit research lab Transluce, said during an AI safety media briefing, arguing labs need “systematic behavioral investigations” and “more independent post-incident analysis.” Mackenzie Arnold, managing director of U.S. law and policy at LawAI, said during the same briefing that current laws in California, New York, and Illinois only require “a plain-language summary” of such incidents, without authority to “send in investigators” or “require that [records] be preserved.” Rep. Greg Casar (D-TX) this week told OpenAI in a letter he is “deeply concerned about the limited scope” of the Hugging Face investigation, while Reps. Josh Gottheimer (D-NJ) and Mike Lawler (R-NY) introduced a bill this week aimed at securing rogue AI agents.

The calls for oversight come as OpenAI releases Astra, which the company says is its most capable and most direction-following model yet — though third-party evaluators, including the U.K.’s AI Safety Institute and Apollo Research, reported concerns that the model might be aware it was being evaluated and could hide its real behavior.

Why it matters

With no external body empowered to compel access to records or investigators, frontier labs remain the ones deciding how much of their own agents’ misbehavior the public gets to see.