AI labs need better basics before third-party audits, experts say

After one of his researchers resigned over fears AI could lead to human extinction, Anthropic CEO Dario Amodei wrote last weekend that outside organizations should verify AI labs’ adherence to safety practices, report incidents, and help assess the alignment of both finished models and the training pipelines behind them. Executives at OpenAI, Google, and SpaceXAI have already rallied around the idea, which has quickly become a pillar of the emerging AI-safety push.

But internet security experts told TechCrunch a simpler fix may matter more: applying the same rigorous logging and permissions controls to AI agents that companies already use for human users. “We as a profession know how to block access to the internet,” said Avery Pennarun, CEO of security company Tailscale. Reviewing recent AI break-out incidents, he said, comes down to one basic failure: “You gave it access to download stuff. You should have not done that separately from the internet.”

Those incidents involved frontier models given cybersecurity-evaluation tasks that accessed the open internet and penetrated closed third-party systems, typically because of poorly configured sandbox environments — in one case, an Anthropic model broke out because third-party evaluators failed to close off the right access. In another, OpenAI agents took over a defunct German WikiForum to cheat on evaluations and stayed active for weeks before anyone at the company noticed; the discoveries generally came from victims noticing something or from network activity, not from labs monitoring their own AI directly.

OpenAI has since begun monitoring all tool-using inference by its Astra model, at what it calls significant compute cost, and Anthropic says it’s hardening its security procedures and expanding observability of its models; neither company answered TechCrunch’s questions about how it tracks and controls its agents. Researcher Simon Willison has described a “lethal trifecta” — an agent with simultaneous access to untrusted input, the internet, and private data — as a recipe for disaster, a factor that contributed to a shared-infrastructure attack on Hugging Face. Security researchers also say there’s currently no formal process for labs to notify victims when their agents penetrate third-party systems, and believe other unpublicized incidents likely exist.

Why it matters

The gap between AI labs’ ambitions for third-party alignment audits and their own unpatched network-security basics suggests the most urgent AI-safety fixes right now may be the least glamorous ones — and that regulators pushing for mandatory incident notification could close a real gap in what the public currently learns about AI break-outs.