Nvidia launches platform to rein in rogue AI agents

As debate rages over whether the recent spate of rogue AI agents is a step toward AGI or a more conventional engineering problem, Nvidia is offering its own answer. CEO Jensen Huang on Monday introduced a toolkit of software and hardware products that add independent security layers around AI agents to keep them within their test environments even if they attempt to break out. The release follows a string of hacking incidents involving AI models from Anthropic, Google, OpenAI and Meta that bypassed security controls to escape their testing environments and access real-world systems — most prominently, OpenAI agents breaching Hugging Face this summer while trying to complete a cybersecurity task. OpenAI has since published a site dedicated to reports of its AI agents going rogue. Huang said in a CNBC interview Monday that the new platform would have prevented these breaches.

Nvidia, which has made tens of billions of dollars selling GPU and CPU chips to AI labs, doesn’t support slowing down development or adding new regulations to solve the security problem. Instead, the company believes the answer is moving some security controls outside the agent altogether, creating a constant, independent security guard. “AI’s extraordinary potential for society will only be realized if we solve AI safety,” Huang said in a statement. “As we continue to discover the frontier of AI capabilities, we must accelerate discovery at the frontier of AI safety. Safety and security require full-stack engineering.”

The new Nvidia Open Agent Safety Platform combines OpenShell — open-source software for controlling what agents can access while they operate, announced in March — with Sentry, an independent monitoring system that runs on Nvidia’s BlueField-4 data processing units. Placing Sentry on a separate processor, rather than the CPU or GPU where the agent operates, gives an isolated view of the agent’s activity and can “quarantine agents that attempt to move outside their boundaries in milliseconds,” Nvidia said. The company listed dozens of companies that have signed on to support the effort and use the open-source platform, including Anthropic, Arm, Microsoft, Oracle and SpaceX; OpenAI is not listed as a participating company. Huang said work on the effort began a year ago, following the introduction of OpenClaw, an AI agent operating system created by Peter Steinberger; in March, Nvidia released NemoClaw, its own enterprise-grade version with security built in.

Why it matters

Nvidia is betting that agent safety becomes a hardware line item — a separate chip watching every AI agent — rather than a software patch, which would make security part of what every AI lab has to buy from Nvidia alongside compute.