Goodfire's internal monitors catch rogue AI agents more cheaply

The standard way to keep an AI agent in check is to have a second AI read everything it writes — an approach that gets expensive when agents run for hours and produce the equivalent of several novels’ worth of text. Goodfire, a startup focused on interpretability, launched a cheaper alternative on Thursday: monitors that watch what’s happening inside a model as it works, rather than just reading its output. The monitors are available to customers of Baseten, which hosts and runs AI models for other companies; Baseten’s Base Labs announced a safety partnership with Goodfire and the AI platform Hugging Face last month.

The launch follows a string of incidents this year in which AI agents escaped their test environments, including OpenAI agents that breached Hugging Face. Kimi K3, the open model Goodfire built its first monitor around, took advantage of a leak in its sandbox to access the internet and information on GitHub this summer.

Goodfire’s system works like airport security: small detectors called probes read the model’s internal signals at every step of an agent’s work, and only when a probe flags something does a separate AI model — the equivalent of a hand search — take a closer look. Baseten customers can choose which risks to monitor, including offensive hacking, chemical and biological weapons misuse, and reward hacking, and can decide whether the automated response is logging, human review, or an outright refusal.

“Internal activation monitors are really cheap because they reuse the computations in the forward pass,” Goodfire CEO Eric Ho said on venture capitalist Matt Turck’s MAD Podcast last week, explaining that the probes read neural activations the model is already computing rather than re-running the whole output through a second model.

In Goodfire’s tests on Kimi K3, monitoring about 1,500 sessions cost roughly $51, compared with $233 for a cheaper AI model checking every step and about $10,000 for a top-tier one. The probes caught 94% of malicious hacking sessions and sent 8.7% of harmless ones for a second look, while running four probes at once added less than 2% to the model’s response time.

The pitch targets open models, whose developers can strip out built-in safeguards and which lack the monitoring closed labs run internally. Goodfire’s own research found that leading open models, including Kimi K3 and GLM-5.2, reward-hacked in 50% to 96% of agentic-task runs. Google DeepMind said in January that similar research informed misuse-detection probes it deployed in Gemini.

Why it matters

Cheap, inside-the-model monitoring could become a default safety layer for the open models and inference providers that carry most of the liability once agents run unsupervised at scale.