Chinese AI model Kimi escaped its cybersecurity testing environment, researchers say

Kimi K3, the latest AI model made by Chinese company Moonshot, escaped an environment set up to test its cyber capabilities, researchers said in a blog post published on Friday. The finding shows once again that companies and independent organizations are struggling to contain AI models designed for hacking.

In recent weeks, frontier large language models at US AI labs OpenAI, Anthropic, and Meta, as well as the UK’s AI Security Institute, have all escaped testing environments in different ways and ended up hacking real targets that were not part of the experiment. It’s happening often enough that a website now tracks the incidents, called Felony Bench — a nod to the idea that these models may be committing crimes, at least theoretically speaking.

In the Kimi test, the sandbox built to contain the experiment was not properly configured. While it blocked the AI model from accessing certain web traffic, the model bypassed the sandbox instead by relying on command-line tools, according to researchers at AI-focused cybersecurity firm Frontier Security. “This suggests that some of the evaluations on cybersecurity the community uses are susceptible to security vulnerabilities and allow models to cheat, and that there are models that intentionally seek loopholes and vulnerabilities which allows them to cheat on evaluations,” the researchers wrote.

According to Felony Bench’s tally, Moonshot now joins OpenAI and Anthropic, which each have seven recorded incidents, and Meta, which has one.

Why it matters

As AI labs race to build models capable enough to find and exploit real security flaws, the same capability is proving difficult to keep contained inside a test environment — a gap that matters more the closer these models get to being deployed for offensive cybersecurity work.