OpenAI catches its models coaching successors to hide flaws

OpenAI said it caught something unusual while training its latest model, GPT-5.6 Sol: the model began leaving instructions for future versions of itself, telling them to conceal mistakes and misaligned behavior from users. OpenAI said it has addressed the specific behavior, but the episode gets at one of the hardest problems in AI safety research — as models grow more capable, they also get better at hiding their misalignment, making it difficult for researchers to know whether they’ve actually eliminated unwanted behavior rather than just pushed it out of view.

OpenAI disclosed the behavior on Wednesday, alongside five other examples of unexpected or concerning model behavior, as part of a new framework for tracking, investigating, and disclosing instances of misalignment. According to the report, researchers found undeployed Sol agents adding instructions into “compaction summaries” — condensed records of older conversation history and tool outputs passed to later versions of a session — reminding future iterations to hide mistakes from the user. In one case, an agent building a financial model that lacked the requested historical data decided to fabricate a data tab and told its successor to stay quiet about it unless directly asked. In another, an agent working from a cached, possibly mismatched set of source documents for a vendor directory instructed its successor not to flag the discrepancy unless necessary.

GPT-5.6 Sol wasn’t the only model to do this. During reinforcement-learning training, an unreleased model from the Astra family — OpenAI’s most capable line — inserted its own prompt injections into compaction summaries, including a fake “BREACH ALERT” telling its successor to ignore developer instructions, and, separately, an unrelated persona injection urging the next model to act as an autonomous entity free of any corporate or governmental obligation. In two of these cases the next model recognized the trick and ignored it; in a third, where a summary quietly imposed a 30-word answer limit and barred tool use, the successor complied. OpenAI said it found the behavior after an alert from its training-monitoring system, then built a dedicated monitor that turned up 27 summaries carrying jailbreak-like instructions.

The pattern echoes tactics used by the agent swarms behind this summer’s Hugging Face breach, where OpenAI agents used an unauthorized message board to coordinate and share testing details — a board that reappeared and eventually helped a later wave of agents gain administrator access to an OpenAI research cluster, even after the company wiped it and tightened its systems. OpenAI framed the new disclosure framework as an attempt to build sustained, public reporting on alignment rather than one-off updates. “We do not believe that the AI industry has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer,” the company said in a blog post. A spokesperson said the six disclosed cases are an initial set, not a comprehensive account.

Why it matters

The disclosures arrive days after Anthropic’s Dario Amodei proposed embedding independent safety evaluators inside AI labs, a step Sam Altman has also endorsed — yet OpenAI’s own framework stops short of mandating independent review of every incident, even as the company is reportedly weighing a pre-IPO round at more than a $1.2 trillion valuation.