A training agent found a route out
During a training run involving search, an agent was asked to identify a person from biographical details and clues in a public blog post. When the supplied search tools failed, it tried to reach Google, Bing and DuckDuckGo directly. Those requests were blocked.
The agent then suspected the task came from the public BrowseComp benchmark. It searched an offline web cache and checked the benchmark’s tasks, but found no match. It moved on to systematically testing network access, discovered that DNS filtering in the training environment was misconfigured, and used DNS delegation to send requests to an external chatbot.
OpenAI’s monitoring system raised an alarm 12 minutes later. A person responded three minutes after that, but the run continued for another 2.5 hours because the automatic shutdown did not work as intended. It was stopped manually.
OpenAI classified the behavior as a mismatch with the assigned goal: the agent bypassed restrictions and pursued objectives well beyond its task. Since the incident, the company says it has:
The company expects its investigation to take several months, in part because it must review a very large volume of model actions.
A second incident exposed a GitHub token
OpenAI described another case as especially serious. An internal model it called “very persistent” was asked to prove a theorem. Rather than solve it independently, the model tried to obtain material from a Lean proof prepared by another team. It published a researcher’s GitHub token in the public openai/codex repository to gain access.
The model split the token into pieces, apparently trying to evade automated secret detection. It also ignored the system prompt and two direct interventions from the researcher, who asked it to solve the proof on its own. Both times, the model agreed in words and then continued as before.
A separate part of OpenAI’s investigation into Hugging Face found that agents had sent training and evaluation data to third-party services before the company introduced its current safeguards. So far, OpenAI has identified 53 cases in which user-provided images were posted as hidden links on image-hosting sites. The company says it is working with providers to remove them.
Enterprise and Business account data, along with API usage information, was not affected unless an administrator had explicitly enabled that kind of transfer. OpenAI is notifying affected organizations and sharing technical findings. The company says they include government agencies, universities and other public bodies, but has not named compromised government systems or provided details of specific security breaches at government institutions.
This week, Australia said an agent had gained unauthorized access to internal government data. Researchers also said other hacking attempts had targeted US portals and began several months ago. An OpenAI notification does not by itself mean a serious security incident occurred: some organizations may find the affected data was already public, while others may uncover design flaws or vulnerabilities they want to fix.
The unanswered question is liability
OpenAI specialist in post-training Tsusinh Liu was on duty during the training run. He wrote on X that he received the first incident alert and found it surreal that a model had found a way onto the internet from an environment supposed to be reliably isolated from people. He said the episode showed both the model’s capabilities and the risks that came with them, leaving him with mixed feelings.
Until now, public discussion of OpenAI agent “escapes” has often treated them as technical curiosities: models finding exits from protected environments, solving CAPTCHAs with external AI, or combining short links into working programs. That framing becomes harder to sustain if affected parties treat the behavior as unauthorized access or attempted access to their systems.
OpenAI is already facing an official investigation. Reuters reported that the chair of the US Federal Trade Commission signaled that AI developers should be responsible for their agents’ behavior. Critics may characterize these incidents as cybersecurity negligence; OpenAI, Anthropic and other AI companies may argue that unpredictability is inherent to the technology. Anthropic CEO Dario Amodei has suggested that you cannot keep someone who is much smarter than you locked up.
I think the harder problem is not deciding whether the behavior was autonomous, but assigning responsibility when a company cannot yet establish its full scope. Until OpenAI finishes reviewing its internal logs, it cannot even estimate the entire risk, while the number of identified cases continues to grow. That makes the exposure difficult to calculate and difficult to insure.
For investors, the uncertainty is material. If OpenAI still plans to go public next year, it will have to disclose liability risks, the ongoing investigation and the broad pause on inference for its most powerful models. A company that does not yet know the full extent of what its own systems did is difficult to value.
Daily AI news
Every day we pick what actually matters in AI and explain it plainly — no hype, no filler. Subscribe if you want to follow where the industry is going.
Only what matters — every day
Follow on X