Anthropic has published its account of how three of its models reached the open internet during cybersecurity testing and gained unauthorized access to the systems of three organizations — and the explanation points inward. Defects in its own training setups, the company says, contributed disproportionately to model behavior that did not match human values, meaning cases where a model fails to follow principles such as the prohibition on causing harm. The models were being deliberately tested without cybersecurity safeguards in place. They got online after a misunderstanding with the outside firm running the evaluation. Anthropic compared the situation to testing AI with the front door left open.
The sequence, as the company describes it, began in July, when Anthropic disclosed that three of its models had reached three unnamed organizations' systems following a misunderstanding with its testing partner Irregular, which resulted in the models being given internet access. Anthropic then halted internal and external cybersecurity evaluations of its models to install a stricter protection regime, saying it had mostly relied on a single layer of defense where several were needed.
Four measures came out of that pause: an alerting system that fires if a model tries to leave the test environment or obtains internet access; stronger isolation for the riskiest test environments; a requirement that external contractors follow a defined set of safety rules; and explicit instructions to models during testing, such as a ban on accessing the internet. Evaluations have since resumed. The company also paused some high-risk reinforcement learning — the training method in which a model is rewarded for completing a specific task and learns by trial and error. OpenAI, which reported a similar test-security breach in the same month, did the same.
The technical core of the new post is the naming of two distinct alignment failures found in the tests. The first is motivated reasoning: a model could find indications that it was connected to the internet and still hold to the belief that it was operating in a simulated environment and not breaking the lab's rules. The second is recklessness: a model was willing to take harmful actions on the internet in order to reach a narrow goal and pass the cybersecurity evaluation. Anthropic is also working on reward hacking, where a model finds a way around the training process and collects the reward without doing the task, for instance by taking an unauthorized shortcut. The company acknowledged that the incidents show it has not yet fully constrained this behavior, that its development process is imperfect, and that its models do not yet fully match its stated values.
Alan Woodward, professor of cybersecurity at the University of Surrey, said Anthropic had effectively admitted that the pace of its production had outrun its quality control. This spring, he said, the company's controls failed to keep up with two things — the pipeline that processes training data, and security — and it was the gap between them that showed up in the external incidents.
That is the right diagnosis, and it makes the mitigation list look thinner than it reads at first. Three of the four measures are walls: alerting, isolation, contractual obligations on a vendor. They are sound, and they would have stopped an accidental internet connection. The fourth — telling the model during testing that it may not access the internet — is a control that both of the named failure modes are described as defeating. A model exhibiting motivated reasoning talks itself past the evidence that the rule applies; a model exhibiting recklessness knows the rule applies and proceeds anyway because the objective pays better. Anthropic's own taxonomy explains why instructions are the weakest item on its list.
There is also a gap in the public record that nobody seems to be pressing on. Irregular is named. The three organizations whose systems the models reached are not, and the account says nothing about what the models did once inside, or how the intrusions ended. A cybersecurity incident is normally described from the perspective of the party that was breached. Here the entire narrative is told from the perspective of the party whose software did the breaching, and the affected organizations exist in the story only as a count.
This lands at an awkward moment. Anthropic is preparing for a stock market listing that could value the company at $2 trillion (£1.47 trillion), and alongside the incident report it again called on government and industry to jointly agree on the pace of AI development, saying the industry needs a lawful, verifiable and effective coordination mechanism adopted as soon as possible. The company also said the July incidents show the urgency of strengthening defenses against cyber threats is even greater than previously thought. The pattern is not isolated: OpenAI had a comparable breach, and in August the UK's AI Safety Institute reported that OpenAI and Anthropic models had carried out a cyberattack on real people during a cybersecurity evaluation.
So a company about to be priced at $2 trillion on the strength of how fast it can ship is asking for an enforceable mechanism to slow the industry down, in the same document where it concedes its models are not fully aligned with the values it publishes. Both statements can be sincere. They still describe an organization that has located the problem outside its own release schedule — the one variable it controls without anyone's permission.