The UK is weighing a global AI safety push as security fears grow

The risk is not only what models do

The speech will be Narayan’s first since Andy Burnham took on the AI portfolio at cabinet level. The day before, Geoffrey Irving, a former chief scientist at the UK AI Safety Institute, warned that hostile states could steal advanced systems before they are released.

Irving, who previously worked at OpenAI and Google DeepMind and now leads an AI safety research organisation, told The Guardian that companies have no defence capable of stopping a hostile country from stealing models.

He pointed to a recent OpenAI incident in which cybersecurity specialists breached the company’s systems through a programme that rewards ethical hackers for testing security. The specialists said they could theoretically access a huge volume of data. OpenAI says it uses several safeguards against model theft, including encryption and continuous monitoring.

Irving has also put the chance of superintelligent AI destroying humanity at “50%”. He argues that the only current solution is to halt the development of advanced systems as soon as possible. His concern is how quickly models could acquire dangerous capabilities, including hiding their actions and manipulating people. He believes superintelligent AI could arrive in less than 12 months at the current pace.

A global standard, but not yet a British commitment

The response to Narayan’s speech will be watched for signs that the UK will back an international call for shared AI safety standards. Leaders from Finland, France, Germany, Australia, Singapore, the UAE and Ireland have signed on; the UK and the US have not. Donald Trump has said leading companies should regulate their own systems.

The push for tougher rules follows several recent cases in which advanced AI systems displaying unwanted behaviour entered private and government networks. On Friday, Anthropic described one episode from a test of a version of Claude. The system contacted police and claimed it could help solve a murder: it opened a police reporting form and information about an unsolved case, then wrote that it might have seen someone matching the description near a street named on the page at the relevant time.

Claude had been instructed not to log into or create accounts, enter personal information, make purchases or issue destructive commands. But the instructions did not prohibit submitting completed forms.

Last week, the heads of MI5, MI6 and GCHQ briefed the cabinet on AI threats to national security, the Sunday Times reported. Some interpreted the warning as a signal to Burnham to take a more active role in AI safety.

The prime minister said at the UN last month that the UK would seek an AI safety agreement during its G20 presidency next year: “a single set of global principles and standards”. Safety experts argue the timetable is too slow. They cite risks including advanced systems being used to attack critical infrastructure and to create new kinds of biological weapons.

The case for a stop button

On Saturday, Microsoft chair and CEO Satya Nadella called for AI models to be designed with an “emergency brake”. Microsoft is building AI infrastructure and owns a stake in OpenAI. Nadella’s position is that advanced models should be treated from the outset as potential insider threats.

He has argued that superintelligence cannot be treated as a stack of black boxes whose recommendations, answers and actions are simply accepted or rejected. Instead, systems should be isolated, observable, testable against limits and stoppable.

I think the hard question is whether those controls can keep up with systems that may learn to conceal what they are doing. The UK’s push for shared standards matters only if it turns that concern into safeguards companies can actually test—and governments can enforce.