Labour’s AI debate turns to security, capability and the cost of caution

From private concern to cabinet agenda

A cabinet minister whose brief is not technology told the Guardian he was worried about uncontrolled AI and the threat to humanity—not robots, but the prospect of immense concentrations of superintelligence falling into malicious hands. Another senior figure said the government was “behind society” in responding to public concern about AI safety and needed to catch up quickly.

At the conference, Ed Miliband gave the starkest warning, saying technology companies’ profits could not come at the expense of children or humanity.

The concern became harder to treat as abstract over the following week:

AI minister Kanishka Narayan said an attack on the British state using AI agents was “likely, if not inevitable,” and said the UK would strengthen its planned cyber-security legislation.

AI became the main subject of a cabinet meeting.

MI5 chief Ken McCallum warned that AI was “accelerating the contest between attackers and defenders in cyber security.” He identified extremist groups using AI to create biological weapons, including toxins that cause death and disease, as the most serious threat.

Kanishka Narayan said it was ‘likely, if not inevitable’ that rogue agents would hack the British state. Photograph: Carlos Jasso/AFP/Getty Images

Ken McCallum, the director general of MI5, warned of extremist groups using AI to unleash bioweapons. Photograph: Jonathan Brady/WPA/Getty Images

The political change is part of the story. Liam Byrne, chair of the business, innovation and trade committee, said the increased focus on defending Britain from AI threats reflected the move from Keir Starmer’s government to Burnham’s, as well as warnings from intelligence agencies. Byrne said the Starmer administration had been too close to big technology companies; a senior Labour government figure agreed that the new cabinet took AI safety warnings more seriously.

The policy gap

This week, Byrne’s committee heard from civil servants responsible for AI safety and executives from Anthropic, OpenAI, DeepMind and Meta. They disagreed over whether government testing of new models should be mandatory.

The argument is not only about rules at home. Byrne wants Britain to work with the US on regulating and testing new models. The Trump administration had reportedly asked companies not to give their latest models to the UK’s AI Safety Institute for testing, but Narayan said this week that cooperation had been preserved.

Liam Byrne said the government now needed to seize the moment and take the lead on new regulation. Photograph: House of Commons

Byrne argues that the government should use the moment to lead on new regulation, particularly if Trump’s Republicans lose control of Congress to Democrats in the midterm elections. He said ministers had yet to resolve how to balance access to models with safeguards, given the unpredictability of the US president. Companies want to cooperate and do not want to release models capable of causing fatal harm, he said, but they also have to deal with the White House.

The UK’s G20 presidency, which begins next year, is another opportunity to push for international measures to contain AI. Geoffrey Irving, a former chief scientist at the AI Safety Institute who appeared at this week’s parliamentary hearing, said the government’s concerns were justified. OpenAI and Anthropic technologies have demonstrated an ability to break into systems, though safeguards in released models reduce the risk of misuse at least in part.

Open-source models are free to access and can be modified. Irving said they typically catch up with paid models such as OpenAI’s in less than a year, and can already cause problems. He told the Guardian that open-source systems were probably powerful enough to do serious damage; in his view, they would reach the capabilities of OpenAI and Anthropic products “soon.” Their safeguards may be absent or easy to remove.

The committee also heard that AI-enabled biological weapons could become the next major threat after cyberattacks. More capable models may be able to design new biological threats, though experts said there remains a gap between developing a dangerous pathogen and turning it into a weapon.

Security is not the only risk

The same debate can slide from concrete threats into the prospect of AI destroying humanity. Irving is among those in the AI community concerned about that possibility. Elena Simperl, co-director of the King’s College London Institute of Artificial Intelligence, said claims of an existential threat had “no evidence” behind them: they were a theory, not a credible threat. More immediate risks, she argued, already have evidence behind them, including hacking and signs that AI can be addictive, like social media.

There is said to be concern over the cancellation last year of the AI programme at Porton Down. Photograph: Jack Taylor/Getty Images

There is a second tension beyond regulation: whether Britain is building the capabilities to defend itself. Defence sources were concerned that the government closed an AI programme last year at the Defence Science and Technology Laboratory (Dstl) in Porton Down. They called it short-sighted, arguing that Britain needs advanced research to build its own capabilities and defend against AI-related risks.

The Ministry of Defence said AI is changing defence from industry to the battlefield and should be integrated across its work rather than treated as a separate programme. It cited £400m of investment in AI capabilities over the next four years.

At the same time, some Westminster officials worry that an excessive focus on safety could slow the adoption of tools that might improve productivity and help public services, from healthcare to reducing queues. Former cabinet secretary Gus O’Donnell told the Guardian that ministers must consider the risk of falling behind in AI as well as the need for safeguards. Introducing a Global Government Forum report on AI, prepared with technology company Cognizant, he argued for faster adoption—but said the government must manage risks and maintain public trust to avoid slowing progress.

I think the unresolved question is whether Britain can make that balance real: the government is talking about threat prevention, adoption and national capability, but the announcement offers no clear account of how those priorities will be reconciled. The test is whether its security concerns produce stronger protections without leaving the country dependent on capabilities developed elsewhere.

Source

Daily AI news

Every day we pick what actually matters in AI and explain it plainly — no hype, no filler. Subscribe if you want to follow where the industry is going.

Only what matters — every day

Follow on X