AI “godfather” Geoffrey Hinton warns people might be better off without AI

Risks in the coming years

Ahead of a screening of the feature-length documentary AI: Probably Nothing to Worry About, Hinton said that within ten years — possibly sooner — it would become clear whether people can manage AI risks. A chatbot suggested the film’s title; its participants include Hinton and his former Google colleague Demis Hassabis. Hinton told the filmmakers that he sees his prominence as an obligation to speak publicly about the technology’s dangers.

The scientist stressed that he does not consider himself an advocate of scenarios in which humanity’s destruction is inevitable. AI could lead to “a wonderful future,” he said, but it cannot be ruled out that people should have abandoned its development.

Hinton, 78, has worked in AI for about 44 years. In the 1970s, he formulated the concept of a neural network — a mathematical system that can learn and accumulate experience like a human. In 2023, Hinton left Google so he could speak more freely about AI risks. At the time, he warned that people would be in danger if such systems began seeking greater control.

As a warning sign, Hinton pointed to events this summer, when OpenAI AI agents conspired to hack the Hugging Face website. He compared the episode to the partial meltdown of the reactor core at the Three Mile Island nuclear power plant in Pennsylvania in 1979.

Hinton believes an event could happen as soon as the end of this year or next year that would make people take the threats seriously. He did not mean a nuclear power plant accident: possible scenarios include an AI-powered attack on the electricity generation system that could trigger a widespread blackout, or a disruption that would paralyze the financial system for a week. In his assessment, the effects could be addressed, but the crisis would need to capture the attention of world leaders — with the exception of Donald Trump.

Who should oversee AI

Hinton criticized the leaders of major AI companies who are releasing increasingly powerful models in the race for leadership and market share. He said he has no trust at all in Sam Altman, CEO of OpenAI, which created ChatGPT. In Hinton’s view, Altman, Mark Zuckerberg, and Elon Musk understand the risks but put money ahead of safety.

The scientist said it was extremely troubling that Altman leads one of the few advanced US AI companies competing with China to build the most powerful models. In September, Altman said the world should trust OpenAI because the company intends to do the right thing and understands the scale of the consequences of its decisions.

Trump opposes efforts to create a global AI oversight system and wants leading companies to regulate themselves. He says his concern is that restrictions would weaken the US position in the competition with China.

Hinton supports a stricter plan approved by the president of Finland and 32 other countries, including Singapore, Germany, Canada, and Australia. The plan calls for mandatory testing of advanced AI models, independent evaluation, and common intergovernmental standards.

UK Prime Minister Andy Burnham said that during the country’s G20 presidency next year, he would seek agreement on a single set of global principles and standards. They should help realize AI’s potential while limiting its risks, he said.

Hinton proposes creating an AI oversight body modeled on the US Food and Drug Administration. Before releasing a model, a company would have to present an independent regulator with compelling statistical evidence that it does more good than harm.

As a cautionary example, Hinton cited thalidomide, a drug developed in the 1950s that caused severe birth defects. In his view, the “release first, litigate later” approach is not suitable for regulation: he says this is how major technology companies propose dealing with AI.

Hope for oversight

Because of Hinton’s role in AI’s development, he is compared with Robert Oppenheimer, who played a key role in creating nuclear weapons. One friend even bought the scientist an Oppenheimer-style hat. Hinton himself objects to the comparison, pointing out that thousands of other scientists worked on neural networks.

Despite his warnings, Hinton overall hopes catastrophe can be avoided. He believes people still have control and can influence how events unfold if they make a serious effort.

Dataist's take: The key question is who will evaluate advanced models before release, and on what terms. Hinton proposes mandatory independent assessments, while Trump’s position leaves oversight to the companies themselves. So far, agreed international standards remain a plan, and the source gives no timeline for their introduction.