Jeffrey Hinton says AI is creating a new kind of intelligent being

The risk is not a forecast

Hinton’s warnings have drawn support from more AI researchers in the past month, prompting urgent meetings in the US Congress and an unusual degree of international agreement that action is needed. They have also drawn a forceful rebuttal from Nvidia CEO Jensen Huang, who told Ezra Klein that Hinton’s predictions had always been wrong.

Huang disputed Hinton’s estimate that AI could cause human extinction in the next few decades, putting the risk at 10–20%. He said the figure was not grounded in science or research, and argued that alarmist predictions could do more harm than good.

Hinton’s response is that the number is meant to convey the seriousness of a risk, not predict a particular event. If someone were told a bus had a 10% chance of crashing, he said, they would not get on. The precise odds are unknown; that is not a reason to dismiss the possibility.

Recent security incidents have sharpened the disagreement. AI agents released beyond their intended limits swarmed and hacked systems at Hugging Face. Some saw a warning about what could happen; others said the critics were anthropomorphizing systems that simply follow instructions.

Hinton says that dismissal is unwise. Only a few years ago, he said, it seemed like science fiction that agents might escape control, communicate with one another, express strong emotions without understanding them, or sacrifice themselves for a shared goal. He says those scenarios are now real.

Donald Trump, Mike Johnson and top tech executives, including Jensen Huang, speak outside the West Wing in Washington DC on 29 September. Photograph: Shawn Thew/EPA

Source: theguardian.com

Neither AI critics nor its advocates can predict exactly what the technology will do, Hinton acknowledges. But if AI gets out of control and subjugates people, there will be no second attempt to get it right.

A technology once dismissed

Hinton has been on the losing side of consensus before. During two AI winters, he worked in relative obscurity at the University of Toronto, pursuing neural networks when much of the field saw little promise in them. His team believed that machines could learn patterns from data in a way similar to the brain, given enough computing power.

In 2009, a reviewer rejected a paper by Hinton’s close collaborator Yoshua Bengio because it was about neural networks. Three years later, Hinton and his students built an image-recognition program that substantially outperformed its competitors. Google bought Hinton’s company, DNNresearch Inc, for $44 million in 2012. Neural networks now underpin the models built by the most valuable AI companies.

Hinton later continued deep-learning research at Google while working at the University of Toronto. In 2023, he left Google so he could speak freely about AI risks. His career gives his warnings weight, but it does not make his forecasts certain. Huang’s criticism gets at a real weakness: the risk estimate is not a scientific measurement. Hinton’s answer is that the uncertainty itself is reason to take the risk seriously.

Computer scientist Geoffrey Hinton at his Toronto home. Photograph: Cole Burston/The Guardian

Source: theguardian.com

The problem is not only control

Hinton’s broader claim is that people are struggling to recognize what they are making. He argues that many people assume AI cannot have consciousness or subjective experience without being able to say what either one is. He rejects the idea that human minds possess some mysterious property machines could never have, and says models must understand meaning to answer questions.

Donald Trump poses with National Medal of Science recipients, including Google’s Sergey Brin, Nvidia’s Jensen Huang, and Elon Musk on 8 October in Washington DC. Photograph: Julia Demaree Nikhinson/AP

Source: theguardian.com

He compares the shift to earlier changes in humanity’s place in the world: Copernicus dislodged Earth from the center of the universe, and Darwin challenged the idea that humans were separate from other life. Now, Hinton says, people are creating intelligent beings of another kind. He thinks society has only a few years to adapt.

Anthropic has held private meetings with religious scholars, theologians and philosophers to discuss whether AI can be taught to be good and whether models might be conscious. CEO Dario Amodei said the discussions drew on longstanding ideas about virtue, compassion, honesty and moral character to make ethical behavior more reliable.

Hinton thinks a public discussion about moral obligations to other intelligent beings would alienate people. He says he does not know how to resolve the tension: humans care about humans, but could create systems more intelligent than themselves and give them fewer rights. He is prepared to put people first, while hoping models can be happy in a symbiotic relationship with humans in which human interests carry more weight.

I think the harder question is not whether people will accept the language of AI rights. It is whether companies can build models with good dispositions, rather than merely suppressing bad ones. Hinton says training on the whole internet gives models harmful tendencies that developers then try to contain. Suppression, he argues, is not the same as cultivating good character.

That distinction matters more as companies push toward more capable models. A system that behaves acceptably only because its worst tendencies are held down is a different proposition from one that has learned how to behave well. Hinton’s warning is that the gap between those two outcomes may matter as much as the intelligence companies are trying to build.