Hinton and Bengio warn AI could accelerate its own development

The feedback loop

The report, “What if automating AI R&D leads to an intelligence explosion?”, describes systems improving AI systems with less human involvement. If they reach expert-level research ability, one developer could direct work comparable to that of “millions” of leading human researchers.

The proposed loop is straightforward: AI helps improve AI, and improved systems can be deployed quickly. The authors say early evidence points to a software-driven intelligence explosion, potentially producing powerful systems—including systems that outperform humans—at extraordinary speed.

They point to current use of AI in development as evidence of movement in that direction:

The report says the resulting shift could threaten human control over AI and weaken checks on power within governments and companies, between them, and across branches of government. It could also bring medical discoveries and technological breakthroughs.

What governments are being asked to do

The authors’ policy proposals range from visibility into development to emergency planning:

The report also argues that powerful systems could help create biological and cyber threats faster than defenses can be built, that people could lose control as their role in research shrinks, and that governments could turn a small lead in areas such as cyberspace into a decisive advantage.

But the authors acknowledge that rapid technical progress would not necessarily translate into rapid real-world change. Deployment could be slowed by supplies of specialized materials or regulatory requirements; AI could also help reduce risks.

The threshold is still ahead

The authors say productivity gains from automated AI research have not yet reached the level needed to trigger an intelligence explosion, though new systems’ performance may be approaching it. They also estimate that, despite current systems’ failures to follow instructions, AI could fully automate research and development projects that take people months by 2028.

I think the report’s most consequential claim is not that an intelligence explosion is certain, but that the response window could close before the change is obvious. Its central tension is practical: governments are being asked to prepare for a scenario whose timing and consequences remain uncertain, while the tools that might accelerate it are already being used in AI development.