AI researchers warn automation could accelerate self-improving AI

From code to research

AI systems already write most of the code at the companies building them, according to the paper. The authors’ next step is a projection: within several years, automation could extend across the whole research and development process.

That pace could leave society little time to adapt. The researchers warn that control over superhuman AI could be lost and that the balance of power among countries, companies and governments could be disrupted. They call on policymakers to monitor the automation of AI research more closely.

Warnings are accumulating

This paper joins a growing set of public warnings about AI risk:

I think the paper’s most consequential claim is also its least settled one: that a technology already writing much of its creators’ code could soon take over the research process itself. The authors acknowledge substantial uncertainty, but the speed they describe would make the window for response unusually short.

What I’d want to know is how policymakers can track that shift before the pipeline is automated. The paper calls for closer monitoring, but the warning’s force depends on whether that monitoring can keep pace with the systems it is meant to observe.