Google DeepMind researcher quits over near-term superintelligence

The risks he sees

O’Callaghan cited a range of possible harms:

He said his next step is to work on something that clearly serves people.

O’Callaghan also linked to If Anyone Builds It, a book by Eliezer Yudkowsky and Nate Soares. But he did not endorse the authors’ claim that building superintelligent AI will inevitably lead to catastrophe.

The authors are associated with the effective altruism movement, which has been criticized for favoring charitable giving over redistribution. Axios reported that a White House memo described the movement as a fringe group resembling a cult. The memo also criticized Anthropic CEO Dario Amodei, calling him the face of a “doomer” view of AI development.

The disagreement is about more than risk

The public argument is already shaped by competing labels: “inherently irresponsible” on one side, “doomer” on the other. My read is that O’Callaghan’s departure matters less as a prediction of disaster than as a sign that some researchers see the decision to build superintelligence soon as a choice that requires justification.

What I’d want to know is what “clearly serves people” means in practice—and whether any work on advanced AI can meet that standard. Without that answer, the debate risks narrowing to a contest over who sounds more alarmed, while the central question of what responsible development would look like remains unresolved.