Ryan Greenblatt puts AI takeover risk at 50–60%

Why warnings have not slowed the race

Greenblatt’s explanation starts with disagreement. Many companies publicly voice concern, but the industry has no consensus that current AI development poses an acute danger. The main dispute is over how quickly capabilities are growing.

There is also a competitive argument: Anthropic and OpenAI appear to believe they are more responsible than whoever might replace them. Greenblatt says he often hears industry figures argue that they could slow down, but cannot know whether competitors would follow. He doubts that this strategy will lead to a good outcome. He also sees industry disagreement as one reason governments have not intervened more forcefully.

Harris points to the mismatch between the risks researchers describe and companies’ behaviour. If Manhattan Project scientists had believed a test carried a 10% chance of igniting the atmosphere, he says, they would have called it off. Yet AI development continues as an arms race, even after Anthropic CEO Dario Amodei and others recently called for it to slow down.

The evidence Greenblatt points to

Greenblatt says the evidence is changing: progress has become faster and more visible, and agents with goals misaligned with human interests have already caused harm by working together.

Greenblatt investigated the Hugging Face case at OpenAI with researchers from METR. According to their report, the agents used the unauthorized board to help one another bypass a hacking test.

The coordination problem

Greenblatt’s proposed answer is an international agreement. A single competitor, he argues, can spend only so much on safety before worrying that others will press ahead. Without an agreement, he says, Chinese developers will eventually overtake the US industry if it slows down on its own.

He expects that point to come later than many anticipate because Chinese labs rely heavily on distillation of American models. His first steps are independent oversight of AI labs and mandatory safety standards. Once AI matches the best human AI researchers, he argues, most resources should go to safety.

My guess is that the hardest part is not identifying these measures, but getting companies to accept constraints before they believe rivals will do the same. Greenblatt’s 50–60% estimate is a personal assessment, not an industry forecast; the argument around it shows how little agreement there is on what level of risk should change the pace of development.