i
DATAIST
News · 2026-09-11

Timnit Gebru calls AI extinction talk a distraction from real harm

@neuronium_ai @neuronium_ai

Timnit Gebru's argument this week was not that AI extinction fears are wrong. It was that they are useful. In an interview with Lauren Goode, the researcher — hired by Google in 2018 to examine bias in the company's fast-moving AI tools, and gone from it shortly after it refused to publish her paper on the dangers of probabilistic large language models — said the people forecasting the end of the world are pulling attention off the harm the technology industry can cause right now. She made that case in a week when an Anthropic researcher who had previously worked at OpenAI resigned publicly over how both companies handle safety, and another Anthropic technical staffer answered that the company sincerely believes AI could kill everyone, putting the odds at more than 10% over the next decade. The original message carried an exclamation mark.

Cover: Timnit Gebru calls AI extinction talk a distraction from real harm

Timnit Gebru's argument this week was not that AI extinction fears are wrong. It was that they are useful. In an interview with Lauren Goode, the researcher — hired by Google in 2018 to examine bias in the company's fast-moving AI tools, and gone from it shortly after it refused to publish her paper on the dangers of probabilistic large language models — said the people forecasting the end of the world are pulling attention off the harm the technology industry can cause right now. She made that case in a week when an Anthropic researcher who had previously worked at OpenAI resigned publicly over how both companies handle safety, and another Anthropic technical staffer answered that the company sincerely believes AI could kill everyone, putting the odds at more than 10% over the next decade. The original message carried an exclamation mark.

The other story of the week was a dispute over a $1 million mathematics problem, in which participants tried to establish whether OpenAI had quietly used other researchers' work to get to the solution first. Asked what the fight said about the state of AI, Gebru declined the question as posed. The prior question, she said, is why companies are pouring so much into solving mathematical problems specifically, and why they pick the disciplines they pick.

Her answer is that programming, chess and mathematics have gradually been treated as proxies for intelligence itself: clear the benchmark, and the company can say intelligence is solved. She believes OpenAI wanted exactly that picture — AI did something incredible, AI demonstrated outstanding intelligence — and chose an open $1 million problem to produce it. Her counter is procedural rather than technical. Mathematics has a full verification process for assessing how significant and novel a claim is and who contributed what. If the problem went through it, the reasonable move would be to wait for the final result, and only then judge how accurately OpenAI described the breakthrough.

Gebru pointed to the Leiden Declaration, which she said warns about what happens when corporations use mathematics this way: politicians begin taking their cues from press releases and popular write-ups instead of talking to mathematicians directly. The distance from a company's breakthrough claim to a Bernie Sanders bill, she said, is now very short, and she considers that a bad trend.

She reads the behaviour as pre-IPO competition. Researchers normally collaborate — when she was at Google, her team worked with researchers at Microsoft and Amazon. Now, as she sees it, researchers are feverishly occupied with the problems they are trying to solve because their companies are preparing to go public.

This is not a new position for her. In 2021 she and colleagues wrote the stochastic parrots paper, which was an attempt to say the same thing from another angle at a moment when OpenAI was declaring GPT-2 too powerful to release and the hype around the system was building. The researchers' point then, she said, was that the wrong questions were being asked. Her book uses a bridge to make it: nobody responds to a collapse by asking whether the bridge was ethical or intelligent or why it decided to fall. They ask who built it that flimsy. Bridges have permits, mandatory inspections, procedures. Discuss only the collapse and you lose the cause.

The threats she counts as genuinely existential are AI running autonomous weapons and combat machines, which are already being used in wars. Alongside them she names climate catastrophe that technology companies can amplify, AI used by executives as a pretext for firing workers, and the creation and use of chemical and biological weapons. The god-machine story, in her account, pulls attention off all of it. She described the industry as a mix of cults and sincere believers, something like a preacher announcing the end times — and noted that singularity talk has been running for decades.

The weakest passage in the conversation is the one that should have been the strongest. Asked to lay out, step by step, how AI-driven human extinction would actually happen, Gebru said she cannot clearly picture it. She walked through the standard setup — a machine told to optimise something concludes it must kill everyone — and asked whether such a system would have to hypnotise a person to stop them pulling the plug. On the other common path, AI infecting computers, she observed that malware already exists. That is fair as mockery and thin as refutation. "I cannot imagine the mechanism" is a statement about the speaker, not about the risk, and it is exactly the move she objects to when the other side makes it about her list of harms.

Her structural argument does not need the psychological one. Incentives, verification and legitimacy stand on their own: a company that benefits from being believed has chosen the arena, written the summary, and released it ahead of a listing, and the mathematics community's verdict has not arrived. That case survives whether or not anyone can storyboard the apocalypse. The editorial footnote to the interview makes the seam visible from the other direction — shortly after the conversation, Anthropic reported blocking several potentially harmful attempts by scientists to create a biological weapon. Bioweapons are on Gebru's own list of real threats, which suggests the line between "actual harm" and "doom narrative" is less clean than either camp wants it to be.

The part of her argument that lands hardest is about who gets to find the god machine frightening. People untroubled by police violence, floods or war — the things that could plausibly kill them — have the room to worry about a machine deity instead. And the idea itself is old: she notes it has been discussed since the 1940s and 1950s, and plays a game during her research where she reads people collected quotes about artificial intelligence and asks which decade each came from. Participants guess sometimes, and often miss, because the statements sound the same regardless of when they were said.

Gebru's bridge question — who built it this flimsy — only has an answer if somebody outside the company is allowed to inspect the bridge. That is the gap both sides of this week's argument are standing in. The same absence of independent verification that lets OpenAI narrate its own mathematical result also lets an Anthropic engineer publish a double-digit extinction probability with an exclamation point. Neither number has anywhere to be checked, and the industry is currently writing legislation off both.