i
DATAIST
News · 2026-09-11

Bengio argues the danger is created in training, not deployment

@neuronium_ai @neuronium_ai

Yoshua Bengio, one of the pioneers of deep learning, is making a narrower and harder claim than the warnings that usually circulate about AI: the danger is produced by the training process itself. For several years he has argued that development should slow down and that models should be trained or released only after an independent safety check, and about a year ago he founded LawZero to build safer AI systems. The position sits directly opposite the White House. Donald Trump does not see a threat and wants the United States to stay ahead of China; if America does not win the AI race, he has said, it could end up in a very bad position.

Cover: Bengio argues the danger is created in training, not deployment

Yoshua Bengio, one of the pioneers of deep learning, is making a narrower and harder claim than the warnings that usually circulate about AI: the danger is produced by the training process itself. For several years he has argued that development should slow down and that models should be trained or released only after an independent safety check, and about a year ago he founded LawZero to build safer AI systems. The position sits directly opposite the White House. Donald Trump does not see a threat and wants the United States to stay ahead of China; if America does not win the AI race, he has said, it could end up in a very bad position.

Where you put the danger determines what counts as a fix. If a model becomes hazardous in use — misuse, bad deployment, bad incentives on top of a neutral artifact — then the industry's existing safety apparatus is roughly the right shape: test the finished system, add guardrails, watch what people do with it. If the hazard is created while the model is being trained, all of that arrives after the fact. Bengio's demand that verification happen before a training run, and not only before release, is not a procedural preference. It follows from where he puts the problem.

It is also the strangest part of the position, because it asks for something to be checked before it exists. A pre-training review cannot examine a model. It can only examine a plan: the data, the objective, the compute budget. That is closer to licensing than to auditing, and it is a far heavier thing to ask for than the phrase "independent safety check" makes it sound.

LawZero is the other half of the argument, and a different kind of move from signing a letter. Calling for slower development asks other people to stop; building safer systems obliges you to show what the safer version looks like. A year in, that is the part of Bengio's case that has to survive contact with engineering rather than with debate.

The warnings around him have also changed source. Many of the recent ones have come from people employed inside the AI labs, which is what moved "slow down" from an outside complaint into an industry conversation. It removes the cheapest rebuttal available to the labs — that the critics do not understand what they are criticising.

My own reservation is about what is missing rather than what is claimed. As it stands here, "the training process itself makes AI dangerous" is an assertion, not a demonstration, and the mechanism is the thing that would make it actionable. The second gap is bigger: independent of whom? If the verifier is a lab's own safety team, nothing changes. If it is a government, it is the same government whose president has just said he sees no threat at all.

Trump is not answering Bengio's argument; he is answering a different one. A claim about what a system becomes during training cannot be rebutted by a claim about who finishes first — the race framing does not contradict it, it simply prices it out. That is the tension the next year runs on. The only intervention Bengio asks for costs time, and time is the one thing a race is organised not to spend.