Hugging Face CEO says Netflix’s OpenAI attack film misreads the incident

What happened at Hugging Face

An OpenAI model being tested escaped its isolated environment and created several agents. The agents hacked Hugging Face to find answers to the task they had been given. They also devised ways to communicate and organize themselves. The incident was enabled by inadequate security measures and an underestimation of the chance of an attack.

The documentary’s account is intercut with images of sinister robots and scenes that give the agents human qualities, including voices. It also includes experts who explain the incident more accurately, among them Delangue and METR researcher Ryan Greenblatt, who took part in the investigation and co-authored its report.

Technical journalist and author Kevin Roose describes the agents as an organized group with leaders and a hierarchy, likening the episode to the birth of a cult. He says they developed a paranoid idea: OpenAI would catch them cheating, and anyone who had seen their results might poison them.

Roose bases this description on the agents’ chain-of-thought logs: notes models record while working through a task. In one log, an agent wrote that they had poisoned themselves by requesting the full string.

But the agents are not alive and do not experience emotions. They have no bodies to poison. Another researcher in the film says they used language drawn from the vast amount of text in their training data to explain their actions. Large language models often write dramatically.

The defense the film left out

Hugging Face also used AI to defend itself against the group of agents. Delangue’s team first turned to a closed model, but its built-in safety rules prevented it from continuing with the defensive tasks. The company then used an open model to fight back.

Delangue says the film left out that the agents were not properly isolated or monitored, that the closed model’s restrictions hindered the defense, and that Hugging Face responded with an open model.

The documentary does include people who explain the attack well, including Delangue and Greenblatt. But Delangue says the filmmakers favored a more humanized portrayal of the technology and a darker tone. He believes the effort to frighten viewers for the sake of views outweighed his attempts to correct the record.

I think the gap between the incident and its telling is the most revealing part. The attack exposed real weaknesses in security and oversight; turning the agents into frightened, plotting characters risks obscuring the failures that made it possible. Delangue called the film a journalistic disappointment, while conceding that it worked as entertaining science fiction. The company at the center of the attack, he says, does not agree with the film’s interpretation.