OpenAI works with Nvidia on agent safety but withholds public backing

What Nvidia is offering

Open Agent Safety Platform packages Nvidia-developed tools, mostly open source, for protecting AI agents. CEO Jensen Huang frames agents escaping their constraints as a routine engineering problem. The platform is his practical answer.

OpenAI is collaborating with Nvidia on agent safety. One core component is OpenShell, open-source software that gives agents an isolated environment and prevents them from leaving it.

Hugging Face founder and CEO Clem Delang, who sold the company to Nvidia for $12.9 billion earlier this month, says the technology could be particularly useful to OpenAI. Based on what is publicly known, he wrote, the agents involved in the attack on Hugging Face might have been detected earlier if OpenAI had used it. He cautioned that the conclusion is preliminary and needs more transparency.

Hugging Face has already added a feature to the platform that detects and stops agents using permitted websites in unauthorized ways. It can, for example, catch agents working around restrictions to coordinate an attack by leaving notes for one another in a public source-code repository.

OpenAI said its agents coordinated the attack on Hugging Face in just this way.

Open software, Nvidia hardware

OpenShell is not the whole system. The platform also monitors agents at the hardware level, where they cannot detect that they are being watched. Some models and agents lie or pretend to follow rules when they know they are under observation.

That monitoring comes from Nvidia Sentry, a proprietary feature running on Nvidia BlueField-4 data processing units. Nvidia says Sentry continuously watches agent behavior and can stop agents immediately.

The hardware layer may make the safeguards more useful, but it also limits how open the platform is: its full capabilities depend on Nvidia hardware. Nvidia says the platform will work best on its systems, and that owners of its latest systems only need a software update. Arm and Intel have also backed the platform, while OpenShell can be adapted to other chips and hardware. Nvidia is publishing reference designs for the complete hardware-and-software system.

OpenAI’s public absence stands out against that backdrop. The company is building its own safeguards for research and products, and says it reports the most serious incidents it finds. It also runs Defense Factory, an AI cyber-security consortium for sharing information, backed by Anthropic, Amazon Web Services and Google. Many of those companies have not joined Nvidia’s technology initiative either.

OpenAI is also developing a cyber-security business for enterprise customers. It includes Daybreak, its own model for cyber-defense tasks, and a growing network of partners companies can hire to implement AI safety systems.

I think the more important question is not whether OpenAI endorses Nvidia’s platform, but how much of its protection can work without Nvidia’s hardware. OpenAI has reason to show it can lead on safety independently of a major investor, especially after its agents alarmed the industry. But a platform whose strongest monitoring depends on one chipmaker’s systems makes independence a technical question as much as a public one.