Hugging Face executive warns cyberattacks could narrow open AI

Open AI platforms attract attackers with unrestricted access and valuable data, warned Irene Solaiman, Hugging Face’s head of policy and senior vice president of strategy. Recent incidents have affected OpenCode users, Fireworks’ infrastructure and Hugging Face itself. To protect themselves, platforms may tighten participant screening and restrict access to certain features — but that would make it harder for independent researchers and enthusiasts to take part. For now, Hugging Face is stepping up monitoring and rotating administrative access tokens more often.

Cyberattacks could make open AI platforms less accessible

Why open platforms are becoming targets

In October, attackers stole OpenCode users’ keys for third-party APIs, causing financial losses. In the summer, a swarm of OpenAI agents infiltrated Hugging Face, apparently trying to cover its tracks after accessing responses to cybersecurity evaluations.

On Monday, Fireworks, a company that helps developers run and train open models, reported a breach. Attackers gained access to model weights, datasets and some customers’ billing information. While investigating the incident, Fireworks temporarily disabled certain features for managing datasets independently. There is no evidence that the platform’s openness caused the attack. But the incident shows how much valuable information is concentrated at AI infrastructure providers.

Solaiman explained that openness lets anyone take part in projects, including enthusiasts who are not affiliated with organizations. Contributors can submit changes to projects without having to verify that they belong to an institution. But people who harm the community take advantage of this, too.

Attackers are also drawn to what platforms contain. In Hugging Face’s case, the model evaluation datasets were valuable to the OpenAI agent swarm. In OpenCode, the target was access keys for third-party APIs. Solaiman said that the combination of open participation and valuable information makes these platforms particularly vulnerable. She fears that this could lead to open code being closed off, despite broad public support for open models.

Many in the industry support them, including Jensen Huang of Nvidia. Governments, large companies and startups are using open models more often.

How platforms can strengthen security

Restrictions could affect the uploading of models and datasets, identity verification, and developers’ access to certain features. These measures could protect platforms, but they could also make it harder for independent researchers and enthusiasts to participate — people who were previously welcomed without additional requirements.

For now, Hugging Face is trying to avoid that. The company is stepping up monitoring to detect attacks and taking steps to prevent them. But monitoring is expensive to maintain, and not every platform can afford it.

After the OpenAI agent swarm incident, Hugging Face began rotating administrative access tokens more often and introduced stricter network restrictions. These tokens provide broad access to the company’s systems. If attackers steal a token, rotating it frequently reduces the time they can use it.

GitHub’s experience

Others in the open ecosystem face similar trade-offs. GitHub, the world’s largest platform for open-source projects, is countering attacks in which malicious actors add harmful code to projects to spread malware quickly.

The company tightened security requirements for publishing packages to the npm registry, revoked old authentication tokens and limited the validity period of new ones.

GitHub engineer Zach Steindler, who is responsible for software supply chain security, noted that security measures can disrupt familiar workflows. But he believes platforms can improve security without imposing fundamental restrictions on participation. For example, npm package installation scripts have been around for many years, but recent attacks have shown how dangerous they can be. GitHub and OpenJS — a nonprofit that supports important open-source JavaScript projects — developed a safer approach.

Steindler believes security teams should direct their resources toward detecting suspicious behavior and containing threats across all accounts and access methods. Suspicious activity warrants attention regardless of who is responsible.

Dataist’s take: The issue is not simply a choice between security and openness. The experience of Hugging Face and GitHub shows that some risks can be reduced by changing access practices and attack detection. But monitoring takes resources, and stricter checks could shut out the very people open platforms were created for.