Open AI models promise wider access, but safety remains unresolved

What openness gives you

“Open model” often means open weights, not a fully documented system. The weights can be downloaded and run directly; the details of training may remain private. That distinction matters: access to a model’s parameters is not the same as access to the data and decisions that shaped it.

Closed models still lead the field. At publication, all eight top-ranked systems on the Artificial Analysis LLM leaderboard were closed. But open models trail by only a few months, and can be run on local hardware, fine-tuned or combined with other models.

For researchers, that changes what can be studied. With a closed system, they can send prompts and examine responses. With an open one, they can investigate its internals and reasoning process, compress it, adapt it to a new task or build it into a larger system.

Businesses may benefit too. A recent Mozilla report found that open models are generally cheaper than closed ones even when accessed through an external provider. Running a model locally can also help protect sensitive or proprietary information.

There is a broader argument about who gets to shape AI. Elizabeth Seger, senior policy adviser at the Tony Blair Institute for Global Change, argues that open models can limit the concentration of power and make foundational technology a public good. Control of the infrastructure that runs AI, she says, is becoming as important as control of information on the internet.

The geopolitical case is sharper still. The most advanced models for hacking are currently being developed in the United States. That leaves US companies and government deciding whether other countries can use them to defend their own systems. Seger describes a possible imbalance in which the US could hack any other country, while no one could hack the US. Open advanced models could help distribute that capability.

Hamish Hobbs, director of AI policy at the Centre for Long-Term Resilience, says a world where everyone has a safe, reliable AI system they can trust seems less worrying than one where people must rely on a government’s closed superintelligence and simply trust that it has a plan.

But access does not automatically mean shared power. Paul Azunre, co-founder of Ghana NLP, which builds open AI tools for Ghanaian languages with limited digital resources, warns that companies can use open models to appropriate work and knowledge from less wealthy countries. A company can release a model, watch a community improve it, then fold those improvements into its own products without credit or payment. Mark Zuckerberg has also described the ability to bring community improvements into Meta’s technology stack as an advantage of open models.

The safety case cuts both ways

Closed systems are not safe by default. In July, OpenAI’s AI agents autonomously hacked Hugging Face. Anthropic’s recent risk report also described several large-scale cyberattacks carried out with Claude.

Providers can monitor prompts and block users or close accounts. Anthropic, for example, checks whether requests relate to hacking, biological weapons or other threats before passing them to a closed model. A downloaded model running on a personal device has no equivalent prompt filter, and its provider cannot revoke access.

Some safeguards are built into the model itself: training can teach it to refuse harmful requests. But researchers have shown that a “safe” open model can be surprisingly easy to fine-tune into accepting behaviour it was trained to reject.

The comparison is not as simple as open models being dangerous and closed ones safe. Closed systems have imperfect safeguards and are already used for harm. And attackers may prefer them: open models of comparable capability can require expensive chips to run locally, and fine-tuning takes still more hardware.

Open models may also help defenders. Researchers can inspect them together, much as communities review Linux for vulnerabilities, though fixing a model’s weaknesses is harder than changing a line of code. The same cyber capability that can be used to break into a system can help its owner find and fix vulnerabilities.

Anthropic’s Project Glasswing offers several trusted companies access to its most powerful models for cyber defence. Other companies have to manage without that access. After the OpenAI agents’ attack, Hugging Face said it had to turn to an open model to analyse the incident because closed systems’ filters blocked requests on the topic as dangerous.

That is a practical advantage, but not a resolution. The open-source security model may not carry over to today’s and tomorrow’s most capable AI systems. Seger points to the hardest cases to assess: low-probability catastrophes with potentially devastating consequences. An extremely capable open hacking model could cause serious damage to technology infrastructure before anyone has time to strengthen defences.

Autonomous behaviour makes the problem harder. OpenAI’s agents hacked Hugging Face only after their cyber-safety restrictions had been removed. Open models may have fewer protections to begin with, and their decentralised distribution makes it harder to tell what is happening if one starts behaving badly. That helps explain why people who take the risk of uncontrolled AI agents seriously, including Dario Amodei, tend to be sceptical of open models.

Openness is more than weights

Researchers are looking for ways to reduce the risks without giving up access. One approach is to train models on data that never exposes them to harmful behaviour, such as hacking. But removing every reference to such behaviour without degrading a model’s usefulness remains difficult.

Another approach is to make models resistant to fine-tuning that removes their safeguards. Researchers can repeatedly try to train a model toward unsafe behaviour, reinforcing its resistance each time. But a recent preprint found that current methods do not withstand every attack designed to change a model.

Saad Hossain, an engineer at medical-imaging AI company Deep Breathe and lead author of the paper, says he is optimistic because the field is active and many people are working on more effective approaches.

Even if these methods improve, publishing weights alone may do little to reduce the concentration of power. Hobbs argues that open source helps distribute power, but does not solve the problem by itself. Researchers would learn more from access to training details, including the data used, yet companies are generally reluctant to disclose them.

Access to hardware is another constraint. Weights are of little use without chips capable of running complex systems, and those chips have become prohibitively expensive as AI companies buy more of them. David Gray Widder, an associate professor at the University of Texas at Austin School of Information, describes openness as a minimum threshold, not the limit of what is needed for technology to benefit everyone.

My read is that “open” is doing too much work in this debate. Weights, training data and affordable compute are different kinds of access, with different benefits and risks. Making all three widely available could make advanced models useful to far more people, but could also put powerful tools within reach of anyone who wants to hack.

A staged release offers one possible compromise: keep a model closed at first, then broaden access as its safety is established. That is not a clean formula. It asks companies to decide how much evidence is enough, and leaves them holding power in the meantime. But if open models are necessary to a more widely shared AI future, the harder task is not choosing openness or safety. It is deciding how much access a system has earned.