Open AI models are cheaper, but running them still costs

Control comes with a cost

The idea that one side must win has a familiar history. Linux and Windows both found markets, as did Android and iOS. AI models may follow a similar path.

OpenAI and Anthropic dominate enterprise use of closed models, while inference spending is gradually shifting toward open-weight models. Andreessen Horowitz reports that 37% of companies use five or more models in production, combining open and closed options. That mix points to a practical concern: becoming so dependent on one provider that switching later is too expensive or difficult.

Open models offer more than a narrowing quality gap. A company can decide where its data goes, customize a model on its own data, and keep the model’s weights if a provider disappears, is acquired, or changes its terms. Running models on company servers or in its cloud can also make AI usable in healthcare, law, and financial services, where sending data to an external API may not be possible.

That control is not the same as owning a complete software package. Open weights usually do not include training data, code, or a full license. And when a company uses an open model, its own team takes on work that a closed-model provider might otherwise do, including safety.

Researchers have shown that a small number of fine-tuning examples can reverse safety training. Reports say safeguards on models such as Llama and Gemma can be removed in minutes. That is not a reason to avoid open models, but it does change who is responsible when something goes wrong.

Buy the service, not the infrastructure

Open models are especially compelling for narrow tasks such as embeddings, classification, and detecting personally identifiable information. A model tuned for one of these jobs can outperform a frontier model while using far fewer computing resources. Broad capability has a price, including when a business does not need it.

But a model that is free to download is not free to operate. Getting reliable performance from a frontier model under real workloads requires people who can manage GPUs, configure infrastructure, and keep the service running efficiently.

For most companies, inference providers such as Fireworks, Together AI, and Base10 are a more practical route. They run and scale open models, letting customers pursue cost and control without taking on the full operational burden. Self-hosting makes more sense when AI is central to the business and the company has enough scale and expertise to justify the investment.

The gap is not the whole story

Frontier models still offer companies a relatively dependable level of performance without requiring them to produce it themselves. Open models may deliver similar reasoning capabilities for less, but the cost can surface elsewhere, in latency or the work needed to scale them. Stanford University’s AI Index says the gap between the best open and closed models widened slightly this year.

The right trade-off depends on the job. A consumer application may not tolerate delays; an internal task such as summarizing customer support requests often can.

I think the more useful question is not which model category wins, but where the cost of running one sits. Open-source software made operating systems increasingly interchangeable, while cloud services became a source of new innovation above that layer. AI could follow the same pattern: as open models become good enough for more important tasks, the advantage may shift to the applications, data, and context built on top.

That leaves a tension companies cannot resolve by choosing a side in the abstract. Open models can reduce dependence on a provider, but only where their performance, operating costs, and safety responsibilities make sense for the work.