One Does Not Simply Ship
an LLM to Production

From DataOps to AIOps: what production-ready AI systems are actually made of, and why the model alone delivers no business value.

These days AI agents that act on their own are nothing new. In demos it all looks great, but in production most of those agents don't work. So how do we make them production-ready? Let's figure it out.

Let's not confuse an LLM with an LLM-based system

A business doesn't need a model as such. A business needs a system that delivers a predictable, controllable, safe and economically viable outcome inside a real process.

That's why the winner isn't the company with the smartest model, but the company with the most manageable AI system. Systems like that have to be designed, measured, controlled and scaled up step by step.

When companies start adopting AI, they often think it's enough to pick a model, plug in an interface, give it access to the data — and the company will magically become AI-First (it won't).

In production the model is the intelligence core. But the end result is produced by the system around it: context management, integrations with various services, complex orchestration, mechanisms for handing a task off to a human, and the economic effects on top of that.

So deploying an LLM is still not the same as deploying an AI system that delivers business value.

Why the gap between demos and production is so wide

In a demo everything is usually sterile: a clear request, good context, and no conflicting rules.

In production it's the other way around: noisy data, chaotic wording, outdated documents — and a mistake can cost money, damage your reputation, or turn into a serious incident.

That's when it turns out that what breaks in production is not so much the model as the whole system around it: the wrong document made it into the context, or a tool call didn't fire. So what does a production-ready AI system in a company actually consist of?

Where value is actually created

For a business, value only appears when AI improves business metrics: it closes support cases faster, shortens the development cycle, and raises the quality of decisions.

If the system produced an output that an employee still has to spend half an hour checking and rewriting by hand, there may be no value in it at all. Sometimes AI in a process like that just adds one more layer of work.

So we need to understand which metric we are improving. AI mostly makes sense in three cases: when it makes the work faster, cheaper, or better.

We should start with the use case, not with the model

Most failed AI projects start the same way. First the team discusses how it's going to fine-tune yet another model and bolt RAG onto it. And only afterwards does anyone work out where all of this is actually supposed to be used.

That's the wrong logic. The right path starts with the process. What task is a person or a team actually solving? Where exactly in the process do time, money, quality or speed leak away? Where do you need AI, and where will plain automation do? And how should the human's role change after the rollout?

First we formalize the process as an AS-IS diagram. Then we identify the automation points, then we design the TO-BE process with a new role for the human and for AI agents. And only after that do we pick the technology for the specific task.

Not every task needs an LLM. If everything is defined by hard rules, that's deterministic logic. If the task is about prediction over large volumes of data, we use classical ML. And if we're working with language and meaning, that's where we bring in an LLM. A mature architecture is almost always hybrid: hard rules, calls to ML models for specific tasks, and an LLM to interpret meaning.

Four roles for AI in a product

One reason so many AI projects fail is that companies try to design every use case the same way. But AI has at least four different roles.

These are four different classes of product. They have different requirements for UX, control, risk, metrics and architecture. And the higher the autonomy, the greater the responsibility.

At the agent level it is no longer enough for the system to simply generate good answers. You need constraints, confirmations and rollback mechanisms.

The real problem is context

Companies often think the model is the main constraint, but in practice even small models can deliver tangible value. The problem is how that context gets used.

Even a powerful LLM will get things wrong if you hand it bad context. And the worst part is that it will be convincingly wrong.

Context inside a company is not just a knowledge base. It is the user's task, the state of the process, the applicable rules and constraints, the history of actions, the available tools and institutional memory: cases, exceptions, templates and accumulated decisions.

That's why context engineering is already a discipline in its own right.

You can't raise autonomy faster than control

This is one of the main laws of agent architecture.

As long as AI works as a feature, an assistant or a copilot, the risk is limited. It helps, but the human still makes the final call.

The moment the system starts acting on its own, the cost of a mistake changes too. The result can be a bad decision, and that carries legal, financial and reputational risks for the company.

That's why autonomy should only grow together with control.

The more an agent can do, the greater its degree of freedom. But we shouldn't be building free-roaming agents: our agents are there to help the business, which means we need more constraints, not fewer.

If autonomy grows faster than control, at some point you will inevitably run into an incident.

Without evals there is no quality management

LLM systems are dangerous because they can quietly degrade.

From the outside everything may look fine: the system responds, the interface is alive, users are getting answers. But inside, quality has already dropped.

You change a prompt and break an adjacent use case. You update an API and the agent gets worse at working with tools. You rebuild the index in the vector database and now RAG retrieves irrelevant information. But all of this can go unnoticed if there is no evaluation system — evals.

Evals are a quality management tool.

You need two approaches. The first is offline: golden datasets, reference and edge cases. The second is online: real requests, spot checks, tool-call failures and system degradation already in production.

DataOps — preparing the context

When people talk about LLMs in production, the attention often goes to the model, and data is treated as something secondary. But in practice, without good DataOps, nothing good ever happens.

A system runs on specific context, and that context has to be sourced from somewhere, updated, cleaned and kept in working order. Obviously we won't walk through the entire DataOps cycle here: it is far broader and also covers governance, labeling, data warehouse architecture and much more. But if we are talking about the foundation for LLM systems, there are several mandatory layers.

MLOps — the model lifecycle

Where DataOps is responsible for data and context, MLOps is about the model lifecycle itself. Again, not the whole of it in this article — only in the context of LLMs.

There is model training, or fine-tuning on new data, and adapting the model to a domain or a specific task. There is version control: which weights are in use, which configuration is in production right now, what exactly changed between releases. There is deployment and, of course, monitoring: how the model behaves after a release and whether quality has dropped on key scenarios because of data drift.

In classical ML, MLOps is the center of the whole system. But today even a perfect model is no guarantee that the entire AI system works well. That said, MLOps stays critical wherever you have your own fine-tuned models built for your tasks.

In LLM systems, the model also has to come with versioning, tests, a clear update cycle and change control.

AIOps — managing the entire AI system in production

AIOps, on the other hand, is the next level of maturity. It means managing the entire AI system in production.

Why do we need a separate term? Because we have to control not just the model, but also quality through evals, the cost of running the system, latency, tools, orchestration, context, RAG, security, agent behavior, fallbacks and everything that happens around inference, including how the system works with users.

In other words, AIOps is about managing the system’s behavior, not only its compute core.

For example, the model may be stable, but retrieval has degraded — and quality is already falling. Or the cost went up not because the model got more expensive, but because the context got longer and the orchestrator started making extra calls. And on top of that, users complain that the chatbot is useless. All of this goes far beyond MLOps.

So in AIOps several things matter: quality, cost, latency, tool reliability, context quality, security and observability.

In a mature company, AI in production is an operational environment where you need to see and control the whole process of solving a task: from the input and the context through to the action, the result, the cost and the risk.

Releasing an LLM system is a controlled experiment

Shipping an LLM to production like an ordinary product feature is a bad idea.

Out in the real world, even a well-tested system runs into things the tests never covered: strange phrasings, new types of cases, unexpected chains of actions and the human factor.

That is why launching such a system has to happen in stages. First — shadow testing on real data with no impact on the user. Then — internal use: the team running it on itself. Next you can simulate the behavior of a real user. After that — a launch for a small share of real users. Then — gradually widening the reach. And only at the very end — full rollout for everyone.

But something else matters even more: a release has to have rollback criteria defined in advance. If a metric goes down, tool-call failures become more frequent or quality drops, the release has to be halted automatically.

How to measure AI’s work properly

The business needs to measure completed work.

But even that is not enough. AI metrics have to roll up to the business level: time to market for products, margin or the cost of operations.

AI economics: count units, not tokens

It is naive to assume that AI economics equals the API bill.

The real cost includes not just tokens, but the work of the entire system: preparing and storing data, RAG, request orchestration, tool calls and escalations to a human.

So the question that actually matters is how much a usefully completed task costs. In a mature company, AI economics is calculated per unit of work or per unit of output.

UX in AI is trust design

In ordinary products, UX usually means ease of use. In AI systems, UX is also about managing trust.

The problem is that a model can sound confident even when it is wrong. People naturally tend to mistake a confident tone for competence. Hence the two extremes: blind trust or outright rejection.

Good AI UX calibrates trust.

The user should understand what the answer is based on, which data was used, which rules were applied, where the system is confident and where there is uncertainty.

What you need to show is not the "model's thinking", but the basis for the decision: fact-checking, what exactly was decided, which documents it was based on and which constraints were in force.

Escalation should also be part of the UX, not a sign that the system is doing a bad job. A good system knows when to stop and hand a human an already assembled, structured case.

Governance has to be executable

Many companies make the same mistake: they write policies and procedures, and assume that governance is now in place.

But it isn't. Governance that cannot be enforced technically is useless: it works at the level of good intentions, but it never works in reality.

If an agent must not move money, that has to be impossible at the level of permissions. If personal data must not be sent to the model, that has to be cut off at the level of guardrails. And if incidents have to be investigated, then every step has to be logged.

Real governance is policy enforced by code.

And the more an agent can do, the more this principle matters. Every new skill an agent gains brings not only more value, but also more potential risk.

Security starts with limiting the damage

As soon as a system gets access to documents, external services, APIs and data, a very real operational risk appears. You get prompt injections through documents and external sources, context leakage or tool errors.

So the basic security principle for agentic systems is this: give agents least-privilege access. Only the data they need and only the tools they need, for a defined period of time. Everything else is closed by default.

Security has to be built so that even when something goes wrong, the damage stays limited and controllable.

What companies can actually do with AI today

Most companies will not win on the strength of their own large model: they don't have the resources to train one. For them, the core intellectual asset sits in context, orchestration, process logic, integrations, evals, governance and applied economics.

That is where long-term advantage is created.

Not in access to the model itself, but in the ability to embed it into the company so that it works better, cheaper, more reliably and at greater scale than it does for everyone else.

AI transformation is a change of operating model

The main mistake companies make is to add AI to an old process: to bolt new intelligence on top of old logic, with people who aren't ready for change and no mature infrastructure in place.

A real AI transformation starts with questions: if an AI system can now do part of the cognitive and operational work, how should the process itself, the role of the human and the company's structure of accountability change? How do we scale the company without growing headcount, leaving people only the "human" work and handing everything else to AI?

In the AI-First paradigm, a person stops being the one who does the work. Their role shifts one level up: the human is now a system operator or a manager of AI agents.

It isn't only the speed that changes, but the very substance of the work. People don't like uncertainty and are afraid AI will replace them. What actually changes, though, is the way they work, which is why reskilling has to start early, right after the TO-BE process has been designed.

First the company's org structure changes to fit the new roles. Then the head of each function runs team meetings on AI adoption, where every employee shows their progress: what was done, what worked, what didn't and what conclusions were drawn. Best practices, working examples and finished agents are then shared company-wide through demo sessions and internal webinars.

In parallel, a separate series of training webinars is launched, covering how to build agents: from framing the task and designing the agent's role to testing, rollout and measuring the effect. Every practice developed along the way is captured in the corporate learning platform and becomes part of the onboarding standard, so that every new employee understands the company's new ways of working from day one.

We need to build a scalable AI platform

Trying to roll out AI everywhere at once usually ends the same way: local experiments, one-off demos and pilots, a different stack every time, no reusable patterns and no shared context, not enough control over multi-agent systems and no overall economics.

The right path looks like this:

Value first. Then the platform. Then scale.

Why AI projects fail

95% of AI projects fail not because of the technology, but for very down-to-earth reasons.

They were built for a demo, not for production. They had no outcome owner. They had no evals. The context turned out to be bad. Integration into the process was weak, or the economics didn't add up. Autonomy outran control, and governance stayed in a PDF. And on top of that, nobody ever decided how exactly to measure success.

But almost all of these reasons are predictable. Which means they can be managed.

Related reading

All technology

Sep 2, 2026 A New Era of Sport: How AI Is Changing the Limits of Human Performance Aug 17, 2026 How AI agents are starting to train robots on their own Jul 17, 2026 AI Product Discovery: How to Automate Product Idea Generation