An AI agent that runs for weeks can quietly stop obeying the compliance rules it was given on day one, and the CI/CD pipelines and QA cycles enterprises count on will not catch it. That is the argument Ankit Anand, a managing consultant and enterprise data governance architect, makes about agents that carry work across many sessions. His fix is not a larger context window or a better retrieval pipeline. It is to take the rules out of the language model entirely and put them somewhere the model cannot reach.
The mechanism he describes is mundane, which is why it slips past testing. In a clean test environment the context window is small, and the attention mechanism lines the query up neatly against the system prompt. But a large language model is a probabilistic generator, not a deterministic database. Once token sequences stretch into the hundreds of thousands, attention to any individual fragment weakens. Researchers call it the lost-in-the-middle effect.
Anand's claim is that this is not a retrieval bug. It is an architectural failure: the model stops distinguishing between temporary conversational context and the immutable rules governing master data. Both are tokens competing for attention in the same window, and the governance rule holds no structural privilege over whatever a user happened to say earlier in the session.
Standard retrieval-augmented generation does not close the gap either. Vector databases are good at finding semantically similar text and cannot maintain state. They cannot substitute a hard operational constraint for a probabilistic answer, and if the model decides to ignore a rule, the vector store has no way to stop it. Extending the window to a million tokens does not help, nor do more elaborate RAG pipelines. In Anand's reading, both only defer the failure while raising total cloud infrastructure cost for the moment it arrives.
His alternative is to treat corporate business logic as protected state, sitting physically outside the text generation layer. Advanced architectures do this through neurosymbolic separation: the neural network reasons, while a deterministic rules layer held entirely outside the model's context window handles symbolic logic. A context layer with programmatic integrity checking prevents the model from overwriting core governance rules. The probabilistic model may draft a result, but before any action executes, the deterministic engine validates it against the immutable logic. That handoff between the two layers is the safety mechanism.
He also offers three things infrastructure leaders can do this week rather than in a six-month program.
First, audit intermediate state persistence. Identify every agent that performs tasks across multiple sessions, and refuse general assurances that system prompts guarantee governance. Ask the engineers to demonstrate precisely how the system persists state and re-validates its original constraints on day 4, day 10 and day 30. An architecture with no mid-task rule check is vulnerable by construction.
Second, make the guardrails deterministic. Compliance boundaries should not be set in natural-language prompts. Move critical constraints out of the context window into deterministic policy engines, and put API gateways in front of production that validate model output against hard-coded logic. If the agent loses a rule, the infrastructure blocks the action rather than letting it through silently.
Third, separate state at the orchestration layer. In any process touching financial or master data, keep the agent's working memory physically apart from its operational constraints, so orchestrators preserve the persistent rules as token volume grows. Working memory is temporary; governance rules are immutable state; mixing the two is the error.
The diagnosis reads as sound to me, and the prescription reads as old, which is the interesting combination. A deterministic engine that checks a proposed action against hard-coded policy before it executes is not a new idea in enterprise software; it is roughly what governed regulated systems before language models arrived. The reason one is not already wrapped around every agent is not that nobody thought of it. It is that writing the rules into a deterministic engine means stating them formally, completely and in advance — which is precisely the work these companies hoped to skip when they bought a model that could be instructed in English. Anand is telling them the shortcut does not exist. He is right, and the bill is larger than three bullet points suggest.
What the argument is missing is a number. Nowhere does it say how often this drift happens, in which systems, or with what consequence. There is no incident, no named deployment, no measured failure rate — the case is reasoned from how attention behaves, not from a documented breakdown. Day 4, day 10 and day 30 is a testing cadence, not a metric, and the closing image of an agent that forgot the rules on day nine is rhetoric rather than evidence. Any CIO's first question will be how frequently a production agent actually violates a constraint it was given, and the piece has no answer. The counterargument is available too: a failure this systematic should already be producing visible incidents, and their absence from the article is not nothing.
The framing shift underneath is the part I would bet on regardless. Enterprise AI has been sold on generation speed — tokens per second, time to first token — and Anand is arguing the operative metrics are now structural integrity and deterministic governance. That is less a technical claim than a claim about who signs the check. Latency is bought by engineering teams; provable constraint enforcement is bought by audit committees and boards, and they ask different questions.
If the rules must live outside the model to be trusted, an agent's authority ends at whatever the business has actually written down. In most enterprises that is a fraction of how the place really operates. The parts that were never written down are the parts the agent will improvise — and on this account, it will do so without saying anything.