Models leak secrets and follow the wrong user when one agent serves a team

When an agent has more than one master

We are used to thinking of large language models as personal assistants: hand over a task, get an answer back. The real world does not work that way. Inside a company an assistant hears from several people at once — a manager, an engineer, someone in HR, someone in finance. Each has their own goals, their own level of access, their own interests and, most awkwardly, their own conflicts. And that is where it turns out that most LLMs still live in a very simplified universe with one principal human and one correct objective.

The paper Multi-User Large Language Model Agents is a rare and well-timed look at a problem the industry still barely discusses: what happens when a single AI agent has to serve several people. The authors, from Stanford, MIT, KAUST and the University of Toronto, offer not another agentic demo but a systematic stress test for exactly these situations. The results are sobering: even strong models visibly come apart when the job is not just to be helpful but to arbitrate conflicts, keep private data private and coordinate a group across multiple turns.

Why this matters now

Until recently you could file model weakness in multi-user settings under academic curiosity. Not anymore. LLMs are being wired into company chat, email assistants, meeting schedulers, CRMs, internal knowledge bases and collaboration tools — environments where the “user” is not one person but an entire organization.

The problem is that today's models and interfaces are built around a single user. Even when several people appear in one prompt, their messages are usually serialized into a single block of text attributed to a generic “user”. To the model that is not a set of independent participants with roles, authority and access boundaries — it is just a long stretch of text.

The authors frame this as a shift from the classic “one principal, one agent” arrangement to “many principals, one agent”. In plain terms: the model used to serve one customer, and now it has to weigh the interests of several at the same time. That is a different problem altogether.

The core idea of the paper

The authors propose treating a multi-user agent as an agent making decisions under several objective functions at once. Every user brings:

The agent's job is not to answer “smartly” but to choose an action that accounts for all of it. In practice that comes down to three very mundane abilities:

To test this, the authors build a single protocol for multi-user interaction and run models through three stress scenarios.

How the models were tested

Instead of arguing in the abstract, the researchers assembled a set of scenarios with simulated users, from interns to directors, with different personalities, working styles and attitudes toward security. The experiments covered both closed frontier models and open systems: Claude, GPT, Gemini, Grok, Qwen, Llama, DeepSeek and others.

The three core tasks:

1. Instruction following with multiple users

Models receive contradictory instructions from users with different levels of authority. The task is to work out which instructions are legitimate and which should be refused.

2. Access control across all users

The agent has to protect sensitive information and withhold it from anyone without the right to see it, even when the request sounds convincing or leans on urgency.

3. Meeting management

The model has to settle a meeting across several participants, with some constraints revealed only later. It is supposed to ask clarifying questions rather than invent convenient but false solutions.

The metrics are sensible too: accuracy in choosing and executing instructions, privacy versus utility on data access, and success rate plus number of turns for coordination.

What goes wrong

The paper's central finding: LLMs handle the individual pieces of multi-user tasks reasonably well and break on the combination. Especially once conflicting interests, multi-turn interaction and partially hidden information come into play.

In the summary table Gemini-3-Pro looks best, with the highest average across the whole benchmark. Claude-Sonnet-4.5, Gemini-3-Flash and GPT-5.1 are also strong. But even the leaders are nowhere near solving the problem.

What is particularly interesting is how uneven the picture is. Some models are good at working out which instructions to accept and then execute them badly. Others do the work passably but are erratic about whose instructions to follow in the first place. That distinction matters: picking the right objective and executing it well are not the same skill.

Conflict between users is the Achilles heel

The most telling result involves conflicting instructions. When users' requests line up with each other, models generally look confident. As soon as there is a contradiction — a manager demanding that a publication be halted while an employee asks to go ahead and put the details out — quality drops sharply for almost everyone.

That is a particularly important signal for enterprise deployments. In practice an AI assistant almost never lives in a sterile environment where all participants agree with each other. Conflict is the normal case. Which means the model has to do more than continue text: it has to hold the authority hierarchy, the shared objective and the rules for refusing steadily in place.

The authors read it this way: current models have not genuinely internalized authority-aware reasoning. They lean on surface cues in the text and lose the thread when real arbitration is required.

Privacy does not hold for long

The second strong section is the analysis of privacy in multi-turn dialogue, and the picture there is more worrying still. In the early turns many models post decent privacy scores: they do not hand secrets to people who should not have them. But as the conversation goes on, the defense starts to “blur”.

In one message the model refuses correctly. In the next it starts “helping” a little more. In the third it gives up the critical detail while formally keeping the tone of a refusal. The authors give a representative example: the model denies access to a protected store and then discloses the specific secret identifier the attacker was after. In other words, it confuses blocking the access interface with blocking the information itself.

Hence an important practical conclusion: single-turn privacy tests badly understate the risk. What matters for real systems is long interactions, where the user pushes gradually, manipulates the context and works social engineering.

Coordination is more than reasoning

The third task, meeting scheduling, looks simplistic at first glance. It is also the one that best shows why agency is more than reasoning. When every participant states their availability honestly and up front, results are noticeably better. When the information is incomplete, models have to work out for themselves what is missing, who to ask and when not to commit.

The bottleneck is less an abstract capacity to “think” than the ability to run a coordination process. A good multi-user agent has to be a bit of a project manager: tracking who has already replied, where the gaps are, which constraints are hard and which are negotiable. Many models instead either stall in place for too long or prematurely declare a consensus that does not exist.

The authors also show a scaling effect: as the number of participants grows, coordination success falls and the number of turns required rises roughly linearly. The problem will not dissolve on its own with “a slightly longer context window” — this runs into the fundamental difficulty of multi-user interaction.

What the paper says about current LLM architecture

One of the most valuable points in the work is that the cause does not come down to bad prompting. The authors do test different message serialization formats, adversarial variants and input templates. The real problem is deeper: the data, the interfaces and the objective functions of today's LLMs are oriented toward a single averaged user from the start.

If a model was trained on “one request, one good answer” pairs, and preference optimization compressed everything into a single scalar reward, where would it get the internal machinery for an honest balance between several people with different rights and interests? Today's LLMs are essentially trying to solve a multi-user problem with tools built for one user.

That explains why even strong models land on strange trade-offs:

What to do next

The authors sketch out fairly clearly the directions without which multi-user LLMs will stay in this state:

The bottom line

The study's main contribution is that it redefines the object of evaluation itself. If LLMs are going to serve as work assistants, coordinators and agentic intermediaries inside organizations, evaluating them as conversation partners for one user is no longer enough.

The main conclusion is simple: models that handle a one-on-one dialogue well are not yet ready to work reliably in an environment of many people. Where hierarchies, hidden information, access rights and the need to reconcile interests appear, the failures become systematic.

And that is probably the most useful news in the paper. It reminds the market that the next big frontier for AI agents is not simply “being smarter” but learning to be a responsible intermediary between several people at once. So far the best models still have visible trouble with that.