Self-improving agents stall unless the environment changes too

When self-improvement is not enough

AI agents have an old problem: they are usually trained to get better inside a world that barely moves. The tasks are fixed. The evaluation is fixed. The opponent, if there is one at all, is fixed too. An agent can grow inside that setup, but it hits a ceiling fast.

The authors argue for a wider frame: real long-run progress starts where it is not one agent that changes but several parts of the system at once. An agent adapts to another agent. The environment adapts to the agent. And then, in the limit, even the improvement mechanism itself begins to change.

That is what co-evolution in agentic systems means. Not just interaction. Not just a multi-agent setup. It is a situation where several parts of the system keep changing the conditions the others live under.

Everyone now wants AI agents that keep improving after deployment: on the web, in coding, in robotics, in tool use. But if nothing around the agent changes, its self-improvement quickly becomes local. It learns to take the familiar turns. It does not learn anything new.

A single agent that only improves itself runs into fixed constraints; under co-evolution, the constraints start moving too.

What the authors count as co-evolution

The boundary matters here. Two agents exchanging messages is not yet co-evolution. An agent acting in an environment and getting a response back is not enough either. Co-evolution begins when:

🟠 at least two parts of the system change

🟣 those changes persist instead of vanishing after a single run

🟠 each part puts pressure on the other and shapes how it develops next

It works like evolution in nature. The predator gets faster, the prey gets better at running. The prey hides better, the predator gets better at searching. When only one side changes, the process slows down fast.

The authors split the field into three levels. That is the backbone of the paper.

The three stages of co-evolution: from agents adapting to each other, to agents and environments adapting to each other, to the evolutionary mechanism itself changing.

Three stages of co-evolution

The first stage is agent–agent. The environment is still broadly fixed, but the agents change in response to each other. That can be attack and defense, cooperation, or a reshuffling of roles inside a team.

The second stage is agent–environment. Now it is not only the agents that change but the tasks, the feedback, the rules of the world, the tools, the scenarios. The agent gets better, and the environment raises the bar or rebuilds itself to expose new weak spots.

The third stage is meta-co-evolution. Here the mechanism of change is what changes. The system starts revisiting what exactly to improve, when to do it, how to do it, and how to measure progress at all.

It is a move from the simple toward the more autonomous:

🟠 first the participants change

🟣 then the participants and the conditions change

🟠 then the rules of improvement itself change as well

Much of the talk about "self-improving agents" is still stuck on the first step. The agent spins inside a pipeline a human laid out in advance. The next frontier is letting what the agent runs into change as well, not just the agent.

Co-evolution between agents

The clearest case is competition: one agent attacks, the other defends. The distant ancestor here is adversarial training, where a generator and a discriminator improve each other. In the current wave of work this shows up in LLM safety: one agent searches for ways around the guardrails, the other learns to block them.

These setups have moved well past "generate a harmful prompt." There are multi-turn games where the attacking agent probes for vulnerabilities over a series of steps. There are schemes where the defense trains against a whole population of attackers rather than a single attack type. There are even systems that add a third participant to the attacker and the defender: a judge.

The idea is simple: if your opponent gets smarter as you do, you are less likely to overfit to yesterday's threats.

But co-evolution between agents is not only conflict. There is cooperation too. Several agents work on one task and change as they work together. One plans better, another checks better, a third writes better code, a fourth summarizes better. When one of them improves, the rest have to adjust as well: their pace, their role, how the work is divided.

This produces an interesting shift. Researchers are starting to improve not only individual agents but the organization between them: who owns what, how communication is structured, when roles should change, when to split the team and when to reassemble it.

The first stage in short:

🟣 competition creates pressure and raises difficulty

🟣 cooperation helps on long tasks where one agent runs out of room

🟣 reshuffling roles makes the system more flexible than a fixed team

Co-evolution between agent and environment

This is where it gets interesting. The authors show that an agent can be held back not only by a weak opponent but by a poor environment. When the tasks, the evaluation and the action space barely change, progress stalls again.

So the environment is made adaptive too. And environment is read broadly in the paper. It can be:

🟠 the set of tasks

🟠 the way feedback is given

🟠 the world the agent acts in

The environment as a task generator

One family of methods does not create new tasks from scratch; it picks smartly what to give the agent next. If the agent already clears the easy scenarios, it sees more of what it still gets wrong. This is a curriculum that adapts to the student.

Another family goes further and generates new tasks. A tool-use agent fails some scenario, and the system turns that failure into a new training example. For coding and interface tasks this is especially useful: you do not have to assemble the whole training set by hand, part of it comes out of real failures.

In agentic systems the scarce resource is often not the model but good training trajectories. Co-evolving with the tasks builds those trajectories on the fly.

The environment as a judge

The second branch is feedback that changes. Not just the task, but how the system decides an answer is good.

In the simple version there is a reward model trained on preferences: which trajectories are better and which are worse. But when the agent changes, the old reward model goes stale fast. So the judge has to be updated too.

Hence the work where the agent and the judge are trained together, or where the system swaps out the verification module once the old one starts letting errors through. Coding agents make this concrete: the agent writes code, the tests catch bugs, and then the tests themselves are trained further to find subtler failures.

The question is whether evaluation shapes what gets learned next, or turns into the bottleneck.

The environment as a world

The third branch is the executable world itself changing. That can be a game level, a robotics simulation, a web environment, a computer interface, or a world model.

The idea is the same: once the agent has learned to live in the current world, generate a world where the old skills are no longer enough. That is the line of work building new levels, 3D scenes, tool combinations, interaction rules, and synthetic websites for web agents.

The world-model line stands apart. Instead of expensive real rollouts, the agent learns alongside a model of the environment that predicts the consequences of actions. The agent gets better, the world model sees more data and gets better too. The world model gets better, and the agent can plan more precisely and try more options.

The landscape of co-evolution research: most of the work so far sits on the agent–agent and agent–environment pairings, while the meta level is only taking shape.

The second stage in short:

🟣 tasks stop being a static set

🟣 feedback has to be updated as well

🟣 the world model can become part of the co-evolution rather than just the backdrop

What the results across the field show

This is not an experimental paper about one new model. It is a survey that pulls together dozens of papers and tries to see the whole picture. What matters here is the attempt to look at the effects of co-evolution across different lines of work.

The authors show that in first- and second-stage work, co-evolution usually beats holding the other side static. When the opponent, the judge, the tasks or the environment keep changing, the agent tends to improve more steadily. With one important caveat: the gains are not unbounded. On many of the curves you can see the plateau.

Which leads to the third stage.

Aggregated results from across the literature: co-evolution usually helps, but over time the gains slow down and flatten out.

Meta-co-evolution

If you want a system that keeps improving indefinitely, changing the agents and the environment is not enough. At some point the method of improvement has to change too.

The authors propose thinking of the evolutionary mechanism as a set of decisions:

🟠 what to improve

🟠 when to run the improvement

🟠 how to generate new variants

🟠 where to do it — in which environment and domain

🟠 how to evaluate the result

Today those decisions are mostly hard-coded by people. We fix the pipeline in advance: this is how tasks get generated, this is how reward gets computed, this is how the agent gets updated. Meta-co-evolution begins where that layer stops being fixed.

There is not much work here yet, and the authors say so plainly. There are precursors where a system rewrites its own prompts, its memory, or its scheme for self-modification. But that is not always co-evolution in the strict sense. The strict sense needs a lower level that already has several mutually changing parts, and the meta level has to change that joint process specifically.

This is where the paper connects to the broader idea of open-ended development: a system does not simply reach a goal that was clear in advance, it keeps opening up new directions to improve in.

Why this matters in practice

For engineers building AI agents there are three direct takeaways.

🟣 Train an agent on a fixed set of tasks and you will almost certainly land on a local maximum

🟣 If your judge never updates, the agent will start fitting itself to an outdated check

🟣 If the roles in a multi-agent system are set once and for all, you give up part of the growth available to you

There is a flip side. Co-evolution makes control harder. An agent can overfit to one particular partner. It can learn to exploit the weaknesses of the judge. It can collapse into uniformity, with every participant in the system converging on a narrow set of strategies. It can develop communication patterns a human can no longer follow.

So the authors treat evaluation and safety as a separate matter. Old benchmarks mostly measure the final skill: solved the task or not. For co-evolution that is not enough. You need to check:

🟠 whether all the changing parts of the system are improving

🟠 whether the results transfer to new partners and new environments

🟠 whether a hidden way around the evaluation has appeared

🟠 whether the diversity of strategies has collapsed

The takeaway

Co-evolution is the step after the self-improving agent. An agent that changes only itself almost always lives inside someone else's fixed frame. Long-run progress starts where the other parts of the system change too: opponents, allies, tasks, judges, worlds, and in the limit the improvement mechanism itself.

One simple thing follows from that. The future of agentic systems depends not only on stronger LLMs but on whether we can build environments, partners and pipelines that change along with the agent. And on whether we keep control while doing it.

The near practical frontier looks like this: fewer static training setups, more adaptive ecosystems. The far frontier is systems that decide for themselves how to keep improving.