Five levels of self-improving AI, and why level 5 barely exists

The last AI humans build

The idea is old and unsettlingly simple: at some point an AI stops merely solving tasks and starts improving itself. Not in the sense of "rewrote that answer a little better," but in the sense of "changed its own way of learning, checking itself, accumulating experience and building the next version." That is the subject of a long paper with a loud title: "The Last AI Humans Build".

The title isn't the interesting part. What the authors are really doing is imposing some order on the noisy conversation about self-improving AI. Where do ordinary agent tricks end? Where does real recursive self-improvement begin? And above all: what is already happening in industry, and what is still a polished demo.

Their central claim: today's LLMs can already do a great deal, but progress is very uneven. On tasks with a clean verifier they improve fast. On long, multi-step, messy workflows they do noticeably worse. And that is exactly where self-improving systems could pay off most.

What recursive self-improvement means

Put briefly, recursive self-improvement is when an AI uses experience and feedback to make durable changes to itself, and those changes then help it improve further.

The key word is durable. A model that catches its own mistake inside one chat is not it. An agent that, after a run of tasks, changes its own scaffolding, its skill library, its data-selection rules or even the way it validates new versions — that is closer.

The authors suggest looking not at a single algorithm but at the improvement loop:

🟠 what the system changes

🟠 who decides how to change it

🟠 what persists between iterations

🟠 whether an update feeds the next round of improvements

That framing matters, because three different things are easy to confuse today:

🟣 improving an answer within one session

🟣 improving the system between tasks

🟣 improving the improvement mechanism itself

It is the third rung the authors care about most.

Five levels of autonomy: from executing a given improvement to changing the mechanism of future improvements.

Five levels: from "follow the recipe" to "improve how you improve"

The authors split progress into five levels. This is the most useful part of the whole work.

Level 1: the AI can execute a given improvement procedure. A human still decides what to fix, how to fix it and by what criteria. The AI only executes, at scale — labeling data, filtering a corpus, running tests, making routine edits.

Level 2: the AI picks the improvement strategy itself. The goal and the criteria stay external, but the system decides where to dig next: which prompt to rewrite, which part of the agent scaffolding to change, which experiment to run.

Level 3: the AI decides what experience it needs next. It doesn't just learn from the data it was handed; it shapes its own next batch of practice — generating tasks, choosing exercises, building a curriculum around its own weak spots.

Level 4: the AI adapts in production. Experience from live deployment flows into memory, skills, code or scaffolding, and then shapes future tasks. This is no longer a training sandbox but a live environment.

Level 5: the AI changes the improvement mechanism itself. Not only does it solve tasks better, it rewrites the research policy, the verifier module, the candidate-selection procedure or the way the next improvement is generated.

How the self-improvement loop widens: at each level the AI takes over one more part of the process.

Compressed to a single thought, progress looks like this:

🟠 first the AI makes the improvements it is told to make

🟠 then it picks which improvements to try

🟠 then it picks what to learn from

🟠 then it learns in production

🟠 then it rewrites the improvement machinery itself

Why this matters

The authors start from a problem: building frontier models is becoming too expensive and too hard.

It is not only model sizes that are growing. Everything around them grows too:

🟣 the amount of compute

🟣 the number of experiments

🟣 the cost of data

🟣 the cost of verification

🟣 the complexity of the tooling

🟣 the volume of manual engineering work

Even when a pipeline already has plenty of automation, humans still decide the most expensive questions: what to improve next, how to check the result, which experience counts as useful, what can carry over into the next version.

The authors say it plainly: the bottleneck is no longer only training the model. The bottleneck is the entire improvement loop.

Hence the interest in self-improving systems. If an AI can take over more and more of that loop, it won't just give more useful answers — it will accelerate its own development.

LLM progress is uneven

One of the most interesting parts of the work is an honest attempt to measure where current models are already near the ceiling and where the ceiling is still a long way off.

The authors collect results across domains and normalize them with their own metric, a closed-gap index. The idea is simple: instead of comparing raw scores from different benchmarks, ask how much of the available headroom to a perfect score the models have already taken.

The picture that emerges is lopsided.

Progress trajectories by domain: interactive and tool-use tasks have noticeably more headroom left.

By 2026 models have come especially far on things like advanced math and science exams. But where the work demands tools, long multi-step action, handling environment state and accumulating context, the gap is much wider.

The short version:

🟠 math and science tests are much closer to the ceiling

🟠 coding tasks are improving, but more slowly

🟠 search and terminal agents lag

🟠 tool-using agents remain among the weakest

That is what brings the authors to the paper's main point: recursive self-improvement is needed most where the loop is long, the state is heavy and verification is expensive. Which means software engineering, robotics, scientific research, medicine, real work environments.

What real systems can already do

The paper is full of concrete examples from academia and industry.

At level 1 there are plenty of systems already. They clean data automatically, synthesize training examples, optimize code, deploy models, run predefined verification pipelines. This is working automation.

Level 2 is where it gets interesting. Here the AI itself chooses which prompt, which agent module or which part of the training process to change. In coding tasks it shows up clearly: the agent analyzes execution traces, finds a weak spot in its own scaffolding and proposes a fix that then passes the tests.

At level 3, systems appear that select their own next experience. For example:

🟣 generating tasks near the current edge of their ability

🟣 running self-play against themselves

🟣 choosing which skill to train next

🟣 building a curriculum out of their own failures

This matters, because choosing the next experience is often worth more than another round of fine-tuning on old data.

Level 4 is about the memory, skills and rules an agent accumulates during real work. It can save a procedure that worked, a new utility, a rule for routing around a bug, a useful fragment of a plan — and use it again later. A lot of the work here is going into skill libraries, memory and updatable scaffolding.

At level 5 there are few examples so far, and nearly all of them are narrow. The closest case is a system that changes not only its task but its research policy or the evaluator that decides what counts as an improvement.

From adaptation inside an environment to meta-improvement: at the top level, what changes is the process of future improvement itself.

The hardest parts: verification, transfer, degradation

The authors keep coming back to the same idea: self-improvement is easy to simulate and hard to prove.

A system can post better numbers for three reasons:

🟠 it genuinely got better

🟠 it fit itself more tightly to one particular evaluator

🟠 it was simply given more attempts, more compute and more search

Which leaves three large risks.

The first is unsafe inheritance. Once a bad update becomes part of the system's persistent state, it can spoil the rounds that follow.

The second is false autonomy. Sometimes it looks as if the AI is improving itself, when in fact the decisions that matter most are still hard-wired from outside.

The third is verification breaking down. A system that interacts with its evaluator too often can learn to game that evaluator rather than actually get better.

For level 5 this is particularly awkward. If you are also changing the evaluator, it becomes hard to tell what really happened: did the agent grow, or did the ruler move?

Why coding became the main laboratory

Of all the application areas, the paper singles out software engineering. The reason is simple: there, both the product and the agent itself are executable code. It can be changed, tested, rolled back, compared version against version.

So coding tasks are where the self-improvement loop is easiest to see in action today:

🟣 the agent solves a task in a repository

🟣 gets feedback through tests and traces

🟣 changes its scaffolding, its tools or its strategy

🟣 persists the change

🟣 uses the new version on the next task

That doesn't mean coding is solved. Quite the opposite: the authors show that even here full recursive self-improvement has not been demonstrated. But this is the easiest place to build reproducible loops, measure what an improvement costs and catch regressions.

What industry shows

Another strength of the work is a large map of industry practice. Plenty of systems out there are assembling the future self-improvement loop piece by piece.

Some companies are building data pipelines where the agent structures, checks and corrects the data over successive iterations. There are systems where production failures turn into new rules for quality checks. There are approaches where a coding agent keeps not just a text memory but a whole bank of experience: code, logs, evaluations, run traces.

The Theseus industrial loop: environment, data and model improve together and feed the next round.

One motif repeats across all of these cases: self-improvement is almost never fully autonomous. Humans still hold the goals, the safety constraints, the protected verification and the final say over changes, from outside the loop.

And that is the most honest conclusion in the whole paper. The conversation is about more and more parts of the improvement loop already passing under the control of the systems themselves.

The bottom line

If you want to understand where the real frontier of AI progress runs right now, don't look only at knowledge benchmarks and reasoning. Look at whether a system can accumulate experience, change itself between tasks and use those changes in the next round of improvement.

The main conclusions fit in a short list:

🟠 recursive self-improvement is not one algorithm but a loop

🟠 today's LLMs have the most headroom left in long interactive tasks

🟠 the clearest progress today is at levels 1 and 2, partly at 3 and 4

🟠 level 5 exists so far only as narrow prototypes and industrial groundwork

🟠 the main problem now is not only training the model but building a reliable improvement loop around it

Which points to something simple: the next big jump in AI may come not from another increase in model size, but from the development of models, agents and their scaffolding becoming a closed, cumulative process. When every iteration leaves behind not only a better answer, but a better machine for the next iteration.