Coding agents hit 80% on single tasks, 38% over a whole project

The end of software engineering?

For half a century the industry has run on a simple rule: a person breaks a task into parts, writes code, then fixes and rewrites it as things change. Zhenfeng Cao's paper argues something different: the era in which code was the main carrier of logic is ending. What moves to the front is the AI agent — a system where the LLM does not merely help write a program but reasons on its own, plans steps, calls tools, generates code as it goes and throws that code away the moment it is no longer needed.

The old chain looked like this: a human or an AI writes software, the software delivers the result. Increasingly a different pattern applies: the agent receives an intent and takes you straight to the result. Code in that scheme is a temporary tool.

This changes more than the developer's tooling. It changes the unit of delivery itself. Instead of “we shipped you a program” — “we solved your problem”.

What actually changes

The author separates two worlds.

In traditional software engineering the logic is wired into the source code in advance. Every important decision has to be anticipated before launch: how the system handles input, what it does in rare cases, how it behaves when something fails. When requirements change, a person goes back into the code and edits the system by hand.

In the agentic scheme the center of gravity moves into the model. It takes the task, looks at the current state, pulls the context it needs out of memory, builds a plan, calls tools, writes whatever fragments of code are required, runs them, checks the result and moves on.

Put another way, in the old world the code stored the solution; in the new one the solution is produced at runtime.

Almost everything around it changes role as a result:

🟠 The main artifact is no longer a repository of static code but an agent with memory, tools and scaffolding.

🟠 The control point is no longer the developer walking the system through a pipeline by hand, but the LLM making local decisions as the task unfolds.

🟠 The human role is no longer “author of every line” but the one who sets the goal, the constraints and the quality bar, and checks the outcome.

🟠 The unit of value is no longer “a finished application” but a result: a bug fixed, a report assembled, a service configured, a ticket closed.

The paper frames this historically. First came local software: the user installed the system, updated it and dealt with the infrastructure. Then SaaS took part of that complexity onto the vendor's side. Now, the author argues, the next step is beginning: complexity moves further still — what is hidden from the user is no longer only the infrastructure but the internal logic of the solution itself.

Why the old approach hits a ceiling

The most interesting part of the paper is its attempt to explain this through growing complexity.

The classic problem of software engineering is old news: the bigger the system, the more connections inside it. A new module affects the old ones. Fixing one bug creates two more. Test suites sprawl. Documentation goes stale. The architecture accumulates compromises.

The author leans on Fred Brooks's old distinction: there is accidental complexity, which better languages, frameworks and tools can reduce, and there is essential complexity, which is built into the problem itself. That one you cannot escape.

The argument runs like this: in traditional software a person has to encode the decision tree explicitly. But the space of possible states and interactions grows too fast, while human capacity to hold all of it in one head barely grows at all. At some point you hit a ceiling not because the team is bad but because of the nature of the problem.

The AI agent promises another route. It does not encode every branch in advance. It builds the solution in place: takes the current subtask, plans a step, calls a tool, checks the result, changes course if it has to. It works not by manually enumerating pre-written logic but through dynamic reasoning at runtime.

In short:

🟣 Traditional software tries to describe the world in code ahead of time.

🟣 An AI agent tries to work the world out as it goes.

🟣 Traditional software handles change badly without a human involved.

🟣 An AI agent can adapt on the fly, given enough context, memory and tools.

This is why the author insists the claim is not that programming gets 10–20% faster, but that the computing model itself changes.

What agentic engineering is

The paper gives this shift a name: agentic engineering. In essence it is a new discipline around designing systems where the work is done not by a single script or a single developer but by one or several AI agents with shared memory, defined roles and observability.

Importantly, this is not only about coding agents that fix a bug in a repository. It is about a wider layer: the agent takes a business goal, translates it into technical steps itself, coordinates tools and people, and returns a result.

The author sets out the architecture of such an agent separately. It looks fairly standard by the standards of today's market:

🟠 A perception module takes in text, files, sometimes images and other inputs.

🟠 Memory holds facts, past episodes, procedures and working context.

🟠 An action module calls APIs, runs code, queries databases, edits files.

🟠 An LLM as the reasoning core ties all of it into a single decision loop.

In practice the scaffolding around the model starts to matter no less than the model itself. Memory, observability, context shared between agents, a history of decisions, criteria for verification — all of it becomes part of the product.

Hence the new role for the human, which the author describes this way:

🟣 Architect of intent — states the goal and the constraints.

🟣 Agent coordinator — sets up roles, memory and interaction.

🟣 Auditor of the result — checks not just “does it work” but quality, risk and compliance with the rules.

You can already see this in real teams. The most productive engineer of the next two or three years is not necessarily the one who writes the best code by hand. More often it is the one who is best at assembling a working pipeline out of agents, tests, memory and checks.

What the benchmarks show

Here the paper moves from broad claims to a few empirical signals.

The first is SWE-bench Verified, one of the main benchmarks for coding tasks drawn from real GitHub issues. The author cites results for Lingma SWE-GPT 72B: it closes 30.2% of the tasks. That is already close to GPT-4o at 31.8%. More interesting still, the 7B-parameter version solves 18.2% — noticeably less, but still at a useful level.

What that means in practice: a well fine-tuned model built around the development process can be more useful than a merely very large general one. For a business that matters — the most expensive general-purpose option is not always the one you need.

🟠 Lingma SWE-GPT 72B — 30.2% on SWE-bench Verified.

🟠 GPT-4o — 31,8%.

🟠 Lingma SWE-GPT 7B — 18,2%.

🟠 Takeaway — specialization and fine-tuning can matter more than model scale alone.

The second signal is multi-agent coordination. In the LangChain pilots the author cites, coordinated groups of agents cut root-cause analysis time by 93% and saved more than 200 engineering hours in a month. The point here is not that any one agent got smarter. Several specialized agents with shared context can investigate a problem in parallel and check each other's conclusions.

🟠 Root-cause analysis time — down 93%.

🟠 Engineering time saved — more than 200 hours a month.

🟠 Takeaway — coordinating several agents buys a gain of its own, on top of any single model's quality.

The third signal is self-improvement. The example given is Hermes Agent from Nous Research. The idea is simple: after a hard task the agent does not just end the session, it saves the useful procedure as a skill. Later it uses that skill again. If the skill worked badly, the agent fixes it itself. The loop: do the task → extract the procedure → reuse it → spot the weak point → patch it.

That looks less like a one-off exchange with a model and more like accumulating operational experience.

The results in brief:

🟠 Standalone coding tasks are already solved by agents at a level you cannot ignore.

🟠 Coordinating several agents buys a gain of its own, on top of any single model's quality.

🟠 Memory and skill reuse are becoming a serious part of these systems rather than research exotica.

Where it still falls apart

The most honest part of the paper is the discussion of limits. And here the lead role belongs not to SWE-bench but to EvoClaw.

This benchmark tests not one-off edits but the continuous evolution of a software system. The agent has to carry a project through a series of changes, commit after commit, keeping the system coherent — not merely close a single task. And here the picture changes sharply.

The author gives the key number: where frontier systems can clear over 80% on isolated tasks, in sustained continuous work that drops to 38% at best.

That matters.

🟣 Isolated tasks — success above 80%.

🟣 Sustained continuous work — 38% at best.

🟣 The gap — the long horizon remains the main limitation.

Agents are good in tidy demos and short autonomous episodes. In long-running development the familiar old problems start:

🟣 Context drift — the agent stops holding a large system's invariants in its head.

🟣 Error accumulation — a small early mistake drags the next ones along behind it.

🟣 A weak grasp of technical debt — the agent optimizes for local success, not long-term maintainability.

🟣 Incomplete verification — the tests can pass while a semantic error sits inside.

This is exactly where you see that “an agent instead of a program” is not yet ready to replace the long engineering cycle. It can fix a ticket quickly, put together a prototype, write a migration, run an investigation — but steadily carry a large codebase for months, not yet.

What companies can do right now

A practical plan follows from the paper. Do not wait for the fully autonomous future; rebuild the process piece by piece.

If you are building teams and products today, the logic runs roughly like this:

🟠 Pick tasks with a clear success criterion — fixing a defect, generating tests, a migration, root-causing an incident.

🟠 Build verification around the agent — tests, observability, step traces, a record of decisions.

🟠 Give the agent the right to run execution, and the human the right to set the frame and accept the outcome.

🟠 Teach people to work with intent, not only with code — a good system prompt, a set of constraints and quality criteria count for a great deal here.

The author also sketches a four-stage roadmap: from assistant tools, through autonomous execution of a single task, to multi-agent teams and on to self-modifying ecosystems. The timelines there are debatable, of course. But the direction looks plausible: first agents assist, then they close discrete pieces of work, then they start coordinating among themselves.

The bottom line

Code is ceasing to be the only place a system's logic lives. More and more of that logic moves into a loop: understand the task, make a plan, call a tool, check the result, adjust the action. That loop is the core of agentic engineering.

The main shift is from shipping programs to shipping results. For the user that is simpler. For companies it changes the product model. For engineers, their role on the team.

The near future is not fully autonomous development but a hybrid mode. The AI agent runs execution; the human sets the goal and the constraints and decides the contested calls.

The main barrier right now is the long horizon. Agents already handle one-off tasks. Continuous evolution of large systems, not yet. Which means the entire market runs into memory, verification, observability and multi-agent coordination.

Short version: software engineering is not disappearing. What is disappearing is the idea that the code is the finished product. Increasingly the product is an agent's ability to get you to the result you need.