Making tests fight the patch lifts SWE-bench Verified to 79.4%

LLM-driven bug fixing stopped being exotic a while ago: a model can read code, propose edits, and run the test suite itself. In real repositories, though, the whole thing runs into an awkward detail. There is often no good way to check whether the bug is actually gone. When tests are missing, weak, or simply don't cover the case in question, the system can produce a patch that turns the suite green without touching the underlying problem. The result is cosmetic repair rather than a fix.

The authors of InfCode: Adversarial Iterative Refinement of Tests and Patches for Reliable Software Issue Resolution take a different angle: reliability comes down to the quality of the verification signal. If the tests don't catch the defect, even a very capable LLM will optimize toward the wrong target. So InfCode makes tests and patches compete, each one forcing the other to get stronger.

The idea: patch and test in one loop, pulling against each other

InfCode runs a multi-agent system with three roles. The first agent writes and strengthens tests, the second proposes code fixes, and the third acts as an editor and picks the most reliable option out of several candidates.

The key mechanism is iterative refinement. First the testing agent tries to reproduce the bug: it adds or edits a test so the failure actually shows up. Then the coding agent writes a patch to make the tests pass. Once the patch clears the current set of checks, the adversary comes back and tries to make the tests stricter: add edge cases, close loopholes, tighten the match to the issue description. The patch has to adapt again. That way the system never settles for the first patch that works and keeps pushing toward something sturdier.

All of this happens inside a Docker container: the agents genuinely explore the repository, run commands, edit files and execute tests. That matters, because many bugs are not about a single function but about how modules, configs and dependencies interact.

Why it's still easy to get it wrong without patch selection

Even with a healthy test ↔ code loop you can end up with several solutions: some not quite robust, some overfitted to one specific test, some passing by accident because coverage is thin. So InfCode adds a separate selection stage: the Selector agent collects all the resulting candidates and grades them more strictly, looking at execution, compatibility with the repository, and signs of over-optimization. Judging by the authors' experiments, this selection step is what contributes most visibly to the final quality.

What the SWE-bench experiments showed

To measure progress honestly, the authors use benchmarks from the SWE-bench family. There a task is stated as a real issue from a repository, and success is decided by whether the official check of the fix passes.

On SWE-bench Lite (300 tasks), InfCode with DeepSeek-V3 posts the best result among the strongest approaches: about 40% of tasks solved. One interesting detail is that InfCode solves a lot of "unique" tasks that other methods do not.

On the stricter SWE-bench Verified the authors run InfCode with Claude 4.5 Sonnet and get 79.4%, which was first place on the leaderboard as of the report.

Expectation vs reality

The authors also look at how the agents use their tools: bash is called most often (running tests and commands), then the editor, then search. Bash can fail on odd or missing commands, and the editor on the strict exact-match requirement when replacing a fragment. Failure rates are low overall, but they show where the system trips most often.

What this adds up to

InfCode makes it clear that in code fixing an LLM is limited not only by its ability to write patches, but by the quality of the feedback it gets. Weak tests can produce the illusion of success. The value of the authors' approach is that it makes tests an active participant and turns verification into a dynamic game: the patch gets better precisely because the tests won't let it relax. And the final selection step keeps the first green signal from being mistaken for the truth.

The limitations are stated honestly too: the test agent can get carried away and start generating checks that formally break the current patch while matching the intent of the issue less well. It's a subtle point. Tests have to be strengthened in a way that keeps them faithful to the task as stated.

💾 Code