SkillOpt trains an agent's skill file, not its weights, and leads in all 52 comparisons

For years the default answer in AI has been the same: if an agent fails, rewrite the prompt, fine-tune the model, or wait for a stronger one. The authors of SkillOpt take a different line — leave the weights alone and train how the agent works instead, as a compact text "skill".

When capability isn't enough: teaching an agent working habits

The setup sounds almost too simple: a frozen LLM, a document of instructions for one domain, and a separate optimizer model that reads successful and failed trajectories and then carefully edits that document. Underneath the simplicity is a real idea: if the skill is an external layer of adaptation, it can be optimized almost the way network parameters are — with a learning rate, validation, a buffer of failed updates and protection against degradation.

And the results suggest this is more than a neat metaphor. In 52 out of 52 comparisons the method is either best or tied for first. In a field where a lot of the work amounts to "we rewrote the prompt and it seemed to help," that is a loud claim.

What SkillOpt actually optimizes

The authors start from a practical problem: in real agent systems, adaptation is not only about what the model knows but about its procedures. How to search for information, when to call tools, how to check a result, what format to answer in, how to avoid looping in a multi-step task. All of that is easier to keep outside the weights, in a portable text artifact — a skill.

In SkillOpt the skill becomes the thing being trained. The target model stays frozen; it simply solves tasks using the current version of the skill. A separate, stronger model then takes over: it analyzes the trajectories, identifies recurring errors and proposes bounded edits of three kinds — add, delete or replace passages of text.

The key point: an edit is accepted not because it sounds smart but only if it improves accuracy on a held-out validation set. If it doesn't, the edit is rejected and kept as a negative example for later iterations.

That matters, because most self-editing agent systems share one failure mode: they happily produce plausible explanations of their own mistakes, and those explanations often do nothing for behavior. SkillOpt turns self-reflection into a stricter loop — propose, test, accept or discard.

Why this matters

The method has several strong practical consequences.

First, no fine-tuning is required. For closed models that is often the only realistic route to adaptation. For open ones it is cheaper and faster.

Second, the output is a small text file you can read, review, move between environments and even edit by hand. As model behavior becomes harder and harder to inspect, that is a rare virtue.

Third, SkillOpt works not only in plain chat but inside agent loops with tools — including environments like Codex and Claude Code. This is not prompt polishing for toy problems; it is a procedure that helps an agent act better in working setups.

Finally, the paper touches a broader question: can you train not only model parameters but external procedural state? If you can, that is an intriguing alternative to the race for ever more expensive models.

How the optimization loop works

The methodology is arguably more interesting than the numbers. The authors borrow the logic of deep learning outright and move it into text space.

At each step the target model runs a batch of tasks with the current skill. Trajectories are collected: messages, tool calls, intermediate observations, final answers, signals from the verifier. The optimizer then splits the examples into successes and failures, analyzes them in mini-batches and proposes local edits.

Those edits are then merged and ranked. Not all of them get applied: there is a text analogue of the learning rate — an edit budget. The reasoning is simple. Let the model rewrite the whole skill every round and it will overwrite rules that worked, overfit to the latest failures and swing from one extreme to the other. Bounded edits preserve continuity.

Two more stability mechanisms complete the picture.

The first is a buffer of rejected edits. When a change makes results worse, the system doesn't just forget it; it feeds the change back as negative signal. That makes it less likely to go in circles.

The second is a slow cross-epoch update. At the end of an epoch the method compares how the skill behaves on the same tasks before and after the run of local edits, then writes the longer-lived guidance into a protected section. Local steps cannot touch that section. In effect, it separates fast tactical fixes from strategic lessons.

This is where SkillOpt stops being another way to auto-generate a prompt and starts to look like a disciplined training loop.

What the experiments show

The experimental section is large: six benchmarks, seven target models, three execution modes. The tasks cover question answering, spreadsheets, office documents, visual documents, math and an embodied environment along the lines of ALFWorld.

The authors state the headline result bluntly: best or tied for best in 52 out of 52 cells.

On GPT–5.5 in direct chat, the average gain over running with no skill was +23.5 percentage points. Inside the Codex loop it was +24.8; inside Claude Code, +19.1.

The most striking cases are tasks where the failure is not "forgot a fact" but "broke the procedure." For example:

In other words, the method is strongest where the agent needs working discipline: inspect the structure of the file first, then extract the data correctly, then return the answer in a strict format.

And it isn't only the top models that gain. Weaker versions and open models such as Qwen pick up a noticeable boost too. That is an important observation: a compact text skill can act as external procedural memory, covering what a small model lacks in its weights.

Why the results look credible

Papers like this always invite suspicion: maybe the gain comes from a longer prompt, a stronger teacher model or a lucky hyperparameter setting. The authors try to close that off with several rounds of checks.

First, they compare SkillOpt not only against having no skill but against hand-written skills, skills generated in a single pass, and competing methods such as TextGrad, GEPA, Trace2Skill and EvoSkill. SkillOpt is consistently ahead.

Second, there are ablations. Remove the bounded edit budget and results get worse. Remove the buffer of rejected edits and they get worse as well. Remove the slow cross-epoch update and some tasks degrade sharply, spreadsheet tasks most of all.

Third, the authors show the per-epoch dynamics: the checkpoint that validation picks usually lines up with the best test accuracy. That does not prove the method generalizes perfectly, but it at least shows the validation filter is not walking the system into an obvious dead end.

This is probably one of the most convincing parts of the paper. The authors don't just show a metric going up; they try to show which constraints keep it from falling apart.

Transferability: the skill as an artifact

The transfer experiments are interesting in their own right. A skill optimized on one model can be run on another; one trained in one agent environment can be moved to a different one; and in some cases it can even be applied to a neighboring benchmark.

A spreadsheet skill trained in the Codex environment, for instance, transfers well to Claude Code. There is cross-model transfer too: a skill trained on GPT–5.4 also helps more compact variants.

This may be the most practical takeaway in the paper. If a skill can be trained once, checked, saved as text and then reused, it becomes an engineering artifact in its own right. Not an ephemeral prompt living in a researcher's notebook, but something closer to a library of procedural rules.

Transfer doesn't always deliver the same gain as training in place, of course, but almost every shift is positive. That is already a good sign that SkillOpt picks up more than the local patterns of one dataset.

What gets learned: habits, not facts

One of the more rewarding parts of the paper is the qualitative analysis of the final skills. They are small — usually a few hundred to a couple of thousand tokens. More to the point, they consist of procedural rules rather than patches for individual examples.

For spreadsheets, the skill learns to inspect the workbook's structure and formulas first instead of trusting a preview. For document tasks, to anchor the question to a specific row, heading or field before extracting the answer. For math, to tell the strongest statement apart from one that is merely correct but weaker. For embodied tasks, to keep a memory of places already visited and not repeat useless actions.

That distinction matters. SkillOpt does appear to train behavioral invariants rather than simply rephrasing the instructions a little more elegantly.

One more detail: large improvements often take only 1–4 accepted edits. The system's real work is not rewriting the whole skill over and over, but picking the handful of useful proposals out of many.

Limits, and the skepticism worth keeping

Strong as the results are, this is not a silver bullet.

First, the method needs reliable quality signals: verifiers, exact metrics, executable checks. For open-ended creative work, where success is multidimensional and subjective, this loop will be much harder to build.

Second, training a skill isn't free. There are no extra calls at inference time, but offline optimization takes a substantial number of runs and calls to the optimizer model.

Third, SkillOpt so far works with one compact skill for one domain. Genuinely heterogeneous workloads would probably need more than a single document: a library of skills, routing between them and something like composition of procedures.

Finally, some of the rules may still be tied to the specific format of a benchmark. The transfer experiments offset that in part, but the question of broad external validity stays open.

The takeaway

SkillOpt is one of those papers that promises no magic but offers a very strong engineering idea: if you can't or don't want to change a model's weights, then change and train its external working instructions with the same discipline you would apply to parameters.

The paper's strength is not only in the numbers, impressive as they are. It is in the framing. The authors treat the skill as a genuine object of optimization: a bounded step, validation, protection against degradation and accumulated negative feedback. Because of that, self-improvement through text stops looking like a pile of heuristics and starts to resemble a coherent training process.

For the industry that matters. We have gotten used to thinking that agent progress means either new weights or new tools. SkillOpt points to a third route: a trainable procedural layer on top of a frozen model. If this line of work pans out, we get not just a way to improve agents without fine-tuning, but a new class of portable, checkable artifacts — text skills that can be stored, versioned and developed almost like code.