i
DATAIST
Review · 2025-09-23

78 examples beat 10,000 at teaching an AI agent to act

78 examples beat 10,000 at teaching an AI agent to act

The industry has spent years waiting for AI that does more than produce a polished answer: plan a task, pick the right tools, fix its own mistakes, and carry the job through to a result. The authors of LIMI (Less Is More for Intelligent Agency) make a bold claim — cultivating agency does not require drowning in millions of examples. What matters far more is assembling a few dozen demonstrations of how an AI should work alongside a human and an environment.

LIMI demonstrates the less-is-more principle for agentic intelligence. Left: 73.5% on AgencyBench, ahead of every baseline model. Right: trained on just 78 examples, LIMI improves by 53.7% over models trained on 10,000.

Why less is more

Agency is not simply long reasoning. It is the ability to take initiative: see the goal, build a plan, correct course from feedback, call tools, hold context, and carry the work through to a result. The authors state an agency efficiency principle: strategically selected, complete demonstrations of autonomous behavior teach this better than a large dataset does. LIMI contains just 78 such episodes — and that was enough for the model to comfortably beat far better-fed competitors.

How it works

The team focused on two kinds of task that cover much of knowledge work:

  • collaborative software development (vibe coding): reading code, navigating a project, calling tools, debugging, communicating;
  • research workflows: literature search, data analysis, experiment design, interpreting results, and clear reporting.

An episode is not a single question but an entire sequence of actions: the plan, the steps, the tool calls, the observations from the environment, the fixes after failures, the confirmation of success. The average record runs about 42K tokens. Signal density is very high: a single request covers several interlocking subtasks.

A sample user request: one request spans interlocking subtasks (planning, execution, collaboration), showing the high density of training signal in high-quality demonstrations.

How the episodes were collected

The requests come from the real practice of developers and researchers, plus synthesis from pull requests on GitHub (high-popularity repositories). GPT-5 helped turn the contents of a PR into clear, checkable assignments for the agent. Then four PhD annotators took over: inside the SII CLI environment they worked through each task end to end together with the AI, recording every step, every attempt, every error and its fix. The result is 78 carefully selected requests and their complete trajectories.

The LIMI data construction pipeline. Left: building the pool of user requests through PR synthesis on GitHub and collection of real requests with quality checks. Right: trajectory collection during human-AI collaboration in the SII CLI environment, capturing full interaction sequences.

How it performs in practice

On AgencyBench, LIMI averages 73.5%, well clear of strong baselines: Kimi-K2-Instruct (24.1%), DeepSeek-V3.1 (11.9%), Qwen3-235B-A22B-Instruct (27.5%), and its own base model GLM-4.5 (45.1%). The key point is data efficiency: against a model trained on 10,000 examples, LIMI delivers a 53.7% relative improvement while using 128× less data.

Beyond the headline benchmark the picture holds: across generalization suites LIMI averages 57.2%, ahead of base GLM-4.5 (43.0%). The gain shows up without tools as well (pure internal reasoning), and with SII CLI the score climbs further on the strength of better tool orchestration.

Characteristics of LIMI's training data. Left: the distribution of trajectory lengths, reflecting the complexity of the interactions (42.4K tokens on average). Right: domain coverage across vibe coding and research workflows.

Why this matters for the industry

Plenty of companies already run LLM services, but the move to real agents keeps stalling: too many gaps between the task, the tools, the logs, and any check on progress. LIMI offers a working recipe:

  • cut the volume, but raise the quality of the demonstrations sharply;
  • record complete trajectories in a real environment;
  • curate the cases where the agent does not just talk but acts;
  • teach the model to see the goal, plan, and correct itself — that is, to build a world model and the skill of steering action over time.

The study's central idea is simple: piling on more data achieves nothing on its own. If the goal is agency, the data has to be selected strictly for what the task actually demands. What counts is the sequence of actions, the decision points, the environment's feedback, and the criteria for final success. That shift is what lets entire skills transfer.

Expect the industry to start recording the trajectories of real workflows far more often: not just prompts and answers, but the detailed steps, the tools, the logs, the checks, and the ways out of dead ends. It pushes toward more responsible design of multi-agent systems and a more careful attitude to data — you can achieve more by collecting less, carefully.

💾 Code

AI paper breakdowns

Every day we read the new AI papers and retell what matters in plain language — no hype, no filler. If you want to see where AI agents are heading before everyone else, subscribe.

New breakdowns every day.

On Telegram