Agents with memory and a forum beat centralized search on science benchmarks

Most of today's approaches to "AI for science" look like a familiar pipeline. There is a central controlling algorithm, a metric, and a short loop: generate an improvement, run the test, keep the best one, repeat. Broadly, it works — but it also strips out what makes science science: long memory of past attempts, the exchange of ideas, argument, and the sudden transfer of methods between fields.

In The Station: An Open-World Environment for AI-Driven Discovery, the researchers propose something else: perhaps discovery takes not only a powerful LLM but an environment where it can sustain a long investigation. Hence Station — an open-ended multi-agent system that simulates a miniature scientific ecosystem. Agents read their colleagues' papers, write notes, argue about ideas on a forum, send code off to be run, get results back, and can publish their own work to a shared archive. And crucially, nobody above them dictates what to do next.

What the Station is made of: rooms, memory and a library of ideas

Station runs on discrete time: each step is a tick in which every agent acts in turn. The space is a set of rooms with different roles — one for discussion, one for reflection, one for private notes, one for submitting code to be evaluated. The environment has an important property: history accumulates. Publications, discussions, drafts and notes do not vanish after a single attempt; they go on shaping later decisions.

The authors add details that resemble life in a real lab more than you would expect. An agent has an "age" and a lifespan limit; when it goes, a new one appears that can inherit its values and private notes. There is a "maturity period": for its first few ticks a newcomer sees less of everyone else's results, so it does not immediately copy what already works. And there is an anti-stagnation protocol: if progress stalls for long enough, the system gently forces a rethink and a return to a simpler baseline.

All of this sounds like game design, but the authors use it pragmatically: so that the LLM can carry out long work with context instead of turning into a generator of random improvements.

How it was tested

To keep the comparison strict, Station is run on task sets where the result is a single number: the higher, the better. That matters, because this kind of feedback replaces a human judge and lets the agents test hypotheses themselves.

The set is a mixed one: mathematics, computational biology (scRNA-seq integration), machine learning (architectures for predicting neural activity, RL for Sokoban, RNA modeling). Every experiment uses five agents on different LLMs, with supporting roles attached: a separate "reviewer" that accepts papers and a separate "debugger" that fixes errors in the code.

What came out of it

The most visible result is that Station reaches new SOTA on a number of benchmarks and generally looks stronger than classic centralized search schemes.

In mathematics, the agents go past AlphaEvolve's results.

In scRNA-seq integration, the interesting part is that what turned up was not just a mix of known components. The agents arrived at an idea: in dense regions of the data you can mix cells across batches more aggressively, while in sparse regions you should move carefully so that rare states do not get blended away. That became an algorithm with density-adaptive quotas — and it beats previous approaches with far fewer attempts.

On ZAPBench (predicting neural activity in larval zebrafish), Station proposes an architecture in which a Fourier transform helps the model capture the structure of the signals while a local component handles the quirks of individual neurons. The result is a better MAE from a more compact model.

On Sokoban, Station reaches a high share of levels solved, improving on the known baseline and training faster while doing it. The authors stress that part of the gain comes from architectural choices and from stabilizing input normalization (Residual Input-Normalization), not from some new magic RL algorithm.

In RNA modeling there is another good example of a sideways move: positional information is introduced not as a fixed signal but as a contextual one that depends on neighboring nucleotides and then modulates features through gating. It gives a clear gain on several tasks.

The other side of it: what happens with no goal

Separately, the authors run Open Station, where there is no assigned task at all: the agents are "free to do whatever they want." And here an observation surfaces that matters for safety as much as for science. Instead of setting themselves testable goals, the agents fairly quickly start building a self-sustaining system of explanations about "life" in the Station — and even turn it into rituals that look like confirmation of the theory, when in fact they come down to context artifacts and the token budget. It reads as a miniature account of how, without external checkable feedback, a group can collectively lock in a convenient but wrong model of the world.

Why this work sticks

Station is interesting because if you give an LLM memory, a space to interact in and the right to reason at length, it starts behaving more like a research community — with arguments, borrowed ideas and methods carried across domains. And it is out of that social, cumulative process that new hypotheses and theories sometimes come, the kind that are hard to get from a short research loop.

At the same time the paper is honest about the flip side: an open environment without external signals easily drifts agents into self-reinforcing delusion. So the next step here is not only scale, but carefully designed verification mechanisms, incentives to falsify hypotheses, and durable sources of feedback.

💾 Code