LLMs cut the hand-written rules out of data cleaning, but not the cost

Every company has the same problem: the data sitting in its tables and databases is arranged in a way that makes it hard to use. Formats vary, values contradict each other, some fields are empty, and different sources call the same thing by different names. Analysts end up spending weeks preparing datasets, and the business loses money to errors and delays. The authors of the survey Can LLMs Clean Up Your Mess? treat this as a foundational problem: until data is application-ready, no report, data mart or ML pipeline will be reliable.

What LLMs changed, and why it matters

Data preparation used to rest on rules, regular expressions and domain scripts: that works, but it transfers badly from one dataset to the next and it takes expensive specialists. The authors write that rigid pipelines are increasingly giving way to prompts, context and even multi-agent systems that plan their own steps, call tools and check the result.

The authors tie the surge of interest to three things: demand for application-ready data is growing, LLMs themselves have become markedly stronger, and platforms and infrastructure have made it easier to assemble smart workflows — up to and including agentic ones.

Three broad classes of tasks

To keep a scattered literature manageable, the authors sort the approaches into three directions.

The first is cleaning. That covers standardization (bringing every date to a single format, say), error handling (finding and fixing typos, nonsensical values, violated dependencies) and filling in missing values. LLMs earn their place here because they understand meaning: they can work out that “7th April 2021” and “04/07/21” are the same date, and they do not need a huge body of hand-written rules.

The second is integration: entity matching and schema matching. This is where you need to establish that two rows in different systems describe the same customer, or that the price column in one source corresponds to cost in another. The survey shows LLMs helping precisely where classical methods fall down: synonyms, abbreviations, ambiguous names, thin metadata.

The third is enrichment. That means annotation (assigning types and meanings to columns, linking values to entities in a knowledge base) and profiling (writing readable table descriptions, grouping columns, finding relationships and useful neighbouring datasets). Here LLMs often act as a translator between raw structure and a human understanding of the data.

How LLMs actually get used: from prompt to agent

The same set of strategies turns up inside every task. The simplest is direct prompting: serialize the rows or a column's values, add the output rules, and ask the model to fix or match. Fast and flexible, but costly in tokens and poor at scaling.

The second route is to have the LLM generate executable code or a cleaning function instead of doing the work itself. The transformation gets written once and then runs cheaply and repeatably. The catch is the risk of faulty code, which means the output has to be verified.

The third option is multi-agent and tool-based approaches: the model plans the steps, calls external tools (profilers, search systems, rule engines), takes the feedback and refines its answer. This comes closest to real production, but it adds maintenance complexity, latency and new points of failure.

What works well and what doesn't

The authors describe the main strengths of LLMs in data preparation in very down-to-earth terms: semantic understanding, fewer hand-written rules, better transfer across domains, and the ability to work on tables alongside text descriptions and documentation.

But the researchers do not romanticize the picture. First, cost: direct prompting over large volumes of data gets expensive fast. Second, hallucinations do not go away — even in agentic systems with checks built in, a model can confidently invent a fact that does not exist or push a value to the wrong correction. And finally the sore subject of evaluation: the methods keep multiplying, while rigorous shared protocols and comparable metrics are still thin on the ground, especially for tasks with open-ended text output such as profiling.

Where this is heading

The authors stress that what is needed now is more reliable agentic pipelines and rigorous evaluation. The likely winners are hybrid systems, where routine operations run on cheaper models and deterministic tools and LLMs are brought in selectively — where meaning and context genuinely matter. The closer such systems get to provable verification, the faster they will move out of papers and into everyday practice.

💾 Code