Useful projects, limited reach
Some companies already have AI projects with measurable returns. One enterprise software company replaced about 250 sales dashboards used by 4,000 employees with a single agent, which handles up to 30,000 prompts a week. One of the retired dashboards had cost the company $3 million a year. Another company used an agent to optimize tax payments, saving hundreds of millions of dollars.
A healthcare company used generative AI to summarize tens of millions of customer-support calls and identify messages that persuaded people to switch to lower-cost treatment. Nelson said one division increased its margin by $170 million, while another $200 million in savings went to customers. At the author’s company, agents helped review data-handling requirements across every country; the time saved and expanded employee capacity were assessed after deployment.
But when Fortune 500 executives were asked about their own companies, the common answer was a handful of live projects with measurable benefits. AI had generally not made its way into core products or strategy.
The distinction matters: a successful project is not the same thing as a company changing how it works. Abacum CEO Julio Martínez argues that automating finance tasks does not automatically make the finance function more strategic. AI can speed up existing processes, but companies also need to decide which processes to stop and what finance staff should do instead.
Scale is not the same as change
One enterprise software company created about 18,000 agents in a quarter, expecting only a few hundred to prove genuinely useful, Nelson said. It substantially exceeded its cloud-services budget and had planned for the overrun, on the view that broad experimentation would teach the organization faster than tightly controlled pilots.
That might be disciplined experimentation, or it might be waste. Nelson said some large companies have begun limiting spending. Either way, the figure shows how cheap it can be to multiply agents—and how quickly that can turn into a cloud bill.
The harder problem is deciding what work should exist at all. At one healthcare company, a process had about 200 steps. Managers proposed automating three. The executive’s response, Nelson recalled, was that AI made most of the steps unnecessary. Automating an inefficient process does not, by itself, change the company.
Agents can be created in seconds; people cannot. The technology improves week by week, but adoption depends on employees’ behavior, confidence and skills.
The work shifts to people
Companies are trying to address that human bottleneck in different ways:
The executives did not think the technology department could handle the transition alone. Nelson recalled one leader telling IT staff tasked with introducing AI across the company: if you are the only people involved, it is time to prepare your résumés. The task, Nelson argued, is less an IT director’s AI rollout plan than a change to the company’s operating model—making it a strategy issue for the CEO.
Nelson said the executives repeatedly made a related point about early-career workers: as AI takes on narrow tasks, companies will need generalists—architects, flexible thinkers, designers and evaluators. More employees may end up directing agents, checking their work and redesigning the processes around them.
I think the 18,000-agent experiment is revealing less about the value of mass experimentation than about the mismatch between the speed of software and the pace of organizational change. Nelson said he did not hear convincing answers about how companies would train employees for the new work. The agents can scale quickly; the workforce that has to supervise them cannot. That leaves the central question not how many agents a company can create, but whether its leaders are willing to change the work around them.
Daily AI news
Every day we pick what actually matters in AI and explain it plainly — no hype, no filler. Subscribe if you want to follow where the industry is going.
Only what matters — every day
Follow on X