AI inference creates a storage bill that outlasts each request

Inference leaves a longer tail

Training is a resource-intensive project with a defined start and finish. Inference runs continuously, and each request can leave data behind.

McKinsey forecasts that demand for inference will grow by an average of 35% a year through 2030, compared with 22% for training. By the end of the decade, inference is expected to account for more than 40% of total data-center workloads. In a global IDC survey supported by WD and covering 763 IT and business leaders, nearly 95% said generative AI had increased their data volumes over the past 12 months; about 61% reported growth of 25% or more.

The server-side footprint can rise well beyond the video example. For longer, higher-resolution videos, it may reach hundreds of gigabytes per clip. A senior product manager at a cloud provider for large enterprises said extra storage helps keep GPUs from sitting idle while they wait for data. But storage costs growing 25% a year can outpace a more predictable compute budget.

Compute demand can ebb with training cycles, product launches and hardware refreshes. Stored data tends not to follow it back down. Model logs, synthetic data and inference outputs continue to accumulate, and companies keep them for longer.

Archives are changing, too. Data once put away and rarely touched can become input for future AI tasks. In the WD and IDC study, 96% of respondents expected to need faster access to archived data for inference or retrieval-augmented generation (RAG). HDDs can provide a high-capacity, low-cost-per-terabyte foundation for data that must be retained and brought back into use.

The budget can miss the lasting cost

For finance and IT leaders, the risk is a budget built around compute while storage appears as a minor add-on. A model waiting for data can be as costly as one short of GPU capacity, but storage bottlenecks may surface only later as operating expenses and retrieval delays—not as a capital line item alongside accelerators.

The pattern applies across different workloads. A retailer may generate product videos for a catalogue; an insurer may process claims documents through a RAG pipeline. Compute costs can be forecast per generation or query. Storage costs build with every video and claim processed, because the data remains after the task ends.

If a team treats storage as a negligible addition to compute, that line item could become a significant share of the program’s total costs in 12–18 months—just as leadership decides whether to expand it.

Plan for the data after the task

Storage strategy often prioritizes an expensive, fast tier for data requiring constant, immediate access. Yet much of AI’s growing data footprint sits in warm, cool and cold tiers. A workable architecture has to balance performance and capacity across the data lifecycle, along with energy use, reliability and total cost of ownership. That also means treating archives as potential sources for RAG and inference, setting retention policies around data’s strategic value, and assessing storage across the fleet rather than drive by drive.

I think the key uncertainty is not how much storage a single request needs today, but how long companies will keep what it creates. Teams should be able to answer three questions before the next budget cycle:

Most companies cannot answer all three confidently. Until they can, scaling AI risks expanding a cost that was barely visible in the original budget.