Google released four TPU variants in a year, intensifying competition with Nvidia

Google has added two systems to its eighth-generation accelerator lineup: TPU 8tx for confidential computing and TPU 8i Max for workloads that require a lot of memory. With these additions, the company has released four TPU systems in a year for the first time. The new variants are built on existing processors but use different packaging, networking and memory configurations. Google is also developing software tools to make the switch to TPU easier and attract external customers; it now has nearly 100.

Google introduces two new TPU systems, bringing its total to four in a year

New configurations on existing processors

TPU 8tx and TPU 8i Max are designed for different workloads. The first is intended for handling confidential data; the second, for applications with high memory requirements.

Google is not changing chip generations: both systems are built on silicon that already exists. Instead, the company is changing the configurations — processor packaging, network connections and memory capacity. Google’s modular approach is intended to let it release new hardware variants in months rather than years.

TPU 8tx protects data during computation

TPU 8tx is intended for companies and government organizations that work with confidential data. Data is usually encrypted at rest and in transit, but a processor must decrypt it in memory to perform computations.

Google says that data and model weights remain protected and isolated on TPU 8tx, even during processing. Nvidia offers a comparable capability for graphics processors in its confidential computing architecture.

The system uses Boardfly, a low-latency network Google developed for real-time inference. According to the company, TPU 8tx should reduce latency up to twice as much as TPU 8t. On a single system, enterprises will be able to fine-tune models, compress large models into smaller ones and train AI agents with reinforcement learning, without moving confidential data between clusters.

TPU 8i Max increases memory and computing power

TPU 8i Max addresses limited memory bandwidth. It can become a bottleneck when AI agents process large codebases or several hours of video at the same time.

Google has placed two TPU 8i chips in one package. In the same rack format, this provides twice the computing power, memory and memory bandwidth. As a result, the system delivers higher performance per square meter of data center floor space.

With the new additions, Google now offers four TPUv8 variants. | Image: Google

Source: the-decoder.com

Google makes switching to TPU easier

Google has released TorchTPU as open source. It is intended to let code written for PyTorch run on TPU with minimal changes. The popular inference frameworks vLLM and SGLang are also supported.

Inference market intensifies competition for chips

Google is increasingly offering TPU to external customers, rather than using them only in-house. It now has nearly 100 external TPU customers.

The expanded lineup reflects growing demand for inference — the day-to-day running of models. As AI applications spread, attention is shifting from expensive training runs to model execution. Specialized chips such as TPU can use their architectural advantages for these workloads.

Amazon, Microsoft and OpenAI are also developing their own accelerators. Nvidia has strengthened its inference offering through a deal with Groq that includes hiring employees and a licensing agreement.

Dataist's take: Google is expanding TPU in two directions at once, offering solutions for confidential computing and improving performance on workloads limited by memory capacity. But software compatibility also matters to external customers: TorchTPU and support for popular frameworks should make it easier to switch to Google’s hardware.