← Back to feed

At Google Cloud Next 2026, Google unveiled TPU 8t and TPU 8i — the eighth generation of its Tensor Processing Units, and the first time in the line’s history that training and inference have been separated into distinct silicon.

The design decision was driven by AI agents. Multi-step agentic workflows demand sustained low-latency inference across thousands of simultaneous interactions. Google determined that a single chip optimised for both workloads was leaving efficiency on the table.

TPU 8t: Training at Scale

TPU 8t is built to cut frontier model training cycles from months to weeks. A single TPU 8t superpod scales to 9,600 chips with two petabytes of shared high-bandwidth memory — double the memory capacity of the previous generation. Compute performance per pod is nearly 3x the prior generation.

The chip prioritises compute throughput, scale-up bandwidth, and productive compute time over inference latency. Citadel Securities is among the early adopters named at the announcement.

TPU 8i: Inference for Agents

TPU 8i trades compute headroom for memory bandwidth and latency. It uses SRAM — the same architectural choice made by Cerebras and Groq — which allows it to serve low-latency token generation without the round-trip cost of HBM reads.

The rationale is direct: “interactions between agents at scale magnify even small inefficiencies.” For a system running thousands of parallel agent sessions, latency improvements compound. SRAM puts TPU 8i in the same architectural camp as dedicated inference hardware rather than the training-first chips that dominated previous TPU generations.

Why the Split Now

TPU hardware roadmaps run three to five years ahead of deployment. Google decided several years ago that inference demand would eventually justify its own silicon — and that agent workloads would make the case even more clearly.

The decision also signals a quiet acknowledgement that training and inference have diverged enough that shared silicon carries a meaningful performance penalty in at least one direction.

Both chips can run general workloads. Specialisation unlocks the efficiency gains; it doesn’t enforce hard walls.

Availability

TPU 8t and TPU 8i are available for preview requests now via Google Cloud. General availability is expected later in 2026. The chips were co-designed with Google DeepMind and are intended to serve Gemini training and serving alongside external customers.