Google unveils TPU 8tx and 8i Max: confidential-computing and long-context AI chip systems

TPU 8tx is designed for regulated enterprises and governments that run frontier or proprietary models, and agents, alongside sensitive data. Google says it also suits small-scale training. It uses the same silicon as TPU 8t, paired with fifth-generation AMD EPYC processors as hosts, and it provides confidential computing in the hardware, so data and model weights stay protected and isolated while they are in use and not only when stored or in transit. It uses the low-latency Boardfly network that debuted with TPU 8i, and Google claims up to two times lower latency than TPU 8t, with what it calls a near-zero performance penalty. Teams can fine-tune on private data, distil large models into smaller ones and run reinforcement learning on one system without moving sensitive data between clusters, according to the post.

TPU 8i Max combines two TPU 8i chips in a single package. Google says it doubles compute, SRAM and high-bandwidth-memory bandwidth compared with TPU 8i, scaling all three together, and fits twice the silicon in the existing rack footprint, so a data centre gets more compute per square foot and per watt. The target is work such as many agents operating across a whole codebase, or hours of video, where memory bandwidth is the bottleneck.

The four-system lineup and the claims Google makes

Google presents the four systems as a purpose-built option for each stage of what it calls the agentic lifecycle.

Google says TPUs now serve nearly 100 customers, including frontier laboratories, trading firms, retailers and universities. It says it is also letting customers deploy TPUs directly in their own data centres, and that feedback from customers in highly regulated industries shaped the new systems. It adds that it uses AlphaChip and Gemini to speed up chip layout design.

Software announced alongside, and what is missing

The announcement was accompanied by software. Google says its TorchTPU project, which is open source, lets PyTorch and Hugging Face models run natively on TPUs, with backends for vLLM and SGLang. It described a preview suite of agents that connects to coding tools to help with onboarding, finding bottlenecks and tuning kernels, and an open-source XLA Serving Engine that lets users train in Python and serve in a C++ runtime across TPUs, GPUs and CPUs. It reports state-of-the-art price-performance on two benchmarks, InferenceX and AgentX.

Other items include an Agent Compute Engine that suspends and resumes idle agent sessions to cut cost, and TPU Sync, an open-source tool that Google says cuts key-value transfer latency and time to first token by more than 90%. Cloud Pathways, which orchestrates jobs across several TPU slices from one controller, is generally available on Google Kubernetes Engine, as is dynamic slicing, which Google says gives up to five times faster job start-up and up to 4.5 times faster fault recovery.

The earlier two systems were announced in April at Google Cloud Next, where the company said TPU 8t and TPU 8i deliver up to two times better performance per watt than the previous generation, called Ironwood. The same blog posts say the pair would be generally available later this year. The new post adds that customers can now run TPUs inside their own data centres, a change from the earlier model in which the chips were available only through Google's cloud.

Several things are left open. The post does not say when TPU 8tx and TPU 8i Max can be used or where, what they cost, or how much memory they carry. The performance multiples are relative to Google's own earlier systems, and the post does not compare the new systems with those of other chip makers. Independent measurements of the new systems were not available when this was written, so the claims are Google's alone.