Google adds 2 TPU systems for confidential inference and long contexts

TPU 8tx uses the same silicon as the 8t training chip, paired with AMD EPYC host processors, and protects data and model weights in memory while they are being processed. Google said it cuts latency for multi-step agents by up to half compared with the 8t, and is aimed at regulated companies and governments that want to fine-tune and run models on sensitive data.

TPU 8i Max puts two 8i inference chips in one package, doubling compute, on-chip SRAM and high-bandwidth memory bandwidth, for workloads with very long contexts. Google said its TPUs now have nearly 100 customers, including trading firms, and that this is the first year it has introduced four TPU systems.