What happened
On 7 November 2025, Google announced general availability of Ironwood, its seventh-generation TPU (TPU7x) and the first it describes as purpose-built for inference — the 'age of inference.' An Ironwood superpod scales to 9,216 chips connected by a 9.6 Tb/s inter-chip interconnect with 1.77 petabytes of shared high-bandwidth memory. Google reports a 10× peak-performance improvement over TPU v5p and more than 4× better performance per chip versus the previous-generation Trillium (TPU v6e), for both training and inference.
Why it matters
Custom silicon aimed squarely at inference is a tell about where AI cost has moved. Training is a one-time capex spike; serving reasoning and agent workloads — models that spend tokens thinking — is a recurring bill that scales with usage. Hyperscalers designing chips for low-latency, high-volume inference signals that the serving side is now the expensive, strategic part of the stack.
You won't tape out a TPU, but the trend sets your constraints. The Staff read: inference economics are becoming the design center — plan for the token cost and latency of reasoning/agent models as a first-class capacity problem, expect the accelerator market (TPU, Trainium, GPUs) to fragment your options, and keep serving portable so you can move between accelerators rather than hard-coding to one vendor's chip.
Why is the AI industry shifting hardware investment toward inference rather than training, and how should that change how you budget and architect a system that serves an LLM?
Probes AI cost structure (capex vs opex), inference latency/throughput, and accelerator portability.