This piece, originally published in Chinese on zhichai.net, argues that the AI compute landscape is fracturing as Google's custom TPU silicon challenges Nvidia's long-standing dominance — the so-called "Nvidia tax."
Key points
Two opposing chip philosophies
- Nvidia (general-purpose): GPUs like the B200 Blackwell are "Swiss Army knives" — two stitched silicon dies delivering 20 PFLOPS, capable of training LLMs, folding proteins, or rendering games. Maximum flexibility at maximum cost and complexity.
- Google (specialized): TPUs are "laser-guided scalpels" — pure matrix-math circuits with graphics hardware stripped away. They exist to run Gemini faster and cheaper, not to play Cyberpunk 2077.
- Compute: ~4,614 TFLOPS (FP8) per chip
- Memory: 192 GB HBM3e
- Bandwidth: 7.37 TB/s
- Dynamic topology: changing the network shape (rings, meshes) is a mirror rotation, not a re-cabling project
- No electronic switch bottleneck: bandwidth is limited only by physics
- Nvidia stays king of training — frontier research still demands B200-class brute force
- Google (and AWS, Meta) dominate the inference factory — where AI becomes a utility and efficiency is everything
- Google Cloud Technical Documentation: TPU v7 "Ironwood" Architecture & Performance Metrics (2025)
- SemiAnalysis: "The Optical Circuit Switching Revolution: How Google broke the datacenter speed limit" (2024)
- DeepMind Research: "JAX: Composable transformations of Python+NumPy programs" & XLA Compiler Whitepaper
- Nvidia Corporation: Blackwell B200 Architecture Whitepaper (2024)
- Financial Times / Bloomberg: "The Shift from Training to Inference: AI's Economic Turning Point" (2025 Analysis)
The paradigm shift: from training to inference
The author frames the industry's move from the Training Era to the Inference Era as the reason TPUs matter now. Training a model is like school; inference is the graduate doing daily work. Once billions of daily requests hit ChatGPT and Gemini, cost and energy efficiency — not raw single-chip power — become decisive.
TPU v7 "Ironwood" reference specs:
Individually, the B200 still wins. But Google packs thousands of Ironwood chips into Pods, where smaller, cooler, denser, more power-efficient TPUs win on economics. If power is the bottleneck and Google uses ~40% less energy for the same work, physics decides the winner.
Optical Circuit Switching (OCS): the real breakthrough
Google's Palomar OCS replaces electronic packet switches with MEMS mirrors that physically steer laser beams from one fiber to another in milliseconds:
1. A data-carrying light beam exits a fiber 2. It hits a mirror smaller than a grain of rice 3. The mirror rotates to redirect the beam 4. The light lands in the target fiber — no optical-to-electrical conversion
Benefits:
Where Nvidia relies on expensive InfiniBand cabling, Google solves interconnect with optics — so a "hive mind" of slower individual chips can outperform a cluster of stronger ones.
Cracking the CUDA moat
Nvidia's deepest lock-in is software: CUDA, the de facto language of AI for 15 years. Google's counter is XLA (Accelerated Linear Algebra), a compiler that translates high-level Python code into machine code for TPUs, GPUs, or CPUs, plus the JAX framework, which relies on compiler-driven optimization instead of hand-tuned CUDA kernels. As major AI libraries become hardware-agnostic, switching costs drop from "impossible" to "manageable" — and capital follows the arithmetic: the same model at 60 units of cost on TPU vs 100 on Nvidia, no rewrite required.
Conclusion: a multi-polar future
The author does not predict Nvidia's death. Instead:
The closing metaphor: Nvidia built the rocket to the moon; Google is laying the railway to the future. "David didn't defeat Goliath — David became another Goliath."