Lessons from the H100 Price Rebound: The Compute War Enters a New Phase
In 2024, if you followed the AI compute market, you may remember a disheartening phenomenon: H100 rental prices were falling fast.
Many at the time interpreted this as "the compute bubble bursting." After all, if the top AI chips were depreciating, did that mean the AI craze was cooling?
But the market played a joke. Starting December 2025, H100 prices rebounded sharply. Even more surprisingly, 4-year-old H100s are now worth more than they were three years ago.
Behind this lies a profound transformation in "compute."
Why Are Old Chips Worth More?
In the traditional electronics world, there is an iron rule: the newer, the better; the older, the cheaper.
Your iPhone 12 can't be pricier than an iPhone 15; your GTX 1080 can't be worth more than an RTX 4090. Technology iteration means old products depreciate—perfectly normal.
But the H100 is breaking this rule.
The reason can be explained by a simple economic principle: when demand grows faster than supply, prices rise.
Specifically:
Chip shortage: Although manufacturers are expanding capacity, advanced-process capacity grows slowly. Building a new fab takes years, while AI demand grows on a monthly timescale.
The rise of inference: GPUs were once mainly used for training—a relatively concentrated process where a batch of GPUs finishes one model and moves to the next task. Now, inference (serving trained models to users) is becoming the main compute consumer. Inference runs continuously, meaning GPUs are occupied for long periods rather than used briefly.
Explosion of application scenarios: From ChatGPT to vertical AI applications, every new app competes for compute. When demand grows exponentially and supply only linearly, prices must rise.
This explains why 4-year-old H100s are worth more—they're "old" but still do the job, and new chip supply simply can't keep up with demand.
The Revival of Local Deployment
Against this backdrop of tight compute, an interesting counter-trend is happening: people are re-examining "local deployment."
Over the past two years, the mainstream voice was "models are getting bigger, you must rely on the cloud." GPT-4, Claude, Gemini—these top models have hundreds of billions or even trillions of parameters; ordinary users couldn't dream of running them locally.
But things are changing.
The arrival of the Qwen3.5 series shows another possibility.
Developer experiences are interesting:
- 14B-parameter Qwen3.5 can run TTS (text-to-speech) locally
- 27B paired with Hermes Agent handles automated tasks
- 35B fits into 24GB VRAM via quantization
- Complete data privacy: your data never leaves your machine
- Zero latency: no network transmission, faster responses
- Customizability: modify and tune the model freely
- Long-term cost advantage: after payback, running costs approach electricity and maintenance
- Compute prices rise
- Innovation shifts from "training bigger models" to "using existing models more efficiently"
- Local deployment and edge computing revive
- Open-source communities play a bigger role
More importantly, performance loss is only about 1%.
What does that mean?
Imagine paying $2,000/month for cloud API access. Now you could spend $10,000 on a Mac Studio M3 Ultra 512GB and run Qwen3.5-397B locally. Payback in about 10 months.
And you gain:
It's like the "buy vs. rent" choice for housing. Renting is flexible; buying requires upfront investment but pays off long-term. When "rent" (API fees) keeps rising while "home prices" (hardware costs) stay stable, buying becomes more attractive.
Quantization: The Magic That Makes Big Models "Small"
A key technology enabling local deployment is quantization.
It sounds complex, but a simple analogy helps:
Imagine a high-resolution photo where each pixel stores 32-bit color (the standard approach). It's like writing a paper where every point is supported by exhaustive argument and rich examples.
Quantization is like "summarizing"—you don't keep every detail, just enough information that the whole "looks about the same."
TurboQuant and RotorQuant are two of the latest "summarization algorithms."
TurboQuant comes from Google; its core idea is optimizing how the KV Cache (a key data structure in attention) is stored. Someone ran Qwen 3.5-9B on a MacBook Air with 20K context and it actually worked. Others achieved a 22.8% decoding speedup at 32K context with a simple 3-line change.
RotorQuant is more aggressive, claiming to be 10-19x faster than TurboQuant with 44x fewer parameters. This has sparked controversy—some question its worst-case theoretical error.
Behind these technical debates lies a broader trend: making large models run on limited hardware is becoming a central problem in AI.
Google and Anthropic: Compute Is Power
The Financial Times reported that Google may fund Anthropic's data center construction.
This news means far more than "Google invested in another company."
It reveals a harsh reality: the core of AI competition is shifting from "algorithms" to "compute," from "talent" to "capital."
How much does training a large model cost?
GPT-4-class models reportedly cost over $100 million to train. Next-generation models may reach $1 billion. And the generation after?
The list of organizations that can afford this is shrinking. OpenAI has Microsoft behind it, Anthropic has Amazon (and possibly Google), while Google and Meta are giants themselves.
This forms a "compute barrier": only players with sufficient compute can compete in the next-generation model race.
More interestingly, while Anthropic prepared to launch Capybara (possibly larger than Claude Opus 4.6), it experienced widespread 5xx errors. Even top AI companies struggle to balance "aggressive scaling" and "service stability."
Implications for Ordinary Developers
What should ordinary developers do in this compute war?
First, don't panic. Cloud APIs won't disappear—they remain the best choice for quickly validating ideas and prototyping. For most use cases, cloud convenience far outweighs cost concerns.
Second, explore local deployment possibilities. If your application is latency-sensitive, privacy-critical, or high-volume, local deployment may be worth exploring. Qwen3.5 and similar models have proven "small models, big capabilities" is possible.
Third, learn optimization techniques like quantization. These save money, but more importantly, they deepen your understanding of AI models. Knowing how to "compress" a model clarifies what a model fundamentally is.
Finally, keep watching the open-source ecosystem. Open-source models are rapidly closing the gap with closed ones. Zhipu GLM-5.1's coding score has reached 45.3, narrowing the gap with Claude Opus 4.6 (47.9). In the future, the best model may not be OpenAI's proprietary product but a community-maintained open project.
Final Thoughts
The H100 price rebound looks like just a market phenomenon, but it reflects a fundamental contradiction in AI development:
AI capability grows exponentially, but the infrastructure supporting it (compute, energy, capital) grows linearly.
As the gap between these two curves widens, certain things will inevitably happen:
After all, when the tide recedes, only those who truly understand the "why" will find new channels.