2024, if you followed the AI compute market, you may remember a disheartening trend: H100 rental prices were falling fast.
Many interpreted this as "the compute bubble bursting." After all, if the top AI chips were depreciating, wasn't the AI boom cooling off?
But the market played a joke. Starting December 2025, H100 prices rebounded sharply. Even more surprising: a 4-year-old H100 is now worth more than it was three years ago.
Behind this is a profound shift in the economics of compute.
Why Are Old Chips Worth More?
In consumer electronics, there's an iron rule: newer is better, older is cheaper. Your iPhone 12 can't outprice an iPhone 15. But H100 is breaking this rule.
The reason is simple economics: when demand grows faster than supply, prices rise.
Specifically:
- Chip shortage: Despite capacity expansion, advanced-process output grows slowly. A new fab takes years to build; AI demand grows month by month.
- Rise of inference: GPUs were once mainly for training—a batch-process task. Now inference (serving users with trained models) is the dominant compute consumer. Inference requires continuous operation, meaning GPUs are occupied long-term rather than used once and released.
- Explosion of applications: From ChatGPT to vertical AI apps, every new application competes for compute. When demand grows exponentially and supply only linearly, prices must rise.
- 14B-parameter Qwen3.5 runs TTS (text-to-speech) locally
- 27B paired with Hermes Agent handles automated tasks
- 35B fits into 24GB VRAM via quantization
- Full data privacy: your data never leaves your machine
- Zero network latency: faster responses
- Customizability: modify and tune models freely
- Long-term cost advantage: after break-even, running costs approach electricity and maintenance
- TurboQuant, from Google, optimizes KV Cache storage. Someone ran Qwen 3.5-9B with 20K context on a MacBook Air. Another reported a simple 3-line change boosting 32K-context decoding speed by 22.8%.
- RotorQuant claims to be 10-19x faster than TurboQuant with 44x fewer parameters—though this sparked debate over worst-case error bounds.
That's why 4-year-old H100s are more valuable—they still do the job, and new chip supply simply can't keep up.
The Local Deployment Renaissance
Against this compute crunch, a counter-trend is emerging: people are re-examining local deployment.
For two years, the mainstream view was "models are too big, you must rely on the cloud." But things are changing, and the Qwen3.5 series shows another path.
Developer experiences are interesting:
Crucially, the performance loss is only about 1%.
Consider this: if you pay $2,000/month for cloud APIs, you could instead buy a Mac Studio M3 Ultra 512GB for $10,000 and run Qwen3.5-397B locally—breaking even in roughly 10 months. And you gain:
It's the "buying vs renting" choice. When "rent" (API fees) keeps rising while "house prices" (hardware) stay stable, buying becomes more attractive.
Quantization: The Magic That Shrinks Large Models
The key enabling technology is quantization.
Think of a high-resolution photo where each pixel stores 32-bit color. Quantization is like summarizing—you keep enough information for the result to "look about the same."
TurboQuant and RotorQuant are two recent "summarization algorithms."
Google and Anthropic: Compute Is Power
The Financial Times reported Google may help fund data centers for Anthropic.
This reveals a harsh reality: AI competition's core is shifting from "algorithms" to "compute," from "talent" to "capital."
GPT-4-class models reportedly cost over $100 million to train. Next-generation models may hit $1 billion. Fewer and fewer organizations can afford this. OpenAI has Microsoft; Anthropic has Amazon (and possibly Google); Google and Meta are giants themselves—a compute moat limiting who can compete for next-gen models.
Notably, while Anthropic prepares to launch Capybara (possibly bigger than Claude Opus 4.6), it suffered widespread 5xx errors—showing even top AI labs struggle to balance aggressive scaling with service stability.
What Ordinary Developers Should Do
1. Don't panic. Cloud APIs aren't disappearing; they remain best for rapid prototyping, and convenience often outweighs cost. 2. Explore local deployment. If your app is latency-sensitive, privacy-critical, or high-volume, it's worth considering. Qwen3.5 proves "small models, big capabilities." 3. Learn quantization and optimization. These skills save money and deepen your understanding of what models actually are. 4. Follow the open-source ecosystem. Open-source models are closing the gap: Zhipu GLM-5.1 scores 45.3 on coding vs Claude Opus 4.6's 47.9. The best model of the future may be community-maintained rather than proprietary.
Final Thoughts
The H100 price rebound reflects a fundamental tension in AI development:
AI capability grows exponentially, but the infrastructure supporting it—compute, energy, capital—grows linearly.
As the gap widens, expect: rising compute prices, innovation shifting from "bigger models" to "using existing models more efficiently," a local deployment and edge computing revival, and a larger role for open-source communities.
For participants in this transformation, understanding these trends matters more than chasing the latest model release. When the tide recedes, only those who truly understand "why" will find a new course.