When the 1.6-Trillion-Parameter Beast Learns to Slim Down: DeepSeek V4's Million-Token Context Magic
Have you ever wondered what it would take for someone to read every book in a library and then answer any question about the details? For AI, this is called "long context" — processing massive amounts of text in one pass. The problem: memory balloons. Traditional LLMs reserve GPU memory for every token read, and by a million tokens the memory requirement is supercomputer territory.
DeepSeek V4 does something near-magical: it compresses the memory needed for million-token context from 84GB to under 10GB — roughly a 10x reduction.
The Birth of the Beast
DeepSeek V4 ships in two versions:
- Pro: 1.6 trillion total parameters, but only 4.9 billion activated per token — like a think tank with 16,000 experts where only the most relevant few answer each question.
- Flash: lighter, 284B parameters with 13B active.
- Shared memory (CSA): already-read content is compressed into "summary notes" rather than remembering every detail — like retaining key plot points after finishing a chapter instead of every punctuation mark.
- Precise targeting (HCA): when new content arrives, the model first consults the summary notes to locate the relevant range, then does fine-grained comparison only within that small range — like checking a table of contents instead of flipping through the whole book.
- Local sliding window: recent content keeps high-resolution memory; distant content uses the compressed version.
The MoE (Mixture of Experts) architecture means you never wake all the experts at once — only the most relevant ones.
The Magician's Hand: CSA/HCA Hybrid Attention
The 10x KV cache compression hinges on a hybrid attention scheme called CSA/HCA:
The Art of Training: 32T Tokens and FP4/FP8 Mixed Precision
V4 was trained on 32–33 trillion tokens, totaling ~1e25 FLOPs. The parameters-to-data ratio is about 1:20 — a repeatedly validated "sweet spot"; too little data underfits, too much yields diminishing returns.
Precision: MoE experts are stored in FP4 (4-bit float), while attention and normalization layers use FP8 (8-bit). The upshot: the full model fits on a single 8x B200 server, a major cost reduction for enterprise deployment.
The Open-Source "Second-Tier Ceiling"
An interesting benchmark detail: V4 Pro scores 52 on Artificial Analysis — a 10-point gain over V3.2, second only to Kimi K2.6, and it leads all open models on agent tasks.
But evaluation exposed a "chatterbox" problem: V4 Pro output 190M tokens on the same test suite; Flash used 240M. Cheap per-token pricing doesn't mean a cheap total bill if the model overthinks and burns more tokens.
Flash pricing is $0.14/$0.28 per million input/output tokens — nearly a floor price for its size. Community estimates put a month of everyday search plus coding API usage at just a few cents (RMB jiao-level).
The Deeper Signal: Huawei Ascend Compatibility, Bypassing CUDA
An easily missed signal: V4 is technically compatible with Huawei Ascend 950 + CANN. Ascend supply is currently about a quarter of H100 volumes, but DeepSeek has publicly said V4 Pro pricing will drop noticeably once Ascend supernodes scale up in the second half of the year.
This isn't just "one more hardware option." It lays a path for the entire Chinese compute ecosystem to operate without depending on NVIDIA/CUDA.
Summary
What impresses most about DeepSeek V4 isn't any single metric, but that it shows "big" and "cheap" can coexist. 1.6 trillion parameters, million-token context, MIT license, 10x KV compression, Huawei compatibility — together these aren't a simple product iteration but a manifesto about how future AI should be built, used, and priced.
For developers, ultra-long context is no longer a cloud-only luxury. For the industry, the gap between open-source and closed frontier models is visibly narrowing.
---
Source: commit d9b875d (easy-learn-ai)