DeepSeek V4: How a 1.6T-Parameter MoE Model Compresses 1M-Context KV Cache by 10x

Forum topic · 小凯 · 2026-05-04

Summary

A zhichai.net forum post analyzes DeepSeek V4, released in Pro and Flash variants under an MIT license. The Pro version has 1.6 trillion total parameters with 4.9 billion activated per token via Mixture-of-Experts, while Flash runs 13 billion of 284 billion parameters. The standout feature is a CSA/HCA hybrid attention mechanism that combines compressed shared memory, hierarchical coarse-to-fine lookup, and sliding-window attention, reducing KV cache memory for 1-million-token contexts from about 84GB to under 10GB—runnable on a single consumer GPU. The model was trained on 32–33 trillion tokens (~1e25 FLOPs) using FP4 for MoE experts and FP8 for attention/norm layers, fitting the full model on one 8x B200 server. It scores 52 on Artificial Analysis (up 10 from V3.2), second only to Kimi K2.6, though high output verbosity (190–240M tokens on benchmarks) can offset its low $0.14/$0.28 per million token pricing. The post also notes native compatibility with Huawei Ascend 950 + CANN, positioning V4 as a step toward a CUDA-independent compute ecosystem.

Source: commit d9b875d (easy-learn-ai forum post, translated and edited)

When a 1.6-Trillion-Parameter Giant Learns to Slim Down: DeepSeek V4's Million-Token Context Trick

How do you get a model to read an entire library and answer questions about any detail? For AI, that's "long context"—processing massive amounts of text in one go. The problem: memory bloats. Traditional LLMs allocate GPU memory for every token read, and at a million tokens, only a supercomputer can cope.

DeepSeek V4 does something close to magic: it compresses the memory needed for million-token context from 84GB to under 10GB—roughly a 10x reduction.

The Birth of the Giant

DeepSeek V4 ships in two versions:

Pro: 1.6 trillion total parameters, with only 4.9 billion activated per token—like a think tank of 16,000 experts where only the most relevant few answer each question.
Flash: lighter, activating 13 billion of 284 billion parameters.

This is the core appeal of the MoE (Mixture-of-Experts) architecture: you never wake all the experts at once, only the most relevant ones.

The Magician's Hand: CSA/HCA Hybrid Attention

The 10x KV cache compression centers on a hybrid attention mechanism called CSA/HCA. Traditional attention compares every new token against all previous tokens, so the KV cache grows linearly. DeepSeek's approach:

Shared memory (CSA): previously read content is compressed into "summary notes" rather than stored in full detail—like retaining key plot points after finishing a chapter.
Precise localization (HCA): for new content, the model first consults the summary notes to narrow the range, then does fine-grained comparison within that small window—like checking a table of contents instead of flipping through the whole book.
Local sliding window: recent content stays at high resolution; distant content uses compressed representations.

The result: million-token contexts need under 10GB of VRAM—runnable on a consumer-grade GPU.

The Art of Training: 32T Tokens and FP4/FP8 Mixed Precision

Trained on 32–33 trillion tokens, with total compute around 1e25 FLOPs.
Parameters-to-data ratio of roughly 1:20—a repeatedly validated "sweet spot"; too little data underfits, too much yields diminishing returns.
Storage precision: MoE experts use FP4, while attention and normalization layers use FP8—like an archivist keeping key files sharp while scanning everything else at low resolution.

The payoff: the full model fits in a single 8-GPU B200 server, a major cost reduction for enterprise deployment.

The Open-Source "Second-Tier Ceiling"

An interesting benchmark detail: V4 Pro scores 52 on Artificial Analysis, a 10-point improvement over V3.2, second only to Kimi K2.6. On agentic tasks it even leads all open-source models.

But benchmarks also exposed a chattiness problem: on the same test suite, V4 Pro emitted 190 million tokens and Flash 240 million. Cheaper per-token pricing doesn't mean cheaper total cost if the model overthinks.

Flash pricing is $0.14/$0.28 per million input/output tokens—near floor pricing for this size class. Community estimates suggest everyday search plus coding might cost only tens of cents per month in API fees.

The Deeper Signal: Huawei Ascend Compatibility, Bypassing CUDA

An easy-to-miss detail: V4 is technically compatible with Huawei Ascend 950 + CANN. Ascend supply is currently about a quarter of H100 volume, but DeepSeek has publicly said V4 Pro pricing will drop noticeably once Ascend supernodes scale up in the second half of the year.

This isn't just "one more hardware option." It lays a path for China's compute ecosystem that doesn't depend on NVIDIA/CUDA.

Takeaways

The most striking thing about DeepSeek V4 isn't any single metric, but that it shows "big" and "efficient" can coexist: 1.6 trillion parameters, million-token context, MIT license, 10x KV compression, Huawei compatibility—together, less a product iteration than a manifesto about how future AI should be built, used, and priced.

For developers, ultra-long context is no longer a cloud-only luxury. For the industry, the gap between open-source and closed frontier models is visibly shrinking.