DeepSeek V4: How One Company Fits a Million-Token Context Window in Reach

*Translated and adapted from a zhichai.net forum post. Source: easy-learn-ai commit d9b875d.*

Key points

• 1M-token context window: Enough to ingest the first two-thirds of the *Three-Body Problem* trilogy in one go. V4 Pro ranks at the top of open-source models on multiple benchmarks, behind only closed frontier systems like GPT-5.x and Claude Opus.
• Massive KV cache compression: KV cache drops from 83.9 GiB (V3.2, 128K context) to 9.62 GiB at 1M context — roughly 8.7x longer context at ~1/9 the memory. FP4 indexing + FP8 cache push compression further.
• MoE efficiency: V4 Pro has 1.6T total parameters but activates only 49B per forward pass — a small subset of "experts" handles each input, like a hospital triaging only relevant specialists.

How the long context works

Traditional attention has quadratic complexity: 1M tokens would mean an intractable number of attention-score computations. DeepSeek's answer is CSA/HCA hybrid attention:

• Shared KV and compressed KV streams
• Sparse attention over compressed ("summarized") distant tokens
• Local sliding windows keep nearby tokens at full fidelity

Analogy: rather than memorizing every page, the model keeps chapter summaries for distant content while reading current pages closely.

Open source as strategy

• MIT license: unrestricted commercial use, no strings attached — unlike Meta's Llama terms or API-only closed models.
• A 58-page technical report discloses everything from mHC (manifold-constrained hyper-connections) to MoE routing, CSA/HCA math, and FP4/FP8 mixed-precision implementation.
• By making its architecture the de facto reference, DeepSeek benefits from a global community adapting and extending it — the open-source flywheel.

Beyond NVIDIA

• V4 supports Huawei Ascend 950 + CANN on day one; prices for V4 Pro are expected to drop once Ascend 950 supernodes scale in H2.
• Meta simultaneously announced tens of millions of AWS Graviton ARM cores for inference — evidence of a broader 2026 trend toward diversified compute and CPU+GPU hybrid inference.
• DeepSeek also open-sourced DeepEP V2 and TileKernels for LLM communication and operator parallelism.

Pricing and caveats

• V4 Flash (284B total / 13B active): $0.14 / $0.28 per million tokens (input/output). Daily search + coding use can cost less than a coffee per month; with reasoning mode on, Flash approaches mid-tier Pro quality.
• Verbosity: in one benchmark run, Flash emitted 240M tokens — far more than peers. Low unit price ≠ low total bill; users should prompt for concise answers.
• Hallucinations: Artificial Analysis measured a 94% hallucination rate for V4 Pro Max on AA-Omniscience, meaning the model tends to guess rather than admit ignorance on questions outside its training data. For document-grounded tasks (its core strength), this matters less; for questions about recent events, fact-checking is essential.

Takeaway

One year ago, open-source models lagged closed frontier models noticeably. With 1.6T parameters, 1M context, MIT licensing, full engineering disclosure, Huawei chip support, and near-free API pricing, DeepSeek V4 is not just another open model — it is a complete, commercializable, decentralized alternative. It isn't perfect (hallucinations, verbosity, long-context UX still maturing), but the direction is clear: large models are becoming infrastructure every engineer can touch, modify, and deploy.