Have you read *Sherlock Holmes*? Holmes has a "memory palace"—a virtual building in his mind where every clue, name, and date is stored in a specific room. When he needs to recall something, he walks through that palace to the right drawer.
That's essentially what DeepSeek V4 does—except its palace holds 1 million "memory fragments," and it compresses those fragments from a space that originally required 84GB of VRAM down to under 10GB.
What Does 1 Million Tokens of Context Mean?
For perspective: a full copy of *Dream of the Red Chamber* is about 730,000 Chinese characters. 1 million tokens can hold roughly the entire book plus annotations for the first eighty chapters. Earlier large models—like early GPT-4 versions—could only retain about 8K to 128K of context, equivalent to a few novel chapters. Beyond that length, the model behaves like someone who finished a book but forgot the beginning: ask "what did Jia Baoyu say when he first met Lin Daiyu," and it might mix things up.
DeepSeek V4 pushes this limit to 1 million tokens. You can drop an entire project codebase, a textbook, months of chat history, or a novella into a single conversation and let the model roam freely through the whole "palace."
But here's the catch: the bigger the memory palace, the higher the maintenance cost.
KV Cache: The Model's "Short-Term Memory Notes"
To understand why DeepSeek V4 is revolutionary, you first need to know how large models "remember" context.
Imagine chatting with a very smart friend. To remember everything you've discussed, after each sentence they jot a few keywords on sticky notes. Those stacked notes form your conversation's "historical archive." When you ask a new question, they flip through the notes to find relevant context.
In large models, these "sticky notes" are called the KV Cache (Key-Value Cache). It is one of the largest memory costs during inference. For a 1-million-token context, a traditional architecture's KV cache can balloon past 80GB—meaning you'd need at least two high-end GPUs to run it.
DeepSeek V4's CSA/HCA hybrid attention architecture essentially redesigns how the "sticky notes" are stored:
- Shared KV: Many pronouns like "he," "it," and "this" in different sentences refer to the same thing—no need to record them repeatedly. Like taking notes where a concept gets one entry and everything else says "see above."
- Compressed KV streams: Simplify unimportant note content, keep the important stuff detailed. "The weather is nice today" becomes just "weather-good," while "the client wants the color scheme on page three changed" gets recorded in full.
- Sparse attention: No need to flip through every note each time. CSA (Compressed Shared Attention) lets the model learn "which drawers to keep open and which to lock."
Not Just Compression—Redesigning How Memory Works
Training is also interesting: 32-33T tokens, roughly 1e25 FLOPs, with a parameters-to-data ratio of about 1:20. That means the model has "read" far more material than its own "brain capacity"—like a person who has read twenty times more books than their life experience would suggest.
Equally notable is FP4+FP8 mixed-precision storage: MoE experts in FP4, attention/normalization/routing in FP8. It's like an accountant who uses shorthand bookkeeping for daily entries but precise ledgers for key numbers. Officially, the full model fits on one 8×B200 server—a new template for large-model deployment costs.
Huawei Ascend: Another Path
One more thing worth noting: DeepSeek V4 is technically compatible with Huawei Ascend 950 + CANN, meaning it can run without relying on the NVIDIA CUDA ecosystem.
Ascend's current supply is roughly a quarter of H100 volume, but DeepSeek has publicly said that once Ascend 950 supernodes scale up in the second half of the year, V4 Pro pricing will drop significantly. This is seen as a key milestone for China's self-reliant compute stack.
Benchmarks: Open-Source Ceiling, but Talkative
Artificial Analysis benchmarks show V4 Pro Max scoring 52, a 10-point improvement over V3.2, second only to Kimi K2.6. On GDPval agent tasks, V4 Pro even leads all open-source models. But there are issues: a 94% hallucination rate remains on AA-Omniscience, and running the same benchmark suite produced 190 million output tokens (Flash: 240 million). A low unit price doesn't mean a cheap total task—the model can be verbose, which increases costs.
V4 Flash is the other extreme: 284B total / 13B activated parameters, slightly weaker than Pro, but API pricing of just $0.14/$0.28 per million input/output tokens. Community estimates put a typical month of search + coding API bills at only a few dozen cents. It's the kind of workhorse that's "good enough, cheap, and can hold an entire book."
Conclusion
DeepSeek V4 isn't the model with the most parameters or the highest scores. But it does one thing: it makes "ultra-long memory" affordable. When a single GPU can hold a novel's worth of context, and local deployment costs drop from "lab-grade" to "small-business affordable," AI use cases expand from "Q&A" to "reading companionship," "long-form novel writing," and "whole-codebase analysis"—dreams once abandoned because the model "couldn't remember" suddenly become feasible.
It's like swapping a notepad that held one chapter for an index-card box that holds an entire library. The design of that card box is the engineering art truly worth studying.
Source commit: d9b875d (easy-learn-ai)