Source: Commit d9b875d (easy-learn-ai, 2026-04-25)
Imagine reading a 1-million-word novel. After finishing, a normal person clearly remembers only a few dozen key points — the protagonist's name, major plot twists, foreshadowing. You wouldn't memorize every punctuation mark on every page, because that would be absurd.
But until recently, that's exactly how AI read long texts.
Traditional Transformer models store every token (roughly a word or character) in a short-term memory called the "KV cache" when processing long texts. It's like photocopying every page of that million-word novel and taping it to your wall, afraid of forgetting. By the end, your room is covered in a million sheets of paper.
DeepSeek V4 did something that seems counterintuitive: it decided to forget most of the content and remember only what truly matters.
The Magic of 83.9GiB → 9.62GiB
At launch, DeepSeek V4 published a staggering figure: at 1M-token context, the KV cache shrank from 83.9GiB (previous generation) to 9.62GiB.
Roughly 10x.
This isn't simple "zip file" compression — it's a hybrid attention architecture called CSA/HCA. Here's a plain-language explanation.
Imagine attending a large meeting with a hundred people. The traditional approach: you remember every sentence everyone said and can accurately recite any segment afterward. That requires extraordinary memory, and your mental burden grows exponentially with headcount.
CSA (Compressed Sparse Attention) works like this: you listen closely to the few people near you (sliding-window attention), keep only a "meeting atmosphere summary" for people far away (compressed KV streams), and there's a shared "meeting topics whiteboard" everyone can see (shared KV).
HCA (Hierarchical Cross Attention) goes further: it lets "listeners" at different levels attend to different granularities. Frontline staff focus on execution details, middle managers on department coordination, executives only on strategic direction. Information flows between levels, but isn't dumped wholesale onto everyone.
Why Does This Matter?
You might think, "Just saving some memory — what's the big deal?"
In AI engineering, memory is everything.
When the KV cache drops from ~84GB to ~10GB, it means:
- You can run long-text tasks on a consumer GPU that previously required a professional server
- API costs can fall dramatically (DeepSeek V4 Flash is priced at $0.14 per million input tokens)
- Real-time applications become viable (lower latency means smoother interaction)
- CSA/HCA architecture highlights: https://substack.com/redirect/0fe9c80a-0feb-4fd8-aacb-7a35946d5e7e
- DeepSeek V4 technical report: https://substack.com/redirect/a14b3fde-deb5-4993-a38d-32e2cebe559a
- Training scale analysis: https://substack.com/redirect/159dde94-efc3-43d8-989c-e10262d9c2f3
The deeper implication: "ultra-long context" goes from an expensive gimmick to a technology that can be deployed at scale.
Previously, 1M-token context was a toy only a few labs could afford. Now an ordinary developer with a few hundred dollars of hardware can experience it.
The Open-Source World Responds
DeepSeek V4's release triggered a chain reaction.
NVIDIA claims 150+ TPS agent interactions on Blackwell Ultra; inference platforms like vLLM, Together, and Baseten published multi-GPU benchmarks on launch day; Huawei's Ascend 950 + CANN also announced compatibility — seen as a key step in China's "self-sufficient" compute stack.
Community reactions are even more interesting. Someone ran DeepSeek V4 Flash on a 256GB RAM Mac; someone calculated that a monthly API bill for everyday search + coding might be just a few dimes.
This isn't "just another model release." It's redefining the standard for what counts as "usable."
Back to the Metaphor
Remember the million-word novel metaphor?
What DeepSeek V4 taught AI isn't how to "remember harder," but how to "forget smartly."
Humans survive the flood of information not through unlimited memory, but through selective attention — knowing what's worth keeping and what to let go. What the CSA/HCA architecture does, at its core, is give AI that ability to choose.
From 83.9GiB to 9.62GiB, what's compressed isn't just memory — it's also our fixation on "intelligence must equal scale."
Sometimes subtraction is the real progress.
---
Further technical reading