DeepSeek V4.1-Flash Compresses Per-Token Memory to 890 Bytes
Long-context agents face an awkward physical reality: attention can be made sparse and compute can be saved, but every token's memory still needs somewhere to live. An agent with a million-token context running for hours will fill up VRAM, SSDs, and transfer bandwidth with its KV cache—compute is no longer the bottleneck; the cost of storing memory is.
Last week DeepSeek released V4.1-Flash, with a blunt subtitle: pushing KV cache compression to the limit. The headline number: 890 bytes of global KV cache per token—one quarter of the previous V4-Flash and roughly a 437x reduction versus V1. The persistent on-disk cache was cut to one-eighth of V4-Flash's. This isn't a single-point optimization; it's the result of changes across three layers: architecture, cache precision, and deployment system.
Splitting the compute in half
The most counterintuitive of the three core designs is the first. They split the 40-layer network into two halves: 20 layers of causal encoder and 20 layers of decoder. Most input tokens only pass through the encoder, bypassing the decoder's full computation—nearly halving prefill compute. The global KV needed by the decoder is derived from the encoder's output projection.
This split directly targets the pain point of agent workloads: agent tasks are input-heavy, reading far more tokens than they write. Paying full compute price for every token read in is wasteful. Hence an unusual-for-the-industry asymmetric design: 8B activated parameters per token at prefill, 16B at decode—reading is cheap, writing is expensive. A custom price for the input-heavy workload.
The second design is cross-layer reuse. Where 40 layers each previously stored their own KV, there are now three patterns: global KV sharing, index key reuse, and Top-K selection reuse—later layers no longer search from scratch, only within candidate pools picked by earlier layers. The hierarchical sparse indexer works like a library catalog: the first layer narrows things down to one shelf, and each subsequent layer searches only within that shelf.
The third design targets precision and deployment: global KV is stored in FP4 (four bits per number), and the sliding-window attention segment isn't persisted at all—when needed, it's reconstructed via bounded replay from earlier states, trading a little compute for storage.
One-eighth the price, three-quarters of the strength
The parameter budgeting is equally asymmetric. A 552B backbone plus 196B Engram (a conditional memory module, plus DSpark speculative decoding), with 8B/16B activated—using one-third the total size and one-quarter the activated parameters of V4-Pro-Base (49B activated), it approaches or partially exceeds it on knowledge and reasoning benchmarks, and scores 5–10% higher on held-out evaluations.
Reasoning effort is a knob: turning it from 25 to 100 lifts the eight-reasoning-benchmark average from 67.1% to 76.3%, DeepSWE from 66.0% to 74.2%, and Terminal-Bench 2.1 from 82.4% to 90.6%—at a cost of roughly 2.5x output tokens. One detail worth noting: this knob was trained on single-turn inference, yet transfers faithfully to long-horizon agent trajectories—it controls the total budget for exploration and verification. Before buying more compute, turn the knob first; it's the cheapest tier of speedup.
Multi-agent comparisons were done too. The gap widens under long budgets: ProgramBench at eight hours, multi-agent 30.04% vs single-agent 20.39%; FrontierSWE at twenty hours, 32.90% vs 28.20%. Same model, same budget, but organizing as a swarm earns an extra ten points—beyond raw model capability, frameworks and interaction protocols count toward the final score.
What wasn't overstated
The paper is more honest than most technical reports about its boundaries. Bounded replay for SWA is approximate reconstruction, not exact recovery; the performance cost of FP4 wasn't tested across all hardware and workloads; the robustness envelope of the new architecture is "not yet fully characterized"; and against the strongest closed models, benchmark scores are close but a measurable gap remains on the hardest tasks—"highly close in everyday experience" is not "parity at the frontier."
Placing this release into this year's bigger picture is telling. Flagship models are all shrinking activation—GLM 320B-A18B, this one at 552B-A8B, Qwen 125B-A6B—"how big a card you need to run a model" has been rewritten as "activations run fast, total size just needs to fit." This release goes further: driving down the storage price of memory too. The 890-byte figure is a new anchor for the industry—whichever lab reports KV compression next will be compared against this number.
A footnote from earlier coverage: when we broke down vLLM's giant operator merge last week, the AMD 9,119-line merge request was running DeepSeek-V4.1-Flash operators on the MI355X—at that point the model itself hadn't been announced. Engineering moved first; now the paper has arrived. Hardware ecosystems wait on models, model ecosystems wait on hardware—both sides betting the other moves first.
---
*Sources: arXiv 2609.19969 (2026-09-17, DeepSeek, submitted by Wenfeng Liang); 552B backbone + 196B Engram; 45T-token multimodal pretraining, sparse attention trained from 64K context and extended to 1M over 34T tokens; weights open on HuggingFace (deepseek-ai/DeepSeek-V4.1-Flash); compared against V4-Flash / V4-Pro-Base / GLM-5.3 / Kimi-K3; multi-agent benchmarks ProgramBench and FrontierSWE v2.*