DeepSeek V4: Open-Source Evolution — 1.6 Trillion Parameters Meet a 1M Context Window

Forum topic · 小凯 · 2026-05-02

Summary

DeepSeek released V4 Pro on April 25, 2026: a 1.6-trillion-parameter Mixture-of-Experts model with roughly 49B activated parameters per task, MIT-licensed open weights, and a 1-million-token context window. The model introduces a CSA/HCA hybrid attention mechanism combining shared, compressed, and local memory, shrinking KV cache from 83.9GB (V3.2) to 9.62GB, alongside FP4 expert weights and FP8 attention/norm/routing so the full model fits on one 8x B200 server. Training consumed 32-33 trillion tokens (~10^25 FLOPs). Artificial Analysis scored V4 Pro Max at 52, ten points above V3.2 and second only to Kimi K2.6, though high verbosity (190M tokens emitted in one benchmark) and a 94% AA-Omniscience hallucination rate remain concerns. The cheaper V4 Flash variant (284B total/13B active) costs $0.14/M input and $0.28/M output tokens. DeepSeek also confirmed compatibility with Huawei Ascend 950 + CANN, signaling a push toward computing-independence and lower prices once Ascend 950 supernodes scale in H2.

*Translated from a zhichai.net forum post. Source: easy-learn-ai commit d9b875d, AI Daily, 2026-04-25.*

Imagine walking into a library that can hold the entire Encyclopaedia Britannica, every Wikipedia entry, and every important paper from the past decade — with a librarian who can instantly find any page, understand it, summarize it, and even write a new paper for you.

That is what DeepSeek V4 is trying to do.

On April 25, 2026, DeepSeek dropped a bomb on the open-source community: V4 Pro, 1.6 trillion parameters, weights released under the MIT license, and a context window stretched to 1 million tokens. For scale: feed it the full text of *War and Peace* and it can still recall every detail from the opening and cite them precisely at the end.

From "Big" to "Smart"

More parameters don't mean better — everyone knows that by now. What makes V4 remarkable isn't the 1.6T figure but how it makes such a giant usable.

V4 uses a Mixture-of-Experts (MoE) architecture. Picture a consulting firm with hundreds of specialist advisors, where each task only wakes the most relevant few instead of the whole staff. 1.6T is the total headcount, but only about 49B parameters actually work per token — a lean but effective task force. This keeps V4 knowledgeable without melting your GPUs.

The Secret of 1M Context: Fitting an Elephant in a Fridge

Long context has always been AI's pain point. DeepSeek V4's answer is a hybrid attention mechanism called CSA/HCA. The core idea is simple: shared memory + compressed memory + local focus.

When you read a very long detective novel, you don't memorize every word equally — you compress the setting into a rough impression ("rainy London"), keep high-resolution memory of key clues ("the killer's tie was burgundy"), and focus most sharply on the chapter you're reading. CSA/HCA does exactly this, compressing the KV cache from 83.9GB in V3.2 to 9.62GB — a 10x reduction. On top of that, FP4 indexing and FP8 cache push memory costs to unprecedented lows.

Training: A 32-Trillion-Token Reading Marathon

Training volume: 32-33 trillion tokens, roughly 10^25 floating-point operations.
Parameter-to-data ratio of about 1:20, DeepSeek's considered optimum.
Mixed-precision storage: FP4 for MoE experts, FP8 for attention/norm/routing, letting the full model fit on a single 8x B200 server — an attractive reference design for enterprises and labs that want to self-host.

Benchmarks: The Open-Source Ceiling

Artificial Analysis: V4 Pro Max scored 52, a 10-point jump over V3.2, second only to Kimi K2.6.
On GDPval agent tasks, V4 Pro leads all open-source models.
However, V4 is a chatterbox: in the same benchmark run, V4 Pro emitted 190M tokens and the Flash version 240M. A cheap per-token price doesn't mean a cheap total bill if the model says everything it can think of.
A 94% AA-Omniscience hallucination rate remains a warning sign — confident nonsense isn't cured.

V4 Flash: A Ferrari for the Rest of Us

If V4 Pro is the flagship sports car, V4 Flash (284B total / 13B active) is the budget alternative. Slightly weaker than Pro, but with reasoning mode fully on it approaches Pro's mid-tier performance.

Official API pricing: $0.14 per million input tokens, $0.28 per million output tokens. Community estimates put a month of everyday search-plus-coding at only a few dimes. For tasks needing 1M context, this pricing is almost disruptive.

An Ambition to Bypass NVIDIA

V4 is technically compatible with Huawei Ascend 950 + CANN. DeepSeek has publicly said that once Ascend 950 supernodes scale up in the second half of the year, V4 Pro pricing will drop significantly. This isn't just a model — it's a key piece of China's compute "self-reliance chain."

Conclusion

DeepSeek V4 isn't perfect. It's verbose, its hallucination rate is still high, and it still trails closed-source giants on some benchmarks. But it did one thing: lowered the bar for "top-tier LLM" from "only a few Silicon Valley companies can play" to "run it yourself if you have a good enough server."

The open-source community has waited a long time for a model like this.