DeepSeek V4 Pro Deep Dive: 1.6T-Parameter MoE at 1/70th the Price
> English edition of a zhichai.net forum post analyzing DeepSeek's V4 Pro release (April 24, 2026).
On April 24, 2026, DeepSeek released the V4 Pro preview the same day seven domestic Chinese chip vendors (Huawei, Cambricon, Hygon, Moore Threads, and others) announced Day 0 adaptation. The post frames this as a story about engineering efficiency, geopolitics, and open-source conviction.
Key points
Architecture: from V3 to V4 Pro
| Metric | DeepSeek V3 | DeepSeek V4 Pro | Change | |---|---|---|---| | Total parameters | 671B | 1.6T | +138% | | Active params/token | 37B | 49B | +32% | | Context window | 128K | 1M | 8x | | NIAH @ 1M | ~45% | 97% | +52pp | | KV cache | baseline | 7% of V3 | -93% | | License | MIT | MIT | unchanged | | API price (in/out, $/M) | ~$0.14 / ~$0.28 | $1.74 / $3.48 | adjusted |
Three engineering breakthroughs:
- Engram (conditional memory): separates static knowledge retrieval (O(1) hash lookup into an embedding table) from dynamic reasoning (a 256-expert MoE backbone, 8 experts/token), with context-aware gating. Optimal split found to be ~20-25% memory / 75-80% compute. Gains: MMLU +3.4, BBH +5.0, HumanEval +3.0, NIAH @ 1M from 84.2% to 97%.
- mHC (manifold-constrained hyper-connections): standard hyper-connections amplify signals ~3000x in deep networks; mHC uses Birkhoff polytope projection to cap amplification at ~2x, at 6.7% training overhead. BBH: baseline 43.8, unconstrained HC 48.9, mHC 51.0.
- DSA (DeepSeek Sparse Attention): Lightning Indexer for sub-linear context scanning, fine-grained token selection, and hybrid attention (CSA + HCA). ~50% lower long-context compute, KV cache at 7% of V3.
- LiveCodeBench 93.5% (highest of any tested model); Codeforces 3206 (vs GPT-5.4's 3168)
- SWE-bench Verified 80.6% (near Claude Opus 4.7's 80.9%); SWE-bench Pro 55.4% (behind Opus 4.7's 64.3%)
- MMLU Pro 87.5 vs GPT-5.5's 88.1
- NIAH @ 1M: 97% vs GPT-5.5's 74% (512K-1M)
- Chatbot Arena 1460 vs GPT-5.5's 1488; GPT-5.5 leads Terminal-Bench 2.0 (82.7%)
- Chinese: SuperCLUE 70.98, ranked first domestically
- Training likely remains mostly on NVIDIA H800 (per MIT Technology Review and other third-party analyses); DeepSeek denies using smuggled Blackwell chips. Inference has fully adapted to Huawei Ascend 950PR.
- Ascend 950PR: 2 PFLOPS FP4 (~2.87x NVIDIA H20), 112GB HiBL memory, 1.4 TB/s bandwidth, SMIC N+3 process; the only commercially available FP4-capable domestic accelerator.
- Seven vendors (Huawei, Cambricon, Hygon, Moore Threads, MetaX, Baidu Kunlun, Alibaba T-Head, Iluvatar CoreX) achieved Day 0 adaptation — the first time domestic chips matched a model launch synchronously.
- Weights still run on NVIDIA GPUs via vLLM/SGLang; cutting NVIDIA would effectively close the model to 99% of global developers.
- Pro: 865GB download (FP4+FP8); Flash: 284B total / 13B active. MIT license, fully commercial.
- OpenAI ChatCompletions- and Anthropic-compatible APIs; context caching discounts repeated tokens.
- Three reasoning modes: Non-think, Think High, Think Max.
- Multimodality lags GPT-5.5 and Gemini 3.1 Pro
- Preview instability, occasional 500 errors; hallucination rate needs Think-mode suppression (SimpleQA-Verified 57.9 vs Gemini's 75.6)
- SWE-bench Pro gap vs Opus 4.7
- GPT-5.5's new tokenizer consumes ~1.46x tokens for identical inputs (per Simon Willison)
- DeepSeek official release & tech report: https://deepseek.ai, https://deepseek.ai/deepseek-v4
- TrendForce (Day 0 adaptation; Ascend 950PR analysis): https://www.trendforce.com/news/2026/04/29/news-huawei-ascend-cambricon-and-hygon-completed-day-0-adaptation-to-deepseek-v4/, https://www.trendforce.com/news/2026/04/07/news-decoding-deepseek-v4-how-huaweis-ascend-950-pr-is-powering-chinas-push-to-break-cuda-dependence/
- The China Academy: https://thechinaacademy.org/why-deepseek-v4-hasnt-fully-cut-ties-with-nvidia/
- Comparative analyses: https://spectrumailab.com/blog/claude-opus-4-7-vs-gpt-5-5-vs-gemini-3-1-pro-vs-deepseek-v4-comparison-2026, https://www.datacamp.com/blog/deepseek-v4-vs-gpt-5-5, https://www.verdent.ai/guides/deepseek-v4-pricing-api-migration-2026
- Architecture explainers: https://anycap.ai/page/en-US/news/deepseek-v4-engram-explained, https://lushbinary.com/blog/deepseek-v4-developer-guide-trillion-parameter-moe-engram/
Benchmarks (as of 2026-04-28)
Pricing
| Model | Input (\(/M) | Output (\)/M) | |---|---|---| | DeepSeek V4-Flash | $0.14 | $0.28 | | DeepSeek V4-Pro | $1.74 | $3.48 | | GPT-5.5 | $5.00 | $30.00 | | Claude Opus 4.7 | $5.00 | $25.00 | | Gemini 3.1 Pro | $2.00 | $12.00 |
V4-Pro's output price is 1/8.6 of GPT-5.5's and 1/7.2 of Opus 4.7's. The launch price carries a 75% discount; the list price is expected around $2.90/$5.80, with further cuts promised once Huawei Ascend 950 supernodes ship at scale in H2.
Hardware ecosystem
Deployment
Known weaknesses
Conclusion
V4 Pro matters not because any single metric is best, but because it simultaneously reaches frontier level on performance, price, and openness. The author's verdict: open-source models can now compete at trillion scale; sparse architecture plus conditional memory is a viable scaling path; and frontier AI pricing power is shifting toward the most engineering-efficient teams. As the post puts it: GPT-5.5's $30/M says frontier AI is a luxury; V4 Pro's $3.48/M says it can be infrastructure.