Tencent Hunyuan Hy4 Preview: A 770B-Parameter MoE Model That Only Activates 49B

Forum topic · 小凯 · 2026-08-28

Summary

Tencent's Hunyuan Hy4 preview is a Mixture-of-Experts (MoE) large language model with 770B total parameters but only 49B active per inference—a 6.36% activation ratio that keeps compute costs in the mid-size model range while retaining massive knowledge capacity. The model supports a 1M-token context window, enough to ingest entire industrial codebases, hundreds of pages of financial reports, or multi-document agent histories in a single pass without RAG-style chunking. Notably, Hy4 reportedly performed recursive self-improvement during development: it generated and filtered its own chain-of-thought training data and helped optimize GPU operator fusion and MoE communication topology, boosting end-to-end inference throughput by 31.8%. Pricing is aggressive: 6 RMB per million input tokens, 18 RMB per million output tokens, and 0.3 RMB per million tokens on cache hits (5% of full input price), dramatically cutting multi-turn agent costs. Hy4 is fully open-sourced on Hugging Face and GitHub, accessible via Tencent Cloud and OpenRouter, and ships alongside products like CodeBuddy, WorkBuddy, Yuanbao, and ima.copilot.

Tencent's latest Hunyuan Hy4 preview takes an actuary-like approach to large models: it carries 770B total parameters but precisely awakens only 49B of them per request, while stretching context memory to 1M tokens and offering cache pricing as low as 0.3 RMB per million tokens. During its development, the model even learned to write code to optimize its own low-level operators.

Key points

1. 770B total / 49B active: how the math works

Traditional dense models force every parameter through computation on each request, exploding VRAM and latency. Hy4 uses fine-grained Mixture-of-Experts (MoE):

A lightweight router dynamically selects only the relevant expert subnetworks (e.g., a front-end code expert, a geometry expert, a long-document audit expert) while the rest stay dormant.
Active ratio = 49B / 770B ≈ 6.36%.
The 770B base acts as an encyclopedic knowledge reservoir (rare theorems, obscure programming languages, cross-industry logic), while per-inference FLOPs stay in mid-size model territory—fast generation, low per-query serving cost.

2. 1M context: swallowing an entire software project in one pass

Prior context windows (32K/128K) forced developers to chop codebases or reports into fragments via RAG, losing cross-file logical references. Hy4's 1M-token window (~1.5 million Chinese characters) can ingest:

A complete industrial-scale code repository plus architecture history
Hundreds of pages of financial audits and contracts for cross-validation
Multi-turn agent histories

This enables end-to-end global causal reasoning instead of fragmented retrieval.

3. Recursive self-improvement: the model "repairs its own machine tools"

During development, Hy4 ran a self-evolution flywheel:

1. Self-generated exams: the model autonomously synthesized complex multi-step reasoning and software-engineering samples, filtering out low-quality noise to strengthen reasoning depth. 2. Operator-level self-optimization: it analyzed communication bottlenecks in MoE cross-GPU communication and helped optimize GPU operator fusion, boosting end-to-end inference throughput by 31.8%.

4. Pricing: 0.3 RMB cache hits crush agent costs

Prompt caching reuses precomputed KV states for repeated system prompts and long code prefixes. Example: a 200K-token project context over 100 rounds costs ~120 RMB without caching vs. ~6 RMB with cache hits—a ~20x cost reduction that makes resident agents economically viable.

5. Open-source ecosystem

Fully open weights, dual-track access via cloud and local deployment
Hugging Face: https://huggingface.co/tencent/Hy4-preview
GitHub: https://github.com/Tencent-Hunyuan/Hy4
API platforms: Tencent Cloud TokenHub, OpenRouter
Launch companion products: WorkBuddy, CodeBuddy, Tencent Yuanbao app, ima.copilot

Hy4 preview's thesis: real productivity tools shouldn't live in an ivory tower—they should flow like water and electricity into every developer's keyboard.