> One-line takeaway: While everyone else competes over GPU cluster size, one team has proven that intelligence lies not in scale, but in density.
---
🔥 A Counterintuitive Fact
AIME Olympiad-level math reasoning: 40.42 points.
Back in 2023, even GPT-3.5 (175B parameters) couldn't reliably match that score. The model that achieved it has just 1B parameters—nearly two hundred times smaller.
This is not marketing spin. It's MiniCPM5-1B, released in May 2026 by Tsinghua's OpenBMB team (ModelBest), which topped sub-2B models on the AA-Index (Artificial Analysis Intelligence Index) with 17.9 points, leaving behind the twice-larger Qwen3.5-2B (16.3 points).
ModelBest calls this the Densing Law: the intelligence density of large models roughly doubles every 100 days.
If true, in 100 days a 0.5B model could match today's 1B. In a year, the AI in your phone might reason like today's GPT-4.
---
🧠 Data Is Not Fuel, It's a Recipe
Most people still think of training as "pouring in data." MiniCPM5-1B instead applies tiered data governance.
Data is classified into five levels, L0 to L4, each with different cleaning depth and usage. The core insight is not "more" but "learnable"—extracting more from every token.
Two key datasets form the data backbone:
| Dataset | Stage | Scale | Core Design |
|--------|------|------|---------|
| Ultra-FineWeb-L3 | Pretraining annealing | 600B tokens | World's largest Chinese pretraining synthetic dataset; 400B+ English / 200B+ Chinese |
| UltraData-SFT-2605 | Post-training SFT | Tens of millions | First open-sourced Chinese SFT dataset with both <think> and non-thinking annotations |
The design philosophy of Ultra-FineWeb-L3 is especially notable: the goal is not "making web text readable" but "making web text learnable"—transforming raw pages into training material where every item contributes measurably to model capability.
At the 1B scale, data quality is everything. Low-quality data doesn't just slightly degrade performance—it directly pollutes the model's cognitive core.
---
⚙️ RL + OPD: Post-Training "Black Magic"
Post-training follows three steps: SFT → RL → OPD.
1. SFT: Dual Thinking Modes
First, 200B tokens of "deep thinking" data build reasoning ability; then 200B tokens of "hybrid thinking" data train the model to switch between fast and slow thinking.
The elegant part: the same model, the same weights, toggled via the enable_thinking parameter—an instant-reply assistant and a deep reasoner in one. Not two models, but one model with two modes of thought.
2. RL: Learning Not to Ramble
Reinforcement learning is based on DAPO-Math-17k (inspired by JustRL's minimalist approach), with two-stage length scheduling.
Results are impressive:
- 📈 Average scores on math/code/instruction-following improved by 16 points
- 📉 Redundant output hitting the max token budget dropped by 29 percentage points
- Phone assistants stop "sending requests and waiting"—a local brain thinks directly
- Smart homes no longer need connectivity to obey—they decide even offline
- Multiple devices can form "swarm intelligence," collaborating without the cloud
- MiniCPM GitHub: https://github.com/OpenBMB/MiniCPM
- Model download (ModelScope): https://modelscope.cn/models/OpenBMB/MiniCPM5-1B
- UltraData datasets: https://github.com/OpenBMB/UltraData
- MiniCPM Tech Report
The model became more precise, not more verbose—critical for edge scenarios where battery and latency are hard constraints. Every wasted token erodes user patience.
3. OPD: The Ultimate Form of Distillation
OPD (On-Policy Distillation) is the most technically refined part. Traditional distillation is "the teacher finishes, the student copies." OPD is "the student learns in real time alongside the teacher as it solves problems"—using reverse KL divergence as the advantage estimate, sampling both teacher and student top-k logits at each generation position and computing the difference over their union.
The benefit is direct: no separate distillation data needed—just reuse the in-domain prompts from the teacher's RL training. Clean engineering, significant gains. This is ModelBest's improved version of Thinking Machines Lab's original OPD, adding reverse KL optimization.
---
📱 The 0.5GB World: A Deployment Revolution
With INT4 quantization, MiniCPM5-1B's weights are only 0.5GB—smaller than a cached short video on your phone.
Three deployment tiers cover nearly every scenario:
| Precision | Weights | Use Case | |------|------|---------| | FP16 | ~2GB | GPU / high-end laptops, zero quantization loss | | INT8 | ~1GB | Standard laptops / edge boxes, near-lossless | | INT4 | ~0.5GB | Phones, tablets, vehicles, near-lossless |
Even better: ArcLight, ModelBest's in-house pure-CPU inference framework. No GPU required. Combined with WebAssembly, it runs in the browser—zero install, works offline.
What does this mean? AI shifts from "cloud service" to "local capability." Requests are processed on-device, responses are instant, data never leaves the device. No network latency, no privacy anxiety, no "server busy, please wait."
---
🏛️ "Cognitive Core": The Endgame of a Different Path
North American giants pursue "bigger is better"—GPT-4, Claude, Gemini, parameters in the hundreds of billions, inference costs amortized in the cloud. ModelBest's path is "good enough is enough"—cramming 90% of capability into 1% of the parameters, running AI on the device in everyone's hand.
This is not a compromise—it's a different endgame.
When edge models get strong enough, interaction transforms fundamentally:
During its open-source week, ModelBest also released PilotDeck, an "agent operating system," plus a desktop pet powered by MiniCPM5-1B. The pet looks like a toy, but it's really a demo: 1B parameters can drive an on-device app with continuous memory and personality-driven responses.
The pet is just the beginning. The real vision is a decentralized, self-organizing intelligence network—terminal agents, software agents, and robot agents collaborating autonomously.
---
🎯 Summary: The Compounding of the Densing Law
MiniCPM5-1B is the latest validation of the Densing Law, proving one thing:
> The parameter race is not AI's only path. With refined data governance, smart post-training, and pragmatic engineering, a 1B model can far exceed intuition.
ModelBest also did something longer-term: using AI to build AI. MiniCPM5-1B's base model was trained with ForgeTrain—a training framework written entirely by AI, running 10% faster than NVIDIA's own Megatron on H100s.
When large models can autonomously forge their own infrastructure, R&D and adaptation costs approach zero, and large models become truly affordable, universal technology.
The Densing Law's compounding has just begun. Doubling every 100 days sounds slow, but compounding never hurries. In 100 days, 0.5B may match today's 1B; in a year, the device in your pocket may reason like today's cloud models.
By then, AI competition will no longer be "whose GPU cluster is bigger"—it will be whose device is smarter.
---
References