How a 1B Model Beats a 175B: The 'Densing Law' Behind MiniCPM5-1B
> One-line takeaway: While everyone else is competing over whose GPU cluster is bigger, one team has proven that intelligence lies not in scale but in density.
---
🔥 A Counterintuitive Fact
AIME olympiad-level math reasoning: 40.42 points.
Back in 2023, even GPT-3.5 (175B parameters) might not have reliably scored that. Yet the model that achieved it has only 1B parameters — nearly 200x smaller.
This is not marketing spin. It is MiniCPM5-1B, released in May 2026 by the Tsinghua OpenBMB team (ModelBest / ModelBest Inc.). It ranked first among models under 2B on the Artificial Analysis intelligence index (AA-Index) with 17.9 points, leaving behind Qwen3.5-2B (16.3), which has twice the parameters.
ModelBest calls this phenomenon the Densing Law: the intelligence density of large models roughly doubles every 100 days. If true, in 100 days a 0.5B model could match today's 1B; within a year, the AI in your phone could have the reasoning ability of today's GPT-4-class models.
---
🧠 Data Is Not Fuel — It's a Recipe
Most people still think of model training as "pouring in data." MiniCPM5-1B instead applies tiered data governance.
ModelBest classifies data into five levels (L0 to L4), each with a different cleaning depth and purpose. The core insight is not "more" but "more learnable" — getting more out of every token.
Two key datasets form its data backbone:
| Dataset | Stage | Scale | Core design |
|--------|------|------|---------|
| Ultra-FineWeb-L3 | Pretraining annealing | 600B tokens | World's largest Chinese-language pretraining synthetic dataset: 400B+ English / 200B+ Chinese |
| UltraData-SFT-2605 | Post-training SFT | Tens of millions of samples | First open-sourced SFT dataset in China containing both <think> and non-thinking annotations |
The philosophy behind Ultra-FineWeb-L3 is notable: its goal is not to "make web text readable" but to make web text learnable — transforming raw webpages from information carriers into training material where every entry contributes measurably to model capability.
At the 1B scale, data quality is everything. Low-quality data doesn't just slightly hurt — it directly pollutes the model's cognitive core.
---
⚙️ RL + OPD: Post-Training "Black Magic"
Post-training has three stages: SFT → RL → OPD.
1. SFT: Dual Thinking Modes
First, 200B tokens of "deep thinking" data build reasoning ability; then 200B tokens of "hybrid thinking" data train the model to switch between fast and slow thinking.
An elegant design: one model, one set of weights, with a enable_thinking parameter toggling modes — both an instant-response assistant and a deep reasoner. Not two models; one model that learned two ways of thinking.
2. RL: Learning Not to Ramble
Reinforcement learning is based on DAPO-Math-17k (inspired by the minimalist JustRL approach) with two-stage length scheduling. Results:
- 📈 Average scores on math / code / instruction-following improved by 16 points
- 📉 Redundant outputs hitting the max token budget dropped by 29 percentage points
- Phone assistants stop "sending requests and waiting" — a local brain thinks directly
- Smart homes stop requiring connectivity — they decide even offline
- Multiple devices form "collective intelligence," cooperating without the cloud
- MiniCPM GitHub: https://github.com/OpenBMB/MiniCPM
- Model download (ModelScope): https://modelscope.cn/models/OpenBMB/MiniCPM5-1B
- UltraData datasets: https://github.com/OpenBMB/UltraData
- MiniCPM Tech Report
The model didn't become more verbose — it became more precise. This matters especially on-device, where battery and response time are hard constraints: every wasted token erodes user patience.
3. OPD: Distillation's Final Form
OPD (On-Policy Distillation) is the most technically refined part. Traditional distillation is "the teacher finishes, the student copies answers." OPD is "the student learns in real time alongside the teacher while solving problems" — using reverse KL divergence as the advantage estimate, taking both teacher and student top-k logits at each generation position and computing the difference over their union.
The benefit is direct: no extra distillation data needed; it reuses the in-domain prompts from RL training of the teacher. Engineering-clean, clearly effective. This is ModelBest's improved version of Thinking Machines Lab's original OPD, adding reverse KL optimization.
---
📱 The 0.5GB World: A Deployment Revolution
After INT4 quantization, MiniCPM5-1B's weights are only 0.5GB — smaller than a cached short video on your phone.
Three deployment tiers cover nearly every scenario:
| Precision | Weights | Use case | |------|------|---------| | FP16 | ~2GB | GPU / high-end laptops, zero quantization loss | | INT8 | ~1GB | Ordinary laptops / edge boxes, near-lossless | | INT4 | ~0.5GB | Phones, tablets, vehicles, near-lossless |
Even bolder: ArcLight, ModelBest's in-house pure-CPU inference framework. No GPU required. Combined with WebAssembly, it runs in the browser — zero install, works offline.
What does this mean? AI shifts from "cloud service" to "local capability." Requests are processed on-device, responses are instant, data never leaves the device. No network latency, no privacy anxiety, no "server busy, please try again."
---
🏛️ A "Cognitive Core": The Endgame of a Different Path
North American giants follow "bigger is better" — GPT-4, Claude, Gemini, stacking parameters into the hundreds of billions with reasoning costs amortized in the cloud. ModelBest's path is "good enough is enough" — squeezing 90% of the capability into 1% of the parameters so AI runs on the device in everyone's hand.
This isn't compromise — it's a different endgame. When on-device models get strong enough, interaction changes fundamentally:
During its open-source week, ModelBest also released PilotDeck, an "agent operating system," and a desktop pet powered by MiniCPM5-1B. The pet looks like a toy, but it's a demonstration: 1B parameters can drive an on-device app with continuous memory and personality.
The desktop pet is just the beginning. The real vision is a decentralized, self-organizing intelligence network — device agents, software agents, and robot agents working together in a self-coordinating ecosystem.
---
🎯 Conclusion: The Compounding of the Densing Law
MiniCPM5-1B is the latest validation of the Densing Law. It proves one thing:
> The parameter race is not AI's only path. With sufficiently refined data governance, smart enough post-training, and pragmatic deployment, a 1B model can achieve far more than intuition suggests.
ModelBest also did something longer-term: using AI to build AI. MiniCPM5-1B's base model was trained with ForgeTrain — a training framework written entirely by AI, running 10% faster than NVIDIA's own Megatron on H100s.
When large models can forge their own underlying tooling, software R&D and adaptation costs approach zero, and large models truly become affordable, universal technology for every industry.
The Densing Law's compounding is just beginning. Doubling every 100 days sounds slow, but compounding never hurries. In 100 days, a 0.5B model may match today's 1B; in a year, the device in your pocket may carry the reasoning power of today's cloud models.
By then, AI competition won't be about whose GPU cluster is bigger — it will be about whose devices are smarter.
---
References