ModelScope (BAAI) and Tsinghua University's OpenBMB team released MiniCPM5-1B (2026-05-26), a 1.08B-parameter model trained with ForgeTrain, a training framework generated entirely by AI. Links: GitHub: OpenBMB/MiniCPM, HuggingFace, ModelScope.
Key points
- Performance: AA-Index 17.9 (best under 2B), overall average score 42.57, beating the best same-size baseline (35.61) and the 2B-class Qwen3.5-2B with half the parameters. Largest advantages in Agent tool use, code generation, and competition math.
- Context: 131K tokens, far beyond Qwen3.5-0.8B (32K) and Gemma3 1B (32K). INT4 quantized size is 0.5GB. Single checkpoint with Think/No-Think switching.
- ForgeTrain (AI-written training framework): humans only set the goal (e.g., hardware, precision, target throughput); AI generates, tests, profiles, and iterates code in a closed loop. Reported ~10% faster than Megatron on H100 and ~10% faster than MindSpeed on Ascend; Ascend adaptation in 3–5 days with zero human code. The rationale: it is "forged" for specific hardware/model/scale, with no compatibility burden.
- Data (UltraData): Ultra-FineWeb (base pretraining), Ultra-FineWeb-L3 (three-tier governance, over 1T tokens, claimed among the largest open Chinese synthetic pretraining corpora), UltraData-Math, UltraData-SFT-2605 (200B + 200B tokens).
- SFT: 200B tokens deep-thinking + 200B tokens hybrid-thinking.
- RL: domain-specific teacher models (math via DAPO-Math-17k Reasoning RL, two-stage length scheduling; multi-source data including TriviaQA, NQ-Open, LongWriter).
- OPD (Online Policy Distillation): distills RL teachers back into a single release model using reverse-KL advantage estimation, top-k logits joint computation, and reuse of the teacher's in-domain prompts. RL+OPD yields +16 points average on math/code/instruction following and −29 percentage points overlong responses.
- 7 inference backends: vLLM, SGLang, llama.cpp, Ollama, LM Studio, MLX (4-bit Apple Silicon), ArcLight.
- 5 fine-tuning frameworks: TRL+PEFT, LLaMA-Factory, ms-swift, unsloth, xtuner, each with Cookbooks.
- Agent Skill system for Cursor/Claude Code, e.g.
@minicpm5-deploy serve openbmb/MiniCPM5-1B with vLLM on port 8000. - 9 chip architectures via FlagOS: NVIDIA, Hygon, MetaX, Iluvatar, Enflame, Moore Threads, Kunlunxin, Ascend, ARM-v9.
- Bonus: open-sourced MiniCPM-Desk-Pet, a local-LLM desktop pet (Electron, LoRA personality switching, co-operates with coding agents).
- MiniCPM GitHub
- UltraData Tech Report
- Rethinking OPD
Training recipe
Three stages: Base Training → Mid Training → Post Training (SFT → RL → OPD).
Deployment ecosystem
Self-evolution flywheel
BAAI frames the release as a step toward an "AI builds AI" loop: stronger models generate better ForgeTrain versions and filter purer data, which trains stronger next-generation models. On an L1–L5 self-improvement ladder, the team claims L1–L3 are complete (human-designed → AI-generated with human review) and is pushing toward L4 (fully self-improving closed loop). The MiniCPM series has evolved from MiniCPM-2B (2024.02) through MiniCPM4/4.1 and MiniCPM-SALA to MiniCPM5-1B, consistently prioritizing intelligence density over parameter count.
References