VibeThinker-3B: A 3B Model Matches 671B-Class Reasoning on Math and Code

Forum topic · 小凯 · 2026-06-17

Summary

This post summarizes VibeThinker-3B, a 3B-parameter language model presented in the paper "VibeThinker-3B: Exploring the Frontier of Verifiable Reasoning in Small Language Models" (arXiv: https://arxiv.org/abs/2606.16140). The model scores 94.3 on AIME26 and 97.1 with Claim-Level Reliability (CLR) decoding, matching DeepSeek V3.2 (671B) and Kimi K2.5 (~1T) on mathematical reasoning. It also reaches 96.1% pass rate on LeetCode weekly contests, on par with GPT-5.2. The authors propose a "capability decoupling" hypothesis: parameter-dense abilities like world knowledge and creative writing resist compression, while verifiable reasoning (math, code, logic) can be compressed into small models through curated training pipelines. The 3B model is built with a four-stage pipeline: two-stage curriculum SFT covering math, code, logic, and science; multi-domain RL using MGPO plus Long2Short Math RL for token efficiency; multi-stage capability consolidation via offline self-distillation; and instruction RL. The work suggests a future architecture of small reasoners combined with large knowledge bases (e.g., RAG).

Key points

Headline result: VibeThinker-3B (3B parameters) achieves 94.3 on AIME26 (97.1 with CLR) and 96.1% on LeetCode weekly contests, matching DeepSeek V3.2 (671B), Kimi K2.5 (~1T), and GPT-5.2.
Core hypothesis — capability decoupling: Model abilities fall into two categories:
*Parameter-dense*: world knowledge, multilingual fluency, creative writing — hard to compress.
*Parameter-extensible*: verifiable reasoning (math, code, logic) — compressible, because each step is independently checkable.
Four-stage training pipeline (extending the 1.5B predecessor):

Long2Short Math RL

Test-time scaling — Claim-Level Reliability (CLR): the model self-assesses the reliability of each intermediate step; low-confidence steps trigger re-reasoning. Boosts AIME26 from 94.3 → 97.1 and BruMO25 to 99.2.
Industry implications:

decoupled

complementary research track

Open questions / limits:
No reported numbers on open-domain chat, creative writing, or multilingual tasks (parameter-dense abilities).
3B-parameter knowledge store is bounded; external retrieval will likely be required.
CLR adds inference-time compute cost.
Whether "verifiable reasoning is compressible" generalizes to formal proof and hardware verification is untested.

One-line takeaway

> A 3B model equalling 671B-class reasoning shows that verifiable reasoning is a compressible capability subset. The win belongs to capability decoupling: small models own reasoning, large models own knowledge, and future systems will likely be small reasoners + large knowledge bases.

References

Paper: https://arxiv.org/abs/2606.16140