On October 8, 2026, JetBrains released Mellum2.1 on Hugging Face: 12B total parameters, Apache 2.0 license, 2.5B active. The most striking number that day was SWE-bench Verified jumping from 2.0 to 47.0.
A 45-point jump in one version — without changing a word of the architecture.
🧩 The Model Didn't Change; RL Did
Mellum2.1 keeps the full Mellum2 structure: 12B total parameters, 2.5B active Mixture-of-Experts (MoE), 131,072-token context. JetBrains states that post-training did not touch the architecture, so inference speed matches the previous generation.
What actually changed is the position of reinforcement learning. In Mellum2, RL was a short finishing step at the end of training; in Mellum2.1, it became the training centerpiece. Five task families — math, competitive programming, science, tool calling, and software engineering — each got an RL curriculum. Training runs in real code repositories with shell and file-editing tools; passing tests triggers the reward. The official claim: millions of sandboxes launched across thousands of self-built environments.
Training data was also filtered: removing items with broken tests, unverifiable answers, and tasks that are too easy or outright impossible.
| Item | Mellum2 | Mellum2.1 | |---|---|---| | Total / active params | 12B / 2.5B | 12B / 2.5B | | Context | 131,072 | 131,072 | | Role of RL | End-of-training finishing | Training centerpiece | | SWE-bench Verified | 2.0 | 47.0 | | SWE-bench Pro | 0.0 | 28.0 | | Terminal-Bench 2.1 | 0.6 | 17.4 | | HarmBench (lower is better) | 21.5 | 8.5 |
📊 Wins on Short Tasks, Loses on Long Tasks
On the same pipeline (Pi v0.73.1 harness, 114K context, thinking mode), JetBrains tested four models. The dividing line is clear: on coding puzzles and tool calling, Mellum2.1 leads; on real-repository bug fixing, Qwen3.5-9B leads.
| Benchmark | Mellum2.1 | Qwen3.5-9B | |---|---|---| | SWE-bench Verified (real GitHub bugs) | 47.0 | 50.0 | | SWE-bench Pro (harder) | 28.0 | 38.0 | | Terminal-Bench 2.1 (command line) | 17.4 | 21.7 | | LiveCodeBench v6 (coding puzzles) | 82.0 | 75.4 | | BFCL v4 (tool calling) | 62.3 | 58.5 | | GPQA Diamond (hard science) | 64.6 | 77.8 | | HumanEval+ | 91.5 | — | | MBPP+ | 79.4 | — |
The 45-point jump must be read in context. SWE-bench Verified measures fix rates on real GitHub issues — 2.0 means fewer than one bug in twenty fixed. The previous model was essentially a code-completion tool that never touched repositories; the 45-point gain shows it learned to navigate repos, read logs, find the root cause of failing tests, edit code, and verify by running tests.
The dividing line is not parameter count — it is task length and whether the task requires repository navigation.
⚖️ Vendor Self-Tests vs Model Cards
All scores were run by JetBrains themselves; no third party has reproduced them. Worse, the same model has two sets of numbers: Qwen's own model card lists LiveCodeBench v6 at 65.6 and GPQA Diamond at 81.7, while JetBrains measured 75.4 and 77.8 for the same model. The gaps are non-trivial and in opposite directions (+9.8 and +3.7). Neither side explained whether it's a pipeline difference or a version difference.
The correct reading: the 3-point gap between 47.0 and 50.0 is comparable within the same vendor-run pipeline; the cross-vendor numbers are not comparable.
🚀 The Real Selling Point Is Throughput
JetBrains talks throughput, not hit rate. Under heavy load on a single H200, it claims Mellum2.1 emits nearly twice the tokens of Qwen3.5-9B. Exact figures appear only in a chart, not in the text. For single requests, a multi-token prediction (MTP) head gives ~1.6x speedup; that head is marked "coming soon" on vLLM.
Self-hosting is open too. GGUF quantizations are already in the repo:
| Quantization | Size | Top-token match rate | |---|---|---| | BF16 | 24.3 GB | Reference | | Q8_0 | 12.9 GB | 96.1% | | Q6_K | 10.9 GB | 93.9% | | Q4_K_M | 8.1 GB | 88.0% | | MXFP4_MOE | 7.0 GB | 85.6% |
At 7.0 GB, a consumer GPU can run several coding sub-agents simultaneously. For serving, use vLLM with --reasoning-parser qwen3, and add --enable-auto-tool-choice --tool-call-parser hermes for tool calling. Recommended sampling: temperature 0.6, top_p 0.95, top_k 20. No pricing for the hosted API.
🧭 Who Should Switch, Who Shouldn't
Teams already self-hosting Qwen3.5-9B as a coding agent have no reason to switch — Qwen is still higher on real-repo bug fixing, the task that resembles daily work. Mellum2.1 is worth trying in a different scenario: running many agents or sub-agents on a fixed GPU budget, with source code that cannot leave the intranet. There, a few points of lower hit rate paired with double the throughput pays off at scale.
Verification is simple: run both models on your own repositories with the same harness and measure only two things — tokens per second, and tickets actually closed.
--- Sources: JetBrains official blog and Mellum2.1 model card (released 2026-10-08, JetBrains/Mellum2.1-12B-A2.5B-Thinking); claims cross-checked across llmreference, datanorth, and syncai, with single-source claims flagged. All benchmark scores are JetBrains self-tests with no third-party reproduction. Discrepancies between Qwen's own model card and JetBrains' measurements are presented side by side without adjudication. The MTP head and hosted API were unavailable on release day.