> *"While everyone chases bigger models, some choose to return to the starting point and build a small house with legally sound bricks—then discover the small house keeps out the wind just fine."*
---
🧱 An Impossible Constraint
Imagine you are an architect. In the AI world of 2026, all your peers build with the same bricks—massive text scraped from the internet. These bricks have murky origins: some copyrighted, some unlicensed, some of unknown provenance. Nobody cares, because more bricks means taller buildings.
Then you impose a constraint on yourself: only bricks with clear origins and explicit licenses.
Your peers think you're crazy. "Legally compliant data" means an order of magnitude less of it. What can a 1B-parameter model trained from scratch on licensed data possibly achieve?
A team led by Peter Schneider-Kamp at the University of Southern Denmark ran this experiment. The result is Mimir v1—a 1B-parameter model using the HRM architecture that, across 20 benchmarks, trails Qwen 3.5 4B by only 0.3 points on English tasks, improves math and code by 36.7% over the original HRM-Text, and sets a new Danish-language state of the art along the way.
This isn't a story about "yet another open model." It's an experiment in how constraints drive innovation.
---
📐 HRM: Not a Transformer, but Hierarchical Reasoning
What first draws attention to Mimir isn't the data strategy but the architecture—it doesn't use a standard Transformer. Instead, it uses the HRM (Hierarchical Reasoning Model).
A standard Transformer reasons "in one straight line": tokens are generated one by one, each step conditioned on everything before it—like a writer rereading the entire draft after every single word.
HRM splits reasoning into two levels:
- Low-level module: rapidly generates candidate content
- High-level module: reviews, corrects, and confirms
- English instruction (13.58B, 19.3%): instruction-following data from Dolci, Tulu 3, Nemotron, etc.
- Math and reasoning (10.40B): problem-solving and logical reasoning
- Sapient mixed category (11.92B, 16.9%): 107 subsets from Flan, Platypus, and Tasksource
- Danish: dedicated Danish instruction and comprehension data
- Other: Q&A, summarization, dialogue, and more
- English tasks: Mimir trails Qwen 3.5 4B by only 0.3 points—a 1B model matching a 4B model.
- Math & code: Mimir improves over HRM-Text 1B by 36.7% (64.1 vs 46.9), trailing only SmolLM3 3B by 3.8%.
- Danish: Mimir sets a new SOTA. No accident—the team is Danish and built dedicated Danish data.
- Math and code still trail SmolLM3 3B (a 3.8% gap). At 1B parameters, reasoning-heavy tasks have a real ceiling.
- The HRM architecture requires FlashAttention support, raising the deployment barrier versus standard Transformers.
- The Danish SOTA partly reflects the team's regional advantage—other small languages may not replicate it.
- Whether 70.4B tokens remains sufficient for larger models is unverified.
- Paper: arXiv:2608.13517
- Model: HuggingFace: danish-foundation-models/DFM-Mimir
- Evaluation tooling: GitHub: UKGovernmentBEIS/inspect_ai
This resembles how humans write—draft first, revise later, with careful review only at key checkpoints.
Mimir uses this architecture at 1B parameters, with PrefixLM (bidirectional attention over the prefix, causal attention for generation) and FlashAcceleration via FlashAttention. It is fully open-sourced on HuggingFace.
---
📊 Data: 161 Datasets, 70.4B Tokens, All "Legal"
This is Mimir's core design decision.
What does "legal data" mean here? In Mimir's context, it refers to permissible post-training data—data with clear authorization for lawful use.
Specifically, Mimir's training mix blends 161 datasets, totaling roughly 70.4B tokens:
A key detail: some of the original Sapient data did not meet the DFM (Danish Foundation Models) data-compliance philosophy. Rather than using it anyway, the team performed "synthetic substitution"—replacing non-compliant originals with 70 Sapient-synth datasets (75M tokens). These synthetic datasets are generate-and-review versions that preserve the original tasks' function while having fully traceable provenance.
It's like discovering a batch of timber with unclear origins: instead of using it, you procure an equivalent batch of fully documented wood based on its dimensions and intended uses. Far more work—but every piece is traceable.
---
📈 Results: A Small Model's Asymmetric Competitiveness
Mimir was compared against three rivals across 20 benchmarks:
| Model | Params | English | Math & Code | Danish | |-------|--------|---------|-------------|--------| | HRM-Text 1B | 1B | baseline | 46.9 | baseline | | DFM Mimir 1B | 1B | near Qwen 3.5 4B | 64.1 | SOTA | | Qwen 3.5 4B | 4B | highest | — | — | | Gemma 4 E2B | ~2B | — | — | — |
Key numbers:
But the most notable pattern is asymmetric competitiveness. On general English tasks, Mimir approaches a model 4× its size. On math and code, it far exceeds its same-architecture predecessor. On Danish, it's outright SOTA.
The lesson: data quality and data compliance are not opposites. Licensed data, carefully curated and proportioned, can match or exceed larger models trained on non-compliant data in specific domains.
---
🔍 Why This Matters
1. Open source is more than releasing weights
Many "open models" open their weights but not their training data, or the data's provenance is unclear—meaning you can neither reproduce nor audit them. Mimir goes further: the full list of 161 datasets, the processing pipeline, and the synthetic substitution scheme are all public. Anyone can inspect, reproduce, and improve.
2. Small languages don't have to be marginalized
Mainstream models are nearly all English-first. Danish (5 million speakers) is a second-class citizen in most models. Mimir was optimized for Danish and set a new SOTA—proving small-language communities don't need to wait for handouts from big labs.
3. The potential of the HRM architecture
HRM is a relatively new architecture. Mimir's results show that at the 1B scale, its "hierarchical reasoning" design can compete with a 4B standard Transformer—empirical support for architectural innovation. Not every road has to be Transformer.
---
⚖️ Honest Limitations
Mimir has shortcomings:
---
🌍 The Bigger Picture
Mimir highlights a neglected question: what is the true "compliance cost" of AI research?
The mainstream narrative says more data is better, provenance be damned. But that narrative assumes you don't plan to publish your data or submit to audit. Once you commit to disclosing data sources, compliance cost becomes a hard constraint. Mimir's experiment shows that under compliance constraints, a 1B model can still produce competitive results. That's not "small is beautiful" self-consolation—it's evidence that constraints breed innovation.
The synthetic substitution scheme is one example. Rather than abandoning non-compliant data, the team rebuilt functional equivalents through a generate-and-review process. Costlier than scraping—but the product is traceable and auditable.
This offers the open-source community a replicable path: not a thought experiment about "what if we trained on legal data," but an empirical report: "we did it, and here are the results."
---
📎 Resources
*When constraints become the starting point of design rather than an afterthought, the results are often surprising. Mimir isn't chasing bigger models—it's proving another path is viable.*