Qwythos-9B: A 9B Model Trained on Claude Mythos Reasoning Trajectories, Running in 4GB VRAM with 1.04M-Token Context

> TL;DR: Qwythos-9B is an open-source reasoning model based on the Qwen3.5-9B architecture, post-trained on 500M+ high-quality Claude Mythos/Fable reasoning trajectories. It extends context to 1.04M tokens via YaRN, supports native function calling and vision, and its Q4_K_M quantized version runs in just 4GB VRAM. Its significance: reasoning *style* is transferable—you don't need hundreds of billions of parameters to learn Claude-level chain-of-thought patterns.

Link: https://ollama.com/richardyoung/qwythos-9b-abliterated

Why Can "Reasoning Style" Be Distilled?

The traditional view: model capability = parameter count × training data. But distillation experiments (DeepSeek-R1 to smaller models) revealed a counterintuitive fact:

> "Knowledge" and "reasoning patterns" may be relatively independent components of model capability.

• Knowledge (world facts, language patterns): genuinely requires many parameters to store
• Reasoning patterns (how to think, decompose problems, verify): more like a "program" that can be learned and transferred

Qwythos-9B's core insight: use Claude Mythos's *thinking process* as training data, teaching a small model *how to think* rather than *what to know*.

• Traditional fine-tuning: teaches the model the correct answer
• Reasoning distillation: teaches the model the thought process that leads to the answer

Technical Breakdown

Reasoning Trajectories

The training data isn't simple Q&A pairs but full chains of thought, e.g., proving 1+2+...+n = n(n+1)/2 with:

• Self-questioning ("let me check that algebra step again...")
• Step-by-step decomposition
• Backtracking and verification
• Metacognition (choosing induction before executing)

500 million such complete trajectories form the training set.

The "Abliterated" (Uncensored) Qwen3.5-9B Base

Standard models undergo safety alignment, which can cause over-refusal and reduced reasoning freedom. The uncensored base removes these constraints, enabling more open-ended reasoning—important for learning Claude Mythos's style.

Presumed Training Pipeline

The blog doesn't detail methods, but based on industry practice:

1. SFT on reasoning trajectories: input = question, target = full reasoning + answer 2. DPO/RLHF: ranking multiple reasoning paths, reinforcing self-verification, backtracking, multi-angle thinking 3. YaRN long-context extension: adjusting RoPE position encoding to extend 32K/128K native context to 1.04M tokens without full retraining

Benchmark Interpretation

| Benchmark | Gain | Interpretation | |---|---|---| | MMLU | +34 pts | Large knowledge-comprehension gain | | GSM8K Strict | +30 pts | Learns more rigorous chain-of-thought | | GSM8K Flex | +19 pts | Progress on creative problem-solving |

The gains likely combine implicit knowledge injection and reasoning-pattern transfer. The large GSM8K Strict gain (+30) suggests reasoning-pattern migration is effective—training on plain Q&A alone would likely show much smaller multi-step reasoning gains.

1.04M Context via YaRN

Transformers use RoPE for positional encoding. Beyond the trained position range, performance degrades. YaRN:

1. Interpolates new positions into the trained range 2. Adjusts attention temperature to counter flattened attention scores 3. NTK-aware extension: high-frequency dimensions extrapolate, low-frequency dimensions interpolate

Practical reality vs. theory: the author admits 1.04M tokens is a theoretical ceiling; 32K–128K covers most use cases. Attention dilution, reduced retrieval precision, and KV-cache memory (tens of GB at 1M context) are real constraints. So Q4_K_M + 16K context on 4GB VRAM is sensible.

Function Calling + Vision

• Native function calling (Qwen3.5 standard): direct tool use (Python, web search, databases), no adapter layer, agent-framework friendly, works offline, keeps data local
• Vision: image analysis, OCR, chart understanding—all locally, no cloud upload

Limitations and Honest Boundaries

| Aspect | Reality | |---|---| | Scale | 9B can't compete with 70B/400B models on complex reasoning | | Knowledge ceiling | Distilled reasoning patterns, not Claude's full knowledge | | Safety | Uncensored version has no safety alignment—may output harmful content | | Long context | Attention precision degrades beyond trained context length | | Quantization | Q4_K_M loses some precision on complex tasks | | Data provenance | How 500M Claude trajectories were obtained is undisclosed—possibly violating API terms of service |

Distillation or copying? Technically legitimate model distillation, but if data came from Claude API outputs, it may violate Anthropic's usage terms—a commercial/legal gray area.

The Broader Trend

1. Reasoning style is transferable: expect small specialist models tuned to specific reasoning styles (math, code, creative writing) 2. Local deployment is viable: a "thinking" model on 4GB VRAM means no cloud API, no subscription, no privacy concerns 3. Long context as standard: 1M context is no longer exclusive to GPT-4/Claude-class models 4. Unresolved questions: legality of distillation, small-model knowledge ceilings, closing the gap between theoretical and effective context use

Qwythos isn't a breakthrough but a signal: AI democratization is accelerating.

Appendix: Deployment Reference

| VRAM | Quantization | Context | |---|---|---| | 4GB | Q4_K_M | 16K | | 6GB | Q5_K_M | 32K | | 8GB | Q6_K | 64K | | 12GB | Q8_K_M | 128K | | 16GB | BF16 | 256K | | 24GB | MTP-BF16 | 512K |

Supported frameworks: llama.cpp, llama-server, OpenWebUI, Cherry Studio, OpenClaw