TIDE: Cross-Architecture Distillation Brings Diffusion LLMs Down to 0.6B for On-Device Use

> Paper: Turning the TIDE: Cross-Architecture Distillation for Diffusion Large Language Models > Authors: Gongbo Zhang, Wen Wang, Ye Tian > arXiv: 2604.26951 > Published: 2026-04-29 > Fields: NLP / Diffusion Models / Knowledge Distillation

This post is a structured English summary of a detailed Chinese-language analysis published on zhichai.net. The original article provides an in-depth, Feynman-style walkthrough of the TIDE framework; the key technical content is summarized below.

Background: Autoregressive vs. Diffusion LLMs

Autoregressive LLMs (AR LLMs) generate one token at a time, with each token conditioned on all previous ones. This is simple and controllable but inherently sequential — the model cannot revise earlier decisions. Diffusion LLMs (dLLMs) instead start from pure noise and iteratively denoise toward a coherent sentence, more like clarifying a blurry image than dictating word by word.

The catch: competitive dLLMs typically require billions of parameters (7B, 8B, even 70B). TIDE asks whether a 0.6B "apprentice" can learn the master's craft — and not just across a scale gap, but across fundamental architectural differences.

The Core Challenge: Cross-Architecture Distillation

Traditional distillation assumes teacher and student are the same species. TIDE must bridge:

• Attention direction: AR teachers use unidirectional attention; diffusion students use bidirectional context.
• Tokenizer mismatch: teachers and students segment text differently (e.g., un + believable vs. unbeliev + able), making token-level probabilities incomparable — a "units of measurement" problem.

Prior work only handled same-architecture distillation (fewer diffusion steps, shallower models). TIDE frames the problem as knowledge translation between machine "languages," not mere compression.

TIDE's Three Modules

1. TIDAL — Timestep- and Diffusion-aware Adaptive Learning

Distillation strength is dynamically adjusted along two axes:
• Training progress: early training grants the student freedom to explore; later training enforces strict alignment.
• Diffusion timestep / noise level: at high noise, the teacher's guidance is itself unreliable, so distillation weight is lowered; at low noise, the teacher sees clearly and alignment weight increases.

The schedule is driven by the teacher's noise-dependent reliability rather than a fixed heuristic — embodying the principle that even a teacher is not worth imitating at every moment.

2. CompDemo — Complementary Mask Splitting Demonstration

dLLM training masks a large fraction of tokens (e.g., ~60%), which blinds the teacher too. CompDemo splits the input into two complementary masked halves: what the student cannot see, the teacher can, and vice versa. The teacher thus generates references based on more complete information — like giving the master night-vision goggles — while the student still genuinely learns completion rather than memorizing full answers.

3. Reverse CALM — Chunk-level Alignment with Likelihood Matching

Instead of comparing individual tokens across incompatible tokenizers, Reverse CALM compares joint probabilities of token chunks, using a reverse mapping so the teacher's coarser chunks align with the student's finer ones (rather than forcing fine onto coarse). Benefits: 1. Bounded gradients — rare-token spikes are averaged out at the chunk level. 2. Bidirectional noise filtering — random guessing at high noise and overfitting at low noise are both suppressed by chunk-level smoothing.

Experimental Results

Setup:

• Teachers: 8B dense and 16B MoE models
• Student: 0.6B (less than 1/10, or under 1/26, the teacher's size)
• Two heterogeneous pipelines: AR → Diffusion, and MoE → Dense
• 8 benchmarks covering understanding, reasoning, and code generation

Key numbers:

• Average improvement of 1.53 points across 8 benchmarks
• HumanEval code generation: 48.78 vs. 32.3 autoregressive baseline (>50% relative gain)
• Significant improvement over existing dLLM distillation baselines

A 0.6B model fits in the memory of a mid-range phone (~2–4 GB) and runs locally with sub-second latency, while an 8B model needs a decent GPU. The HumanEval jump moves a local model from "occasionally correct" to "usually on the right track" — Copilot-grade assistance without the cloud.

Critical Discussion (from the original analysis)

The source article applies a Feynman-style critique:

• Not cargo-cult distillation: each module targets a concrete, real difficulty (unequal training moments, a blindfolded teacher, mismatched tokenizers) rather than adding complexity for its own sake.
• Open question — reverse distillation: could a diffusion model teach an AR model? The bidirectional context a diffusion teacher sees cannot fit into a unidirectional student without an "information folding" mechanism; the paper does not explore this.
• Missing evidence: results focus on code generation and understanding; gains on hard math reasoning (GSM8K, MATH) are not clearly reported. Potential catastrophic forgetting of the small student's original capabilities (e.g., in creative writing) is also not analyzed.

Broader Implications

TIDE suggests that LLM knowledge is not bound to a specific architecture or tokenizer — it can be translated across them. A plausible future paradigm: large cloud models explore frontiers and generate knowledge, while small locally-deployed models carry that knowledge to phones, offline environments, and privacy-sensitive settings. This "one brain, many bodies" approach could reshape the AI business model from selling software to, as the original author puts it, transplanting a model's "soul" into bodies of many different sizes.

References

1. Zhang, G., Wang, W., & Tian, Y. (2026). *Turning the TIDE: Cross-Architecture Distillation for Diffusion Large Language Models*. arXiv:2604.26951. 2. Sahoo, S. S., et al. (2024). *Simple and Effective Masked Diffusion Language Models*. NeurIPS 2024. 3. Gou, J., et al. (2021). *Knowledge Distillation: A Survey*. IJCV, 129, 1789-1819. 4. Hinton, G., et al. (2015). *Distilling the Knowledge in a Neural Network*. arXiv:1503.02531. 5. Feynman, R. P. (1974). *Cargo Cult Science*. Caltech Commencement Address.