GRN: Generative Refinement Networks for Visual Synthesis
> TL;DR: ByteDance's GRN unifies image/video generation by fixing the two weaknesses of autoregressive models: a Hierarchical Binary Quantization (HBQ) tokenizer that matches continuous VAE quality (rFID 0.56), and a global refinement mechanism that lets the model revise earlier predictions like a painter correcting a canvas. Combined with entropy-guided adaptive-step sampling, GRN achieves gFID 1.81 on ImageNet with only 2B parameters.
Core insight: generation is not one-shot writing - it is repeated revision.
Paper info
| Field | Detail | |---|---| | Title | Generative Refinement Networks for Visual Synthesis | | Authors | Jian Han, Jinlai Liu, Jiahuan Wang, Bingyue Peng, Zehuan Yuan | | Institution | ByteDance Research | | arXiv | 2604.13030 (2026-04-14) | | Code | https://github.com/MGenAI/GRN | | Models | https://huggingface.co/bytedance-research/GRN |
Background: diffusion vs. autoregressive
Diffusion models (DiT, FLUX, Stable Diffusion, Sora, Wan) apply the same computational budget to every sample - a simple prompt ("a red apple") and a complex one both take 50 denoising steps. Because they optimize MSE on a velocity field without explicit likelihood, they cannot tell how "hard" a sample is.
Autoregressive models (LlamaGen, VAR, MaskGIT) have variable-length likelihood and are naturally complexity-aware, but suffer two flaws:
1. Weak discrete tokenizers - VQ quantization error makes reconstruction far worse than continuous VAEs. 2. Irreversible error accumulation - once a token is generated, it can never be revised. No eraser, like drawing in pen.
GRN merges the strengths of both: variable-length computation, high-quality discrete tokens, and global revision.
Technique 1: Hierarchical Binary Quantization (HBQ)
Instead of enlarging the codebook (as Infinity/BitDance do), HBQ quantizes hierarchically - approximating each feature value bit-by-bit like a binary fraction. Given feature F compressed to (-1, +1):
- Round 1:
q_1 = 1[F > 0](highest bit) - Round i: threshold
c_i = sum of previous bits, thenq_i = 1[F > c_i] - Reconstruction:
F_hat = sum_j delta[q_j] * 2^-j - "Writing vs. painting": NLP's causal next-token paradigm treats generation like writing a draft. GRN argues vision is a 2D field where global context matters more than causal order - though AR loss remains a useful training inductive bias.
- HBQ's deeper meaning: discrete representations are not inherently inferior - the discretization method matters. HBQ's soft binary approximation removes the quality bottleneck blocking unified text-visual token spaces.
- Metacognition: entropy-based self-monitoring at inference lets the model allocate compute by difficulty - something MSE-trained diffusion cannot do.
- Largest model is 2B, behind 14B+ SOTA; video samples occasionally lack fine detail; each step predicts the full token map, raising per-step cost (mitigated, not eliminated, by adaptive sampling).
- Future directions: unification with LLM vocabularies, step distillation, dynamic resolution, 3D/4D extension, and competition with Transfusion-style architectures - the paper closes by noting "GRN has the potential to emerge as a strong competitor to the currently dominant Transfusion architecture."
- Han et al. (2026). *Generative Refinement Networks for Visual Synthesis*. arXiv:2604.13030
- Ji et al. (2025). *JiT: Back to Pixel-Level Purity*
- Tian et al. (2024). *Visual Autoregressive Modeling*. NeurIPS 2024
- Chang et al. (2022). *MaskGIT*. CVPR 2022
- Han et al. (2025). *Infinity: Bitwise Visual Tokenizer*
- Wan Team (2025). *Wan 2.1*
Quantization error decays exponentially: |e_j| < 1/2^j, so M=8 rounds gives error < 1/256.
Results (ImageNet 256x256 reconstruction): HBQ (M=4) achieves rFID 0.56 at 96x compression with 16 latent channels - beating SD-VAE (0.87 at 24x), VAR (0.85), and LlamaGen (2.19). For the first time a discrete tokenizer matches or exceeds continuous VAEs at the same channel count.
The multi-round coarse-to-fine process is analogous to Haar wavelet multi-resolution decomposition.
Technique 2: Global Refinement - giving AR an eraser
GRN formalizes painter-like revision. At each iteration:
1. State: a hybrid map F_t = S_t * Y_t (+) (1-S_t) * Y_rand mixing predicted tokens with random "blank canvas" tokens.
2. Predict: the Transformer sees the full map and predicts the complete next token map Y_t+1.
3. Update: a new random mask S_t+1 keeps a fraction l_t+1 of tokens; the rest are replaced by fresh predictions or reset to random tokens.
This unifies three operations: filling (blank areas get new content), refining (kept areas get better predictions), and erasing (kept areas can be reset and repainted). Training is a simple "corrupt-and-reconstruct" objective: mix ground truth with random tokens, predict the full ground truth with cross-entropy.
Ablations confirm necessity: replacing global refinement with a mask-based AR scheme collapses FID from 3.63 to 185.62 (best tuned mask variant: 18.13). Counter-intuitively, confidence-based token sampling (FID 10.64) performs far worse than random sampling (3.63), because it breaks the uniform-mixture distribution seen in training.
Technique 3: Entropy-guided complexity-aware sampling
GRN measures prediction uncertainty via normalized entropy H(Y_t) and schedules the keep-ratio accordingly: low-entropy (easy) samples converge in fewer steps; high-entropy (hard) samples get more refinement steps. After a 5-step warm-up, 62.7% of ImageNet samples finish before 50 steps (~200 samples at the 20-step minimum), with FID only rising from 3.6 to 3.8.
Benchmark results
Class-conditional ImageNet 256x256:
| Model | Type | Params | FID | |---|---|---|---| | DiT-XL/2 | Diffusion | 675M | 2.27 | | JiT-G/16 | Flow | 2B | 1.82 | | VAR-d30 | AR | 2B | 1.92 | | GRN-G | AR | 2B | 1.81 |
GRN beats VAR-d30 at equal size, showing global refinement mitigates AR error accumulation.
Text-to-image (GenEval): GRN (2B) scores 0.76, beating SD3 Medium (0.62) and Infinity (0.71) at the same scale.
Text-to-video (VBench): GRN (2B) scores 82.99, surpassing CogVideoX-5B (81.61) and Emu3-8B (80.96), trailing only Wan 2.1 (14B) and InfinityStar (8B).
Bit vs. index prediction: nearly equal at small scale; bit prediction wins at large scale (2.47 vs 2.64 FID for L-size) with fewer video artifacts.