GRN: Generative Refinement Networks — Autoregressive Visual Synthesis That Revises Like a Human Painter
> TL;DR: ByteDance proposes GRN, a unified image/video generation paradigm. It uses Hierarchical Binary Quantization (HBQ) to break the reconstruction bottleneck of discrete tokenizers (rFID 0.56, matching continuous VAEs for the first time), and a global refinement mechanism that gives autoregressive models an "eraser" — the ability to revisit and fix earlier mistakes instead of the traditional AR "no turning back" approach. Combined with entropy-guided adaptive-step sampling, GRN achieves a new record gFID of 1.81 on ImageNet with only 2B parameters, and beats diffusion models on T2I and T2V at comparable scale. Core insight: generation is not one-shot writing — it is repeated revision.
Paper Info
| Field | Detail | |-------|--------| | Title | Generative Refinement Networks for Visual Synthesis | | Authors | Jian Han, Jinlai Liu, Jiahuan Wang, Bingyue Peng, Zehuan Yuan | | Affiliation | ByteDance Research | | arXiv | 2604.13030 | | Code | https://github.com/MGenAI/GRN | | Models | https://huggingface.co/bytedance-research/GRN |
Background: The Diffusion vs. Autoregressive Dilemma
Diffusion models (DiT, FLUX, Stable Diffusion, Sora, Wan) dominate visual generation but spend the same compute on every sample — a trivial prompt and a complex one both take 50 denoising steps. Because they optimize a velocity field with MSE, they lack explicit likelihood and cannot tell how "hard" a sample is.
Autoregressive models (LlamaGen, VAR, MaskGIT) have variable-length likelihoods and are naturally complexity-aware, but suffer two critical flaws:
1. Poor discrete tokenizers — VQ quantization error makes reconstruction far worse than continuous VAEs (e.g., SD-VAE). 2. Irreversible error accumulation — once early tokens are wrong, all later tokens build on that error. Like drawing with a ballpoint pen: no eraser.
GRN fuses the strengths of both: variable-length, complexity-aware compute; high-quality discrete tokens; and the ability to revise.
Technique Breakdown
1. Hierarchical Binary Quantization (HBQ)
Instead of enlarging the codebook (as Infinity/BitDance do), HBQ performs hierarchical quantization, approximating each feature value like a binary fraction.
Given VAE-encoded features \(F \in (-1, +1)\) (compressed via tanh):
- Round 1: \(c_1 = 0,\ q_1 = \mathbb{1}[F > c_1]\) (most significant bit)
- Round \(i\): \(c_i = \sum_{j=1}^{i-1} \frac{\delta[q_j]}{2^j},\ q_i = \mathbb{1}[F > c_i]\)
- Reconstruction: \(\hat{F} = \sum_{j=1}^{M} \delta[q_j] \cdot 2^{-j}\)
- \(Y_t\): current predicted token map; \(Y_{rand}\): random tokens ("blank canvas"); \(S_t\): binary selection mask.
- Global refinement is essential: a mask-based variant collapses (FID 185.62; best-tuned still 18.13 vs. 3.63).
- Random token sampling beats confidence-based sampling (3.63 vs. 10.64) — counterintuitive, but confidence-based selection breaks the training-time uniform mixture distribution.
- Bit prediction overtakes index prediction at larger model scale (2.47 vs. 2.64 FID for L-size).
- Generation as painting, not writing: GRN challenges the assumption that causal order is essential for visual generation — images are 2D fields, not linear sequences; global context matters more than causal ordering during inference (though AR loss remains an effective training inductive bias).
- HBQ's deeper implication: discrete representations aren't inherently inferior — it depends on *how* you discretize. If discrete tokens can match continuous quality, a unified text-visual token space is no longer bottlenecked by quantization.
- Complexity-aware sampling as proto-metacognition: the model monitors its own uncertainty at inference and reallocates compute accordingly — something diffusion's fixed schedules lack.
- Caveats: largest model is only 2B; video samples sometimes lack fine detail and overfit certain data categories; per-step cost is higher than scale-by-scale AR since each step processes the full token map.
- Han, J., et al. (2026). *Generative Refinement Networks for Visual Synthesis*. arXiv:2604.13030.
- Code: https://github.com/MGenAI/GRN
- Ji, Y., et al. (2025). *JiT: Back to Pixel-Level Purity*.
- Tian, K., et al. (2024). *Visual Autoregressive Modeling*. NeurIPS 2024.
- Chang, H., et al. (2022). *MaskGIT*. CVPR 2022.
- Han, J., et al. (2025). *Infinity: Bitwise Visual Tokenizer*.
- Wan Team (2025). *Wan 2.1*.
The quantization error decays exponentially: \(|e_j| < 1/2^j\) — with M=8 rounds, error < 1/256. The process resembles a multi-resolution decomposition akin to Haar wavelets.
Advantages: no extra latent channels; higher compression (16 channels × 4 bits = 64 bits/token); reconstruction rFID 0.56 on ImageNet 256×256 at 96× compression — beating all continuous and discrete baselines including SD-VAE (0.87 at only 24× compression).
2. Global Refinement Mechanism: An "Eraser" for AR Models
Human painters don't finalize in one stroke — they sketch, refine, and erase. GRN formalizes this:
Each iteration: input the hybrid map, predict the complete next token map via a Transformer, then randomly retain a fraction \(l_{t+1}\) of tokens. This unifies filling (new predictions for blank regions), refining (replacing kept tokens with better predictions), and erasing (resetting some tokens to random). Training uses a destroy-and-reconstruct objective: given a mix of ground-truth and random tokens, the model predicts the full ground truth — similar in spirit to diffusion noising and BERT's MLM.
3. Entropy-Guided Complexity-Aware Sampling
Since each step predicts the full token map, easy samples converge early. Prediction entropy \(H(Y_t)\) serves as a difficulty signal: low entropy → grow the retention schedule faster and stop early; high entropy → proceed slowly with more refinement steps. In practice, 62.7% of samples finish before 50 steps, with only a negligible FID penalty (3.6 → 3.8).
Results
Tokenizer (ImageNet 256×256 reconstruction): HBQ (M=4) achieves rFID 0.56, LPIPS 0.13, SSIM 0.71, PSNR 23.01 at 96× compression — surpassing SD-VAE, VAR, LlamaGen, and Open-MAGVIT2.
Class-conditional ImageNet 256×256: GRN-G (2B, discrete) reaches FID 1.81 / IS 299.0, beating VAR-d30 (1.92) and matching continuous JiT-G/16 (1.82), confirming that global refinement mitigates AR error accumulation.
Text-to-Image (GenEval): GRN (2B) scores 0.76, beating SD3 Medium (0.62) and Infinity (0.71) at the same scale.
Text-to-Video (VBench): GRN (2B) scores 82.99, beating CogVideoX-5B (81.61) and Emu3 (80.96), trailing only 14B Wan 2.1 and 8B InfinityStar.
Ablations:
Discussion
Limitations & Future Directions
Current limits: scale gap vs. 14B+ SOTA, video detail fidelity, and per-step compute. Promising directions: unification with LLM vocabularies for seamless multimodal generation, step distillation, dynamic resolution, 3D/4D spatiotemporal refinement, and competing with Transfusion-style architectures — the paper explicitly notes GRN "has the potential to emerge as a strong competitor to the currently dominant Transfusion architecture."