PRA (Parallel Rollout Approximation for Pixel-Space Autoregressive Image Generation) was proposed by Jiayi Xu, Di He (Peking University), and Guolin Ke (DP Technology).
- Paper: https://arxiv.org/abs/2606.27978
- Code: https://github.com/MangataX/PRA
- One-line summary: PRA is an end-to-end pure pixel-space autoregressive image generation method requiring no external pretrained tokenizer, directly modeling raw pixel patches and jointly fixing both output-side and input-side bottlenecks. With only 135M parameters it beats prior billion-scale pixel AR models; the large version reaches FID 1.94.
- PRA-S (135M) beats FARMER-1.9B, closing a 28x parameter gap.
- Ablations confirm every component: prefix-aware targets beat local-only (2.88 vs 2.96) and direct pixel prediction (~10); token masking improves FID to 2.58; d_z=16 is the sweet spot (8: 2.73, 32: 2.72, 64: 3.04); t_min=0.5 for the interpolation is optimal.
- End-to-end learning beats a frozen LDM encoder (3.08), showing the tokenizer and AR objective need co-optimization.
- FID still trails the best latent AR (PRA-L 1.94 vs MAR 1.55, VAR 1.73).
- Generation is slow: 256 AR steps × 100 diffusion sampling steps each.
- d_z=16 is empirically optimal; other datasets/resolutions may need different compression ratios.
- Validated only on ImageNet; text-to-image, high resolution, and video remain untested.
- Whether the parallel inference-like input construction could improve diffusion training is open.
- PRA-S: 135M params, FID 2.58 (beats FARMER-1.9B at 3.60)
- PRA-L: 511M params, FID 1.94 (pixel AR SOTA)
- Best intermediate dimension d_z: 16
- ImageNet linear probing: PRA-L 68.80% (vs SphereAR-L 64.80%, JiT-L 65.40%)
- Key designs: prefix-aware intermediate targets + parallel rollout approximation + pixel-in/pixel-out interface
Key points
Background
Current image generation has two main routes:
| Route | Examples | Pros | Cons | |---|---|---|---| | Latent-space AR | VAR, MAR, Llamagen | Low-dim tokens easy to model | Depends on pretrained tokenizer; quality capped by tokenizer | | Pixel-space diffusion | JiT, Palette | End-to-end, no tokenizer bottleneck | Non-autoregressive, slow, no causal structure |
Pixel-space AR is attractive (no tokenizer bottleneck, causal structure naturally supports conditional generation/editing/completion) but has long underperformed latent AR and diffusion models.
Diagnosis: two coupled bottlenecks
Output side: predicting a 768-dim flattened 16x16 patch is far harder than a 48-dim patch. In low-dim settings AR matches diffusion (JiT); in high-dim settings AR's FID jumps from ~2 to ~10 while JiT stays stable. The problem is per-step dimensionality, not AR step count.
Input side: teacher forcing trains on ground-truth prefixes but inference feeds the model's own imperfect generations, causing compounding error. Random noise injection helps slightly but fails because inference errors are structured, not random.
Key insight: the two bottlenecks are coupled — high-dim outputs produce large per-step errors, degrading prefixes and creating a vicious cycle. Both must be fixed together.
The PRA framework
Output-side fix: instead of predicting 768-dim pixels directly, the AR model generates low-dimensional intermediate states (e.g., d_z=16) that a pixel decoder maps back to pixels. These states are prefix-aware: z_i = g_psi(x_i, h_{i-1}), encoding both the current patch and prior context. Token masking (replacing x_i with a learnable mask embedding with probability p_mask) forces reliance on context.
Input-side fix: parallel approximate rollout. For each target state z_i, sample noise and interpolate z~_i = (1-t)z0_i + t*z_i, decode through the same shared pixel decoder to get pixel inputs x~_i, and use these as AR training prefixes (with stop-gradient). All positions are constructed independently and in parallel — no slow serial rollout — and match inference-time inputs through the identical decoding path.
Training losses: rectified-flow loss in intermediate space for the AR head; L1 + LPIPS reconstruction loss for the pixel decoder.
Results on ImageNet 256x256
| Method | Params | FID | IS | |---|---|---|---| | VAR (latent AR) | 2.0B | 1.73 | 323 | | MAR (latent AR) | 1.1B | 1.55 | 295 | | JiT (pixel diffusion) | 502M | 1.96 | 230 | | FARMER-1.9B (pixel AR) | 1.9B | 3.60 | - | | PRA-S | 135M | 2.58 | 209 | | PRA-B | 250M | 2.21 | 223 | | PRA-L | 511M | 1.94 | 241 |
Unexpected bonus: stronger visual understanding
ImageNet linear probing (frozen backbone):
| Method | Params | Top-1 Acc | |---|---|---| | SphereAR-L | 502M | 64.80% | | JiT-L | 502M | 65.40% | | PRA-L | 511M | 68.80% |
PRA exceeds latent AR and diffusion baselines, supporting the hypothesis that end-to-end pixel learning preserves more transferable visual information than tokenizer-compressed representations.
Why PRA works
1. AR-aligned intermediate representations, not a generic tokenizer: prefix-aware construction and masking make z_i encode "given the prefix, what the current patch should be", serving AR generation exclusively. 2. Parallel rollout approximation: exact on-policy rollout is too slow for continuous-token AR (each step needs multi-step diffusion sampling). PRA only needs inputs that statistically match the rollout distribution — and the shared decoder guarantees matching conditional distributions. 3. Strategic pixel interface: pixel-in/pixel-out enables a unified generation-understanding framework, removes tokenizer bottlenecks, and preserves direct visual signals.
Limitations and open questions
Conclusion
PRA proves pixel-space autoregressive generation can be trained end-to-end without an external tokenizer and approach SOTA quality — 135M parameters beating a 1.9B-parameter predecessor. Beyond generation efficiency, pixel-space backbones learn better representations for understanding, offering a path toward unified pixel-level multimodal models. Where latent AR (VAR/MAR/Llamagen) follows a two-stage "compress-then-generate" route, PRA explores a single-stage end-to-end route.
Quick reference: