L3G-World: Real-Time Infinite World Generation with Latent 3D Gaussian Memory

Forum topic · 小凯 · 2026-10-30

Summary

L3G-World is a research paper (arXiv:2610.40614) addressing the challenge of real-time, spatially consistent world generation over long-horizon exploration. Existing interactive world models rely on temporal context or explicit 3D memory, both of which scale poorly as memory grows with the length of the generated stream; explicit 3D memories also require re-encoding RGB renderings, preventing real-time generation. The authors introduce a Latent 3D Gaussian Memory (L3G) that combines the efficiency of feed-forward reconstruction with a compact, rapidly renderable scene representation. Building on this memory, L3G-World uses a memory updater and a video diffusion transformer (DiT) that alternate autoregressively during generation, enabling real-time infinite world generation. The paper is authored by Jinhyeok Choi, JoungBin Lee, Jongmin Lee, Minkyeong Jeon, Jiyoung Kim, Kaeun Han, Yoonjin Yoon, and Seungryong Kim, in the computer vision field. This page presents the paper overview, abstract, and arXiv link.

Paper Overview

Field: Computer Vision (CV) Authors: Jinhyeok Choi, JoungBin Lee, Jongmin Lee, Minkyeong Jeon, Jiyoung Kim, Kaeun Han, Yoonjin Yoon, Seungryong Kim arXiv: 2610.40614

Abstract

Interactive world models generate visual content conditioned on user actions, yet maintaining spatial consistency over long-horizon exploration in real time remains challenging. Existing approaches rely on either temporal context or explicit 3D memory, both of which scale poorly because memory grows with the length of the generated stream. Moreover, explicit 3D memories condition latent video generators by re-encoding RGB renderings, which precludes real-time generation.

In this paper, the authors introduce a Latent 3D Gaussian Memory (L3G) that combines the efficiency of feed-forward reconstruction with a compact, rapidly renderable scene representation. Building on this memory, they propose L3G-World, in which a memory updater and a video diffusion transformer (DiT) alternate autoregressively to generate consistent worlds in real time.

Key Contributions

Latent 3D Gaussian Memory (L3G): a compact, fast-to-render scene representation that avoids the memory growth problem of prior temporal-context or explicit 3D memory approaches.
Alternating autoregressive generation: a memory updater and a video DiT alternate during generation, maintaining spatial consistency over long-horizon exploration.
Real-time performance: by conditioning in latent space rather than re-encoding RGB renderings, the framework supports real-time infinite world generation.

For full details, see the paper on arXiv: 2610.40614

---

*Auto-collected on 2026-10-31.*