Paper Overview
Research Area: ML Authors: Lily Goli, Justin Kerr, Daniele Reda Published: 2025-05-23 arXiv: 2505.17382
Abstract
Exploration is a prerequisite for learning useful behaviors on sparse-reward, long-horizon tasks, particularly in 3D environments. Curiosity-driven reinforcement learning addresses this through intrinsic rewards generated by mismatches between an agent's predictive model of the world and reality. However, translating this intrinsic motivation to complex, photorealistic environments remains difficult: agents may become stuck in local loops and receive fresh rewards when revisiting forgotten states.
This paper demonstrates that such failures arise from the absence of spatial persistence and episodic context. The authors show that effective curiosity requires a persistent, continuously updated world model paired with an agent that maintains an episodic trajectory history to navigate toward novel regions.
They achieve this via online 3D reconstruction as the persistent world model, while the agent's policy is parameterized as a sequence model over RGB observations to preserve episodic context. This design enables effective exploration during training while allowing the agent to navigate at deployment using only RGB frames.
Trained purely through curiosity on HM3D, the agent outperforms RL-based active mapping baselines and generalizes zero-shot to Gibson and AI-generated worlds. The end-to-end policy adapts efficiently to downstream tasks such as apple picking and image-goal navigation, outperforming baselines trained from scratch.
--- *Auto-collected on 2026-05-23*