Paper Overview
- Field: Machine Learning / Reinforcement Learning
- Authors: Lily Goli, Justin Kerr, Daniele Reda
- Published: 2025-05-23
- arXiv: 2505.17382
- A curiosity-based exploration framework that achieves effective exploration during training while allowing the agent to navigate at deployment using only RGB frames.
- Pure curiosity-driven training on HM3D outperforms RL-based active mapping baselines.
- Zero-shot generalization to the Gibson dataset and AI-generated worlds.
- The end-to-end policy efficiently adapts to downstream tasks such as apple picking and image-goal navigation, surpassing baselines trained from scratch.
Summary
Exploration is a prerequisite for learning useful behaviors in sparse-reward, long-horizon tasks, especially in 3D environments. Curiosity-driven reinforcement learning tackles this by rewarding agents for mismatches between their world-model predictions and reality. However, transferring intrinsic motivation to complex, realistic environments remains difficult: agents tend to get stuck in local loops and earn fresh intrinsic rewards when revisiting forgotten states.
The authors demonstrate that this failure stems from the absence of spatial persistence and episodic context. They argue that effective curiosity requires:
1. A persistent and continuously updated world model. 2. An agent that retains episodic trajectory history to navigate toward novel regions.
They implement this by using online 3D reconstruction as the persistent world model, while the agent policy is parameterized as a sequence model over RGB observations to preserve episodic context.
Key contributions:
Original Abstract
*Not provided in source post.*