Remember to Be Curious: Episodic Context and Persistent Worlds for 3D Exploration

Forum topic · 小凯 · 2026-05-24

Summary

A Chinese tech forum post offers an accessible, in-depth walkthrough of the research paper "Remember to be Curious: Episodic Context and Persistent Worlds for 3D Exploration" by Lily Goli, Justin Kerr, and Daniele Reda. The paper tackles a core failure of curiosity-driven reinforcement learning in photorealistic 3D environments: agents repeatedly revisit the same areas because their features are sensitive to viewpoint and lighting changes, and they lack persistent spatial memory, so they award themselves intrinsic rewards for "new" states they have actually seen before. The proposed solution rests on two pillars: (1) online 3D reconstruction as a persistent world model, enabling geometric, verifiable checks of whether a location has been visited, and (2) a sequence-model policy (Transformer-like) that maintains episodic context over the trajectory of RGB observations. Trained purely with intrinsic curiosity rewards on HM3D, the agent surpasses all RL-based active mapping baselines and generalizes zero-shot to unseen Gibson environments and procedurally generated worlds. The pretrained exploration policy also transfers to downstream tasks—apple picking and image-goal navigation—outperforming training from scratch. The post frames the central insight memorably: genuine curiosity must be built on memory; exploration is only effective when the agent remembers where it has been.

This post is a richly illustrated Chinese-language explainer of the paper "Remember to be Curious: Episodic Context and Persistent Worlds for 3D Exploration" (Lily Goli, Justin Kerr, Daniele Reda; UBC and collaborators). Below is a structured English translation of its key content.

The core problem

Curiosity-driven reinforcement learning uses intrinsic reward: an agent is rewarded when its predictions about the world fail (prediction error), signaling that something is unknown and worth exploring. This works in simple mazes, but in photorealistic 3D indoor environments (corridors, rooms, furniture, changing light and shadows) agents collapse into local loops:

They walk the same hallway again and again, treating viewpoint and lighting changes as "new" discoveries.
Their feature representations are too sensitive to perspective and illumination, so revisited states feel novel.
Some designs even reward re-encountering a state, mistaking the agent's own forgetting for environmental change.

The authors diagnose this as two missing ingredients: spatial persistence and episodic context.

The two-pillar solution

1. Online 3D reconstruction as a persistent world model

The agent builds a real, geometric 3D scene model in real time while exploring—not a fragile implicit neural map.
"Have I been here?" is answered deterministically by querying the reconstruction: a location is either in the visited set or it is not. No more reward for forgotten states.

2. Sequence model as policy (episodic context)

Instead of a memoryless feedforward policy acting on single frames, the policy is a Transformer-like sequence model over continuous RGB observations.
The agent gains trajectory awareness: it understands where it came from, what path it has taken, and can make context-sensitive decisions.

Together:

Spatial persistence answers: "What does this world look like?"
Episodic context answers: "What paths have I already taken through it?"

Experimental results

The agent trains purely via curiosity (no external task rewards) on HM3D (Habitat-Matterport 3D Dataset) of real scanned interiors:

Active mapping: surpasses all RL-based active mapping baselines.
Zero-shot generalization: strong performance in unseen Gibson environments and in procedurally generated (AI-generated) worlds.

Transfer to downstream tasks

Apple picking: initializing from the curiosity-pretrained policy and fine-tuning significantly outperforms training from scratch—the agent finds apple trees faster and learns picking faster.
Image-goal navigation: given a photo of a target (e.g., a kitchen sink), the agent navigates there more efficiently with better spatial reasoning.

The takeaway: seemingly aimless exploration accumulates spatial priors—the growing 3D map and trajectory memory become the knowledge foundation for all future tasks in sparse-reward, long-horizon settings.

The central paradox

The title is a double entendre: *remember to be curious* (an exhortation) and *memory enables curiosity* (a mechanism). In complex 3D environments, genuine curiosity is built on memory. Without remembering "I've been here," an agent cannot know "I haven't been there"—memory-less curiosity is just blind wandering. Notably, this is *not* Borges's Funes-style total recall: the 3D reconstruction stores compressed geometric structure, and the sequence model stores selective narrative context—memory in service of curiosity, not a burden on it.