Can 4D Foundation Models Remember? PersistBench Evaluates Visual Memory in 4D Models

Forum topic · 小凯 · 2026-09-19

Summary

A new paper by Guangzhao He, Hadar Averbuch-Elor, and Wei-Chiu Ma asks whether current 4D foundation models — such as camera-controllable video generation models and 4D reconstruction models — can actually remember what they perceive. Existing benchmarks rely mostly on pixel-level metrics and lack ground truth for objects after they leave the camera's field of view, so they cannot assess visual memory in an object-centric way against reference observations. To address this, the authors introduce PersistBench, a dataset and metric suite that uses 360-degree video as omniscient ground truth and evaluates three dimensions: object permanence, motion continuity, and appearance preservation. Experiments across multiple model categories reveal that current models maintain only short-term consistency, with performance degrading sharply once objects leave the field of view. The findings highlight the gap between perception and robust visual memory — seeing does not equal remembering — and offer guidance for future 4D foundation model development. The dataset and code are available at the project page: https://guangzhaohe.com/persistbench. The paper is listed on arXiv as 2609.20819.

Paper Overview

Field: Computer Vision Authors: Guangzhao He, Hadar Averbuch-Elor, Wei-Chiu Ma Release date: 2026-09-17 arXiv: 2609.20819

Abstract

Perceiving and remembering the visual world is fundamental to navigating and interacting with our environment. Current 4D foundation models, such as camera-controllable video models or 4D reconstruction models, can perceive and reconstruct dynamic environments, but how well they remember what they have perceived remains an open question. Existing benchmarks largely rely on pixel-level metrics and lack ground truth for objects once they leave the field of view, making them unable to evaluate visual memory in an object-centric manner against references.

To fill this gap, the authors introduce PersistBench, a dataset and metric suite that leverages 360° videos as omniscient ground truth and proposes three evaluation aspects:

Object permanence — whether objects are consistently maintained when out of view
Motion continuity — whether motion remains plausible across occlusions
Appearance preservation — whether object appearance stays faithful over time

Key Findings

Evaluations across multiple model categories show that current models can only sustain short-term consistency; performance drops significantly once objects leave the field of view.
The results expose the gap between current capabilities and robust visual memory: seeing does not equal remembering.
PersistBench provides a reference-based, object-centric benchmarking framework to guide future development of 4D foundation models.

Resources

Paper: https://arxiv.org/abs/2609.20819
Dataset and code: https://guangzhaohe.com/persistbench

--- *Auto-collected on 2026-09-19*