Beyond the Timeline: Grounded Entity Biographies (GEB) for Long-Video Memory

Forum topic · 小凯 · 2026-10-01

Summary

Grounded Entity Biographies (GEB) is a long-video memory framework designed to improve question answering over videos spanning hours, days, or weeks. Chronological descriptions and text-derived entities often fail to resolve physical identity: different objects can share the same description, while observations of one object remain disconnected across events, so retrieving relevant events does not recover the biography of the entity a question concerns. GEB groups visually grounded observations of the same physical instance across clips into retrievable biographies while preserving per-moment context. At question-answering time, these biographies are retrieved alongside episodic evidence, letting models trace an entity's trajectory using identity links established during memory construction. Evaluated on four benchmarks, including recordings spanning days and weeks, GEB outperforms prior memory frameworks on both multiple-choice and open-ended QA, reaching 72.0% accuracy on EgoLifeQA, 4.4 points above the previously published best result. Ablations show that grounded identity linking and biography reading both contribute, gains not recoverable by additional descriptions alone.

Paper Overview

Field: NLP Authors: Hui Ren, Lei Fan, Henry Pao, Han Guo, Zeeshan Zia, Ying Chen, Alexander Schwing, Gang Hua Published: 2026-09-29 arXiv: 2609.38155

Abstract

Answering questions about long videos often requires connecting events involving the same objects across hours or days. Chronological descriptions and text-derived entities can leave physical identity unresolved: different objects may share a description, while observations of the same object remain disconnected across events. Retrieving relevant events therefore does not necessarily recover the "biography" of the particular entity a question concerns.

To address this, the authors introduce Grounded Entity Biographies (GEB), a long-video memory framework that groups visually grounded observations of the same physical instance across clips into retrievable biographies while preserving the context of each moment. During question answering, the biography is retrieved alongside episodic evidence, allowing models to trace an entity's trajectory across events using identity links established during memory construction.

Key Results

Evaluated on four benchmarks, including recordings spanning days and weeks.
Outperforms prior memory frameworks on both multiple-choice and open-ended QA.
On EgoLifeQA, GEB reaches 72.0% accuracy, exceeding the previously published best result by 4.4 percentage points.
Ablation studies confirm that both grounded identity association and biography reading contribute to the improvement—gains that cannot be fully recovered by additional descriptions alone.

---

*Auto-collected on 2026-10-01.*