Overview
- Field: Computer Vision
- Authors: Chuyue Li, Jinpeng Yu, Haozhe Wang
- arXiv: 2508.03420
- Videos are converted into knowledge graph (KG) representations, then reconstructed back into video.
- Hierarchical and state nodes store structured text; a linked asset layer holds generated images, audio, and video.
- Typed edges preserve the relationships between textual descriptions and assets in a form agents can easily understand, query, and edit.
- Reconstruction error drives a text-gradient optimization framework: evaluation feedback is expressed as natural-language update directions, used for:
- Outer loop: pseudo-training of a data-independent encoding strategy.
- Inner loop: test-time refinement of the data-dependent KG representation.
- Improves over the strongest external baseline by 20.7 percentage points.
- In a controlled policy-only setting, the pseudo-trained shot-level agent video encoder policy outperforms carefully hand-tuned policies while using 74.3% fewer system prompt tokens.
Key points
Creative agents still lack an effective way to learn from high-quality human films, limiting their ability to produce cinematic-quality videos. A central challenge is the absence of a structured video representation that is both faithful to cinematic content and directly usable for agent reasoning and manipulation. The paper proposes the Agentic Video Auto-Encoder (AVA-Encoder), a framework for learning agent-native video representations via agent-based auto-encoding.
Method
Results
Releases
The authors release:
1. The complete AVA-Encoder framework. 2. A reliable agent video reconstruction benchmark. 3. The first high-quality cinematic KG representation dataset.