ECHOES: Seeing Beyond the Current Image via Recursive History Compression for Long-Horizon GUI Agents
Field: Computer Vision Authors: Tengda Guo, Jiawang Hu, Yifei Tao, Yin Wang, Haotian Hu, Fanyi Wang, Liujuan Tang Published: 2026-10-19 arXiv: 2610.22408
Chinese Summary (translated)
Long-horizon GUI agents rely on visual history to maintain task state, but keeping historical screenshots drives up visual-token usage and inference costs, while pure text summaries lose critical state information and hurt long-horizon reliability. ECHOES uses training-time privileged information to guide textual history compression: it adopts a "Compression-Think-Action" (CTA) format to recursively update a textual history state while reasoning. The same model, given privileged multi-image inputs during training, acts as a teacher performing dense token-level on-policy distillation on the student's CTA rollouts. A PMF Gap-guided OPD-RL framework dynamically balances privileged distillation and task-reward optimization, with periodic teacher updates supporting continued improvement. The team also releases ECHOESBMK, a fine-grained diagnostic benchmark derived from real GUI agent failure trajectories, jointly evaluating visual perception and cross-modal state retention. Experiments on four GUI benchmarks and ECHOESBMK show significant gains in task performance and memory fidelity; at deployment, competitive performance is achieved without retaining historical screenshots, yielding a better performance-inference-cost trade-off and positive zero-shot transfer to unseen visual planning tasks.
Original Abstract
Long-horizon GUI agents rely on visual history to maintain task states, but retaining historical screenshots increases visual-token usage and inference costs, while textual summaries can discard critical state information, undermining long-horizon reliability. We propose ECHOES, which leverages training-time privileged information to guide textual history compression and strengthen long-horizon reasoning. Specifically, ECHOES adopts a Compression-Think-Action (CTA) format to recursively update a textual history state and reason. The same model, given privileged multi-image inputs, serves as a teacher that provides dense token-level on-policy distillation on student-generated CTA rollouts. Our Perception-Memory Fidelity (PMF) Gap-guided OPD-RL framework dynamically balances privileged distillation and task-reward optimization, with periodic teacher updates supporting continued improvement. We further introduce ECHOESBMK, a fine-grained diagnostic benchmark derived from real GUI agent failure trajectories to jointly evaluate visual perception and cross-modal state retention. Experiments on four GUI benchmarks and ECHOESBMK demonstrate substantial gains in task performance and memory fidelity. Without retaining historical screenshots at deployment, ECHOES achieves competitive performance with a favorable performance-inference-cost trade-off and exhibits positive zero-shot transfer to unseen visual planning tasks.
--- *Auto-collected on 2026-10-19*