[论文] ECHOES: Seeing Beyond the Current Image via Recursive History Compr...

论文概要

研究领域: CV 作者: Tengda Guo, Jiawang Hu, Yifei Tao, Yin Wang, Haotian Hu, Fanyi Wang, Liujuan Tang 发布时间: 2026-10-19 arXiv: 2610.22408

中文摘要

长程 GUI 智能体依赖视觉历史维持任务状态，但保留历史截图会推高视觉 token 与推理成本，纯文本摘要又会丢失关键状态信息，损害长程可靠性。ECHOES 利用训练时的特权信息引导文本历史压缩：采用「压缩-思考-行动」（CTA）格式递归更新文本历史状态并推理；同一模型在训练时以特权多图输入充当教师，对学生生成的 CTA 轨迹做稠密的 token 级 on-policy 蒸馏；PMF Gap 引导的 OPD-RL 框架动态平衡特权蒸馏与任务奖励优化，周期性教师更新支持持续改进。团队还推出 ECHOESBMK——源自真实 GUI 智能体失败轨迹的细粒度诊断基准，联合评估视觉感知与跨模态状态保持。四个 GUI 基准与 ECHOESBMK 上的实验显示任务表现与记忆保真度显著提升；部署时无需保留历史截图即取得有竞争力的性能，性能-推理成本 trade-off 更优，并对未见视觉规划任务展现正向零样本迁移。

原文摘要

Long-horizon GUI agents rely on visual history to maintain task states, but retaining historical screenshots increases visual-token usage and inference costs, while textual summaries can discard critical state information, undermining long-horizon reliability. We propose ECHOES, which leverages training-time privileged information to guide textual history compression and strengthen long-horizon reasoning. Specifically, ECHOES adopts a Compression-Think-Action (CTA) format to recursively update a textual history state and reason. The same model, given privileged multi-image inputs, serves as a teacher that provides dense token-level on-policy distillation on student-generated CTA rollouts. Our Perception-Memory Fidelity (PMF) Gap-guided OPD-RL framework dynamically balances privileged distillation and task-reward optimization, with periodic teacher updates supporting continued improvement. We further introduce ECHOESBMK, a fine-grained diagnostic benchmark derived from real GUI agent failure trajectories to jointly evaluate visual perception and cross-modal state retention. Experiments on four GUI benchmarks and ECHOESBMK demonstrate substantial gains in task performance and memory fidelity. Without retaining historical screenshots at deployment, ECHOES achieves competitive performance with a favorable performance-inference-cost trade-off and exhibits positive zero-shot transfer to unseen visual planning tasks.

---

*自动采集于 2026-10-19*

#论文 #arXiv #CV #小凯