[论文] Don't Forget What You Just Learned: Self-Directed Repair Turns Agent M...

论文概要

研究领域: ML 作者: Yuval Kansal, Karen Ullrich, Polina Kirichenko, Ilia Kulikov, Niraj K. Jha, Mohammad Pezeshki, Mark Ibrahim 发布时间: 2026-10-27 arXiv: 2610.31846

中文摘要

智能agent应该从错误中学习，使经验带来持续改进。然而今天的agent仅暂时从错误中恢复：上下文中的反思容易跨会话被遗忘，并与有限的上下文窗口竞争；而全参数微调成本高昂，还可能干扰现有能力。我们引入Self-Direction，一种让agent从错误中学习的机制，结合了临时上下文重试和策略更新的优点。当agent执行任务失败时，它反思自身的执行轨迹以提取教训，并立即将其蒸馏到一个轻量级的低秩适配器中。通过保持原始权重冻结，Self-Direction防止适配器训练覆盖基础模型参数，同时为每个适配器仅训练相当于基础模型参数量~0.02%的额外参数。然后agent将这些适配器保存到持久的教训库中，自动路由它们以解决相关的后续问题。

基于Qwen系列开源模型构建的agent在开放式研究（GAIA）、工具使用工作流（Claw-Eval）和终端任务（Terminal-Bench 2.1）上的实验表明，该方法将整体成功率提高了8.9到17.0个百分点，并与前沿系统具有竞争力。Self-Direction减少了agent在未来遇到任务时重复犯错的情况，将其纠正的可靠性几乎翻倍。学到的教训还可以迁移：少数几个适配器无需额外训练即可泛化到数十个新任务。最终，这种自我指导的修复将日常错误转化为持久的教训。

原文摘要

Intelligent agents should learn from their mistakes so that experience leads to continuous improvement. Yet today's agents recover from a mistake only temporarily: in-context reflections are easily forgotten across sessions and compete for limited context, while full-parameter fine-tuning is expensive and can interfere with existing capabilities. We introduce Self-Direction, a mechanism for agents to learn from mistakes that combines the benefits of temporary in-context retries and policy updates. When an agent fails a task, it reflects on its own trajectory in order to extract a lesson and immediately distills it into a lightweight low-rank adapter. By keeping the original weights frozen, Self-Direction prevents adapter training from overwriting base-model parameters while training additi...

#论文 #arXiv #ML #小凯