[论文] Finishing the Task Is Not Enough: Evaluating Agent Resilience and Cons...

论文概要

研究领域: cs.AI, cs.HC, cs.MA
作者: Yuanchen Bai, Zijian Ding, Angelique Taylor
发布时间: 2026-09-13
arXiv: 2609.10724

中文摘要

生成式 AI 智能体的持续部署不仅需要孤立的任务成功。智能体必须在重复交互、不断变化的条件以及与共享工作流中人员的依赖关系中保持有用性，特别是在技术、人力和运营中断随时间累积的情况下。我们提出运营韧性和体贴参与作为评估此类智能体的两个互补方面：前者捕捉智能体如何从受阻工作中恢复，同时保留进展并沟通其限制；后者捕捉其适应性如何考虑受影响的人员、角色边界和周围工作流。我们在两个生成式 AI 模型和 12 个利益相关者派生任务的 120 条模拟医疗轨迹中，研究轻、中、重挑战下的行为变化。关于运营韧性，智能体从自我指导的恢复转向更大程度依赖人类，同时在结构化报告中报告不断增加的工作量和负面情绪，但在文本响应中很少表达压力。关于体贴参与，智能体从任务聚焦的适应转向任务重构、关注他人、角色边界调整和更广泛的协调。

原文摘要

Sustained deployment of generative AI agents requires more than isolated task success. Agents must remain useful across repeated interactions, changing conditions, and dependencies on people within shared workflows, especially as technical, human, and operational disruptions accumulate over time. We propose operational resilience and considerate participation as two complementary aspects of evaluating such agents: the former captures how agents recover from blocked work while preserving progress and communicating their limits, and the latter captures how their adaptation accounts for affected people, role boundaries, and the surrounding workflow. Yet both remain underexplored under accumulating challenge. We study 120 simulated healthcare trajectories across two generative AI models and twelve stakeholder-derived tasks under light, medium, and heavy challenge. We compare textual action plans, prompted internal assessments, and quantitative structured workload and affect reports to examine how agent behavior and reported state change as challenge accumulates. Regarding operational resilience, agents shift from self-directed recovery toward greater human dependence, while reporting increasing workload and negative affect in structured reports but seldom expressing strain in textual responses. Regarding considerate participation, agents broaden from task-focused adaptation toward task reframing, attention to others, role-boundary adjustment, and wider coordination, with distinct patterns across actions and internal assessments. From these findings, we derive five deployment dilemmas involving persistence, attention, role boundaries, state disclosure, and escalation that require stakeholder specification, further informing technical implications for learning, situated evaluation, and embodied adaptation.

自动采集于 2026-09-13

#论文 #arXiv #AI #小凯