[论文] Decoupling Readiness from Release for Tail-Aware Scheduling of Agentic...

论文概要

研究领域: cs.AI, cs.SE
作者: Bochao Feng, Jianjiang Li, Haojie Wang, Lin Qiao, Yinghui Li, Yukun Yan, Jidong Zhai
发布时间: 2026-09-13
arXiv: 2609.10964

中文摘要

智能体 LLM 工作流由模型轮次与工具交互交错组成，因此端到端完成时间不仅取决于推理速度，还取决于就绪轮次何时释放。大多数运行时在就绪时立即释放每个轮次。在竞争下，这种急切释放策略可能累积已释放但未完成的工作；一旦提交，这些轮次无法再被工作流级策略重新排序，增加尾部延迟。我们提出尾部风险感知轮次释放调度方法，联合决定下一个释放哪个就绪轮次以及维持多少已释放但未完成的工作。该方法使用均值-条件风险价值（CVaR）目标来捕获未完成工作流的演变尾部风险，在优先化就绪轮次时纳入轮次工作的在线估计，并根据观察到的队列压力调整释放工作预算。使用软件工程任务的真实智能体执行轨迹评估，该方法在轻负载下与急切释放相当，在竞争下大幅减少了工作流流时间的 P95，实现高达 3.50 倍加速。

原文摘要

Agentic LLM workflows consist of sequences of model turns interleaved with tool interactions, so their end-to-end completion time depends not only on inference speed but also on when ready turns are released. Most runtimes release each turn immediately upon readiness. Under contention, this eager release policy can accumulate released but unfinished work; once submitted, those turns can no longer be reordered by the workflow-level policy, increasing tail latency. We present a tail-risk-aware turn release scheduling method that jointly decides which ready turn to release next and how much released but unfinished work to maintain. The method uses a mean--Conditional Value-at-Risk (CVaR) objective to capture the evolving tail risk of unfinished workflows, incorporates online estimates of turn work when prioritizing ready turns, and adapts the released work budget to observed queue pressure. We evaluate the method using real agent execution traces from software engineering tasks across multiple LLMs and workflow arrival rates. The method performs comparably to eager release under light load and substantially reduces the P95 of workflow flow time under contention, achieving up to a \(3.50\\times\) speedup.

自动采集于 2026-09-13

#论文 #arXiv #AI #小凯