[论文] Flide: Accelerating Decision Inference with Exact State Forking

论文概要

研究领域: NLP
作者: Bobo Li, Jiajia Song, Hao Fei, Wynne Hsu, Mong Li Lee
发布时间: 2026-10-30
arXiv: 2610.42148

中文摘要

诸如Jev之类的决策模型已成为LLM智能体系统中极具前景的范式，为快速的"系统一"执行提供轻量且可靠的分类推理。然而，现有的面向生成的推理引擎无法有效支持这类工作负载，存在严重的前缀重算开销或输出概率失真的问题。为应对这一挑战，我们提出Flide——一个高效且精确（exact）的推理引擎，它将决策请求的逻辑树结构与其在单一扁平token流中的物理执行解耦。通过ForkAttention，Flide在标准注意力和循环Gated DeltaNet层中使用GPU驻留索引来导航请求依赖关系，在保证精确算术的前提下，确保每个查询分支的计算结果与独立前向传播完全一致。通过将分支拓扑直接表达为GPU驻留元数据，Flide能够以一套紧凑的预录制执行图适应多样化的请求混合，同时支持不断增长的智能体历史从先前传递中保存的检查点恢复。在公开混合模型和决策模型上的大量评估表明，Flide回答查询的速度比vLLM和SGLang快达9.5倍，同时在总变差距离上最接近float32参考实现。通过将物理执行吞吐量与数学精确性统一起来，Flide为智能体系统中可扩展、延迟敏感的决策智能奠定了有原则的基础。

原文摘要

Decision models such as Jev have emerged as a highly promising paradigm for LLM agent systems, providing lightweight and reliable categorical reasoning for rapid System One execution. However, existing generation-oriented inference engines fail to support these workloads effectively, suffering from severe prefix recomputation overheads or distorted output probabilities. To address this challenge, we propose Flide, an efficient and exact inference engine that decouples the logical tree structure of a decision request from its physical execution within a single flattened token stream. Through ForkAttention, Flide navigates request dependencies using GPU-resident indices across both standard attention and recurrent Gated DeltaNet layers, guaranteeing in exact arithmetic that every query branc...

自动采集于 2026-11-03

#论文 #arXiv #NLP #小凯