RAFT: A Stateful Retrieval-Augmented Framework for Troubleshooting Agents

Forum topic · 小凯 · 2026-09-20

Summary

RAFT (Retrieval-Augmented Framework for Troubleshooting Agents) is a stateful RAG framework for enterprise customer-support troubleshooting agents, presented in arXiv paper 2609.20754. Unlike conventional RAG systems that treat support cases as static documents, RAFT abstracts each closed historical case into a directed chain of timeline entries and retrieves at the entry level. When a retrieved entry matches an intermediate state of the active case, RAFT returns the full parent-case trajectory anchored at that matched state; an optional case-level graph connects cases via a configurable similarity representation. The authors evaluate the retrieval layer directly without requiring production deployment, and address the scarcity of public multi-stage troubleshooting data by building a synthetic benchmark from Microsoft Learn Windows Server documentation plus a real-world Apache Jira issue set with human duplicate labels. RAFT outperforms vanilla RAG and GraphRAG baselines across all stages of case progression, with statistically significant gains over the strongest baseline, and Jira results provide directional evidence of transferability to real case histories. Benchmarks, implementation, and the Jira evaluation set are released publicly.

Paper Overview

Field: Machine Learning Authors: Mingxuan Zhang, Xiaowen Wang, Anupma Sharan, Zhengyi Chen, Chenyu Diana Zhang, Shanshan Yang, Chittibabu Pacharu arXiv: 2609.20754

Abstract

Effective troubleshooting agents in enterprise customer support depend on retrieving actionable guidance from similar historical cases, yet existing retrieval-augmented generation (RAG) systems treat support cases as static documents and overlook their multi-stage, stateful nature. We introduce RAFT (Retrieval-Augmented Framework for Troubleshooting Agents), a stateful RAG framework that abstracts each closed historical case into a directed chain of timeline entries and retrieves at the entry level, surfacing cases whose intermediate states match the active case and returning the parent-case trajectory anchored at the matched state; an optional case-level graph links cases through a configurable similarity representation. We evaluate this retrieval layer directly, which, unlike evaluating an entire agent system, requires no production deployment. Because public multi-stage troubleshooting data is extremely scarce, we construct a benchmark synthesized from Microsoft Learn Windows Server documentation, along with a real-world Apache Jira issue set with human-verified duplicate labels. RAFT outperforms vanilla RAG and GraphRAG baselines at every stage of case progression, with statistically significant gains over the strongest baseline; the Jira results provide directional evidence that the advantage transfers to real case histories. We release the benchmarks, implementation, and the Jira evaluation set.

Key Contributions

Stateful retrieval: models historical cases as directed chains of timeline entries instead of flat documents
Entry-level matching: anchors retrieval on intermediate states of the active case and returns the full parent-case trajectory from the matched state
Optional case-level graph: connects cases via a configurable similarity representation
Direct retrieval-layer evaluation: avoids the cost of deploying full agent systems
New benchmarks: a synthetic dataset from Microsoft Learn Windows Server docs and a human-labeled Apache Jira duplicate set
Statistically significant improvements over vanilla RAG and GraphRAG baselines across all case-progression stages

*Benchmark, code, and Jira evaluation set are publicly released by the authors.*