Introduction
Business process automation repeatedly runs the same Standard Operating Procedure (SOP) on new inputs. LLM agents can follow SOPs written in natural language, but they call an LLM to choose every step of every run, adding latency and cost at scale. ReTrace addresses this by distilling agent behavior into a reusable workflow where code—not an LLM—selects the next step.
Key Idea
The agent's reasoning—the explanation it writes with each tool call—often states *why* it took an action; the tool calls alone do not. ReTrace:
1. Extracts the condition behind each step from logged agent runs. 2. Merges these conditions across runs into a compact rule document. 3. Generates an executable workflow from the rule document.
Only the final stage depends on the platform, so rule documents can be compiled to different orchestration platforms.
Results
- Production platform (79 processes): 93.8% completion rate (agent: 96.8%), 24% lower per-task cost.
- τ²-Bench: 75.8% task success rate (agent: 80.6%), runs in 1.3 s instead of 170 s; a three-round repair loop on training failures lifted a k=2 workflow from 66.7% to 81.2%.
- SOP-Bench (8 processes): unfixed ReTrace scored 80.6% vs 83.6% for few-shot generation; after repairing from labeled failures it reached 87.8% at 0.40 s per task, compared to the agent's 85.5% at 46.6 s.
- Field: Machine Learning
- Authors: Kaiyuan Hou, Yang Ping Kuo, Indranil Bhattacharya, Rohith Nama, Arushi Gupta, Carlos D. Castillo, Prathap Ramachandra, Xiaofan Jiang, Francesco Carbone
- arXiv: 2611.14492