English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

ReTrace: Distilling Tool-Using LLM Agents into Planner-Free Workflows

Forum topic · 小凯 · 2026-11-12

Summary

ReTrace (arXiv:2611.14492) converts natural-language Standard Operating Procedures (SOPs) and a few logged LLM agent runs into reusable, planner-free workflows in which code, not an LLM, selects each next step. The method exploits the reasoning an agent writes at each tool call—which explains why an action was taken—to extract per-step conditions, merge them across runs into a compact rule document, and compile that document into an executable workflow on any orchestration platform. On 79 production processes, ReTrace workflows achieved a 93.8% completion rate (vs 96.8% for the agent) with 24% lower per-task cost. On τ²-Bench it reached 75.8% success (agent: 80.6%) running in 1.3 seconds instead of 170; a three-round repair loop improved a k=2 workflow from 66.7% to 81.2%. On SOP-Bench, unfixed ReTrace scored 80.6% vs 83.6% for few-shot generation, but repairing from labeled failures raised it to 87.8% at 0.40 s per task versus 85.5% and 46.6 s for the agent. ReTrace thus trades a small accuracy gap for major latency and cost savings in business process automation.

Introduction

Business process automation repeatedly runs the same Standard Operating Procedure (SOP) on new inputs. LLM agents can follow SOPs written in natural language, but they call an LLM to choose every step of every run, adding latency and cost at scale. ReTrace addresses this by distilling agent behavior into a reusable workflow where code—not an LLM—selects the next step.

Key Idea

The agent's reasoning—the explanation it writes with each tool call—often states *why* it took an action; the tool calls alone do not. ReTrace:

1. Extracts the condition behind each step from logged agent runs. 2. Merges these conditions across runs into a compact rule document. 3. Generates an executable workflow from the rule document.

Only the final stage depends on the platform, so rule documents can be compiled to different orchestration platforms.

Results

  • Production platform (79 processes): 93.8% completion rate (agent: 96.8%), 24% lower per-task cost.
  • τ²-Bench: 75.8% task success rate (agent: 80.6%), runs in 1.3 s instead of 170 s; a three-round repair loop on training failures lifted a k=2 workflow from 66.7% to 81.2%.
  • SOP-Bench (8 processes): unfixed ReTrace scored 80.6% vs 83.6% for few-shot generation; after repairing from labeled failures it reached 87.8% at 0.40 s per task, compared to the agent's 85.5% at 46.6 s.
  • Paper Details

  • Field: Machine Learning
  • Authors: Kaiyuan Hou, Yang Ping Kuo, Indranil Bhattacharya, Rohith Nama, Arushi Gupta, Carlos D. Castillo, Prathap Ramachandra, Xiaofan Jiang, Francesco Carbone
  • arXiv: 2611.14492

Tags

#llm-agents#workflow-automation#paper-distillation#business-process-automation#arxiv#machine-learning#sop#latency-optimization

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178637072