A Crash, a Black Box, 120 Records
Imagine an aviation accident investigator. A flight crashes; the black box logs 120 operational steps. Frame-by-frame review reveals the pilot pressed the wrong button at step 18, misread the altimeter at step 47, dropped a word in tower communication at step 73, forgot to lower landing gear at step 95, and pulled the yoke too hard at step 112.
Five errors, one crash. Which one is the real culprit?
You cannot list all five as "causes"—that equals no answer. You must find the earliest decisive error: the one that triggered the cascade, where even seemingly corrective middle steps could not prevent the final outcome.
This is the same dilemma in LLM Agent debugging. The paper TrajDebug, released by Tsinghua University's KEG Lab and the Tencent Hunyuan team in August 2026, offers a systematic answer: track the entire lifecycle of an error, not just the error itself.
The Crash Site: 7.62 Errors on Average, Only One Fatal
The researchers first ran a striking pilot experiment. They collected 50 failed trajectories from two mainstream Agent benchmarks, τ²-Bench and SWE-Bench Pro, and manually labeled whether each step was erroneous. The results:
- 381 local errors across 50 failed trajectories, averaging 7.62 per trajectory
- Only 1 critical error per trajectory actually caused the final failure
- Of the remaining 331 non-critical errors:
- 205 (61.9%) were later self-corrected
- 104 (31.4%) persisted but did not cause failure
- 22 (6.6%) remained dormant—neither fixed nor impactful
- High granularity: full detail of the most recent steps (for immediate action context)
- Medium granularity: moderately compressed summaries (for recent development)
- Low granularity: highly compressed full-trajectory overview (for global direction)
- Task conflict: contradicts the original task instruction
- History conflict: contradicts the Agent's own prior reasoning or actions
- Intra-step conflict: self-contradiction within the same step
- Environment anomaly: contradicts environmental feedback
- Clean Resolution: error corrected, no residual impact
- Costly Resolution: error corrected but at the cost of resources or detours
- Manifest Active: error persists and leaves traces in the final failure
- Latent Active: error persists but does not directly manifest in the final failure
- 486 failed trajectories: 400 from τ²-Bench (customer-service tool use), 86 from SWE-Bench Pro (software engineering)
- Average length: τ²-Bench 29.3 steps; SWE-Bench Pro 119.7 steps
- Annotation content: all local errors per trajectory, error types, execution stages, criticality
- Paper: https://arxiv.org/abs/2608.06346
- HTML full text: https://arxiv.org/html/2608.06346v1
- Code repository: https://github.com/THU-KEG/TrajDebug (code and data under internal review, open-sourcing imminent)
- Author team: Tsinghua KEG Lab + Tencent Hunyuan
This data overturns an intuition. We assume "failed trajectory = a sequence of compounding mistakes," but reality is different: across a 120-step trajectory, an Agent commits 7–8 errors, corrects most of them, and only one truly sows the seeds of destruction.
Detection accuracy also decreases with trajectory length. The team evaluated seven major LLMs (including GPT, Claude, DeepSeek variants). They could barely locate critical errors in short trajectories; on long ones, accuracy dropped off a cliff. The reason is simple: judging whether step 47 is wrong may require looking back at the instruction in step 3, the observation in step 28, and the Agent's own reasoning in step 35. Evidence scatters across the entire context, and the model forgets the beginning by the time it reaches the end.
TrajDebug's Three-Phase Investigation: From "Find the Error" to "Trace Its Lifecycle"
TrajDebug's core insight: errors are not isolated events; they have lifecycles. An error triggers, develops, gets repaired or persists, and concludes—a full process. Looking only at the error itself is like looking only at the crime scene, ignoring what happened next.
The framework proceeds in three stages, each producing auditable evidence:
Phase 1: Error Trigger Detection via Multi-Granularity Historical Compression
The challenge of long trajectories: judging step 47 requires both local detail (the current action) and global context (prior instructions and observations). TrajDebug constructs three granularity levels of historical view:
Like an aviation investigator watching three screens simultaneously: one zoomed in on a specific action, one showing the last ten minutes of flight attitude, and one showing the entire route.
Across these views, TrajDebug detects wrong commitments—judgments made by the Agent during reasoning, planning, action, observation, or verification that conflict with a reference. References fall into four categories:
Key design: every error trigger must have verbatim evidence—both the original commitment text and the violated reference text. This prevents "hallucinated accusations."
Phase 2: Error State Classification—Four Fates
After detecting triggers, TrajDebug does not rush to conclusions. It clusters triggers related to the same object into "error instances," then tracks each instance's subsequent development into one of four states:
Only Costly Resolution and Manifest Active states can be causally linked to final failure. This parallels how doctors distinguish self-healing colds, chronic conditions managed with illness, and acute fatal episodes—not every abnormality needs accountability.
Phase 3: Candidate-Set-Guided Causal Attribution
In the final step, TrajDebug hands all "Manifest Active" and "Costly Resolution" error instances to an LLM for final attribution. The key point: the LLM no longer faces the entire trajectory, only a refined candidate set.
This reduces the task from "find the critical error in 120 steps" to "pick the most critical one from 3–5 candidates." The candidate set already comes with complete evidence chains (trigger location, violated reference, subsequent development); the LLM only needs to make a causal judgment.
TrajErrBench: 486 "Accident Files"
To enable reproduction and comparison, the team built TrajErrBench—the first large-scale, human-annotated Agent failure trajectory benchmark:
Domain differences emerged in annotation statistics:
| Dimension | τ²-Bench | SWE-Bench Pro | |---|---|---| | Task conflict | 52.4% | 73.4% | | History conflict | 46.3% | 24.1% | | Reasoning-stage failures | 60.7% | 57.1% |
In τ²-Bench's customer-service scenarios, Agents more easily forget what they previously said (high history conflict). In SWE-Bench Pro's coding scenarios, Agents more easily misunderstand task requirements (high task conflict). Across both, however, the reasoning stage is the dominant failure point—not action errors, but thinking errors.
Experimental Results: Diagnosis as Productivity
TrajDebug surpassed all baselines on TrajErrBench, with greater advantages on long trajectories. Two application experiments are more compelling:
Application 1: Diagnosis-guided retry, +10.80% success rate Use TrajDebug to diagnose failed trajectories, feed back "which step was the critical error, what did it violate, why did it cause failure," and let the Agent re-execute the same task. Average success rate improved by 10.80 percentage points.
Application 2: Failure memory transfer, +5.70% success rate Aggregate diagnoses from a small set of historical failures into a "failure memory" library, and transfer it to entirely unseen tasks. Success rate improved by 5.70 percentage points.
The second result is especially important. It proves TrajDebug's diagnosis is not mere "post-hoc wisdom" but transferable structured knowledge. Failure patterns can be abstracted into Agent immunity.
Engineering Insights: Errors Have Lifecycles; Debugging Needs a Time Dimension
TrajDebug offers three practical lessons for Agent engineers:
1. Don't look only at the final failure point—trace the error's origin An Agent fails at step 95, but the root cause may lie at step 18. Patching step 95 treats the symptom, not the cause. TrajDebug's error-lifecycle view reveals the propagation chain.
2. Not every error needs fixing 61.9% of local errors are self-corrected by the Agent. If you spend effort fixing every error, you waste 60% of engineering capacity. Classify first, then decide which to fix—that is the value of TrajDebug's four-state classification.
3. Failed trajectories are gold mines, not garbage Most teams discard failed trajectories. TrajDebug demonstrates: 486 failed trajectories can train transferable failure memory, boosting new-task success by 5.70%. Failure is a source of structured knowledge, provided you have tools to diagnose it clearly.