A Plane Crashed, and the Black Box Contains 120 Records
Imagine you are an aviation accident investigator. A flight crashed, and the black box recorded 120 operational steps. Reviewing frame by frame, you find the pilot pressed the wrong button at step 18, misread the altimeter at step 47, omitted a word in communication with the tower at step 73, forgot to lower the landing gear at step 95, and pulled up too hard at step 112.
Five errors, one crash. Which one was the real culprit?
You cannot list all five as "causes"—that would mean no answer at all. You must find the earliest decisive error: the one that triggered the chain reaction that ultimately caused the crash, even if intermediate steps appeared to "correct" things.
This is the same dilemma facing LLM agent debugging. TrajDebug, a paper released in August 2026 by Tsinghua University's KEG Lab and Tencent Hunyuan, offers a systematic answer: trace the entire lifecycle of an error, not just the error itself.
The Crash Scene: 7.62 Errors on Average, Only 1 Is Fatal
The researchers first ran a striking pilot experiment. They collected 50 failed trajectories (from two mainstream agent benchmarks, τ²-Bench and SWE-Bench Pro) and manually annotated whether each step contained an error. The results:
- 50 failed trajectories contained 381 local errors—an average of 7.62 per trajectory
- Each trajectory had only 1 critical error that actually caused the final failure
- Of the remaining 331 non-critical errors:
- 205 (61.9%) were self-repaired later
- 104 (31.4%) persisted without causing failure
- 22 (6.6%) remained dormant—neither repaired nor consequential
- High granularity: full detail of the most recent steps (context for the current action)
- Medium granularity: medium-length summaries (recent developmental arc)
- Low granularity: a highly compressed view of the entire trajectory (global trajectory)
- Task conflict: contradicts the original task instruction
- History conflict: contradicts the agent's own prior reasoning or actions
- Intra-step conflict: internal contradiction within the same step
- Environment anomaly: contradicts environment feedback
- Clean Resolution: the error was corrected with no lasting impact
- Costly Resolution: the error was corrected, but consumed resources or took a detour
- Manifest Active: the error persisted and left traces in the final failure
- Latent Active: the error persisted but did not manifest directly in the final failure
- 486 failed trajectories: 400 from τ²-Bench (customer service tool-use scenarios), 86 from SWE-Bench Pro (software engineering scenarios)
- Average length: τ²-Bench 29.3 steps; SWE-Bench Pro 119.7 steps
- Annotations: all local errors per trajectory, error types, execution stages, criticality
- Paper: https://arxiv.org/abs/2608.06346
- HTML full text: https://arxiv.org/html/2608.06346v1
- Code repository: https://github.com/THU-KEG/TrajDebug (code and data under internal review, open-sourcing soon)
- Team: Tsinghua University KEG Lab + Tencent Hunyuan
This upends an intuition. We assume "failed trajectory = error after error," but in reality, an agent commits 7–8 errors across a 120-step trajectory, most are self-corrected, and only one plants the seed of destruction.
Worse, detection accuracy declines with trajectory length. The researchers evaluated 7 mainstream LLMs (including GPT, Claude, DeepSeek, etc.): they could barely locate critical errors on short trajectories, but accuracy plummeted as trajectories lengthened. The reason is simple: judging whether step 47 is wrong may require revisiting the instruction at step 3, the observation at step 28, and the agent's own reasoning at step 35—evidence is scattered across the entire context, and models forget earlier content as they read on.
TrajDebug's Three-Stage Investigation: From "Finding Errors" to "Tracing Lifecycles"
TrajDebug's core insight: errors are not isolated events—they have lifecycles. An error's journey from triggering, to development, to repair or persistent influence is a complete process. Looking only at the error itself is like examining only the crash site without investigating subsequent developments.
The framework has three stages, each producing auditable evidence:
Stage 1: Error Trigger Detection—Multi-Granularity History Compression
The challenge of long trajectories: judging whether step 47 is wrong requires seeing both local detail (the action at that step) and global context (earlier instructions and observations). TrajDebug builds three granularities of history views:
It's like an aviation investigator watching three screens simultaneously: one zoomed into an operational moment, one showing the last 10 minutes of flight attitude, one showing the whole route.
Across these three views, TrajDebug detects "wrong commitments"—judgments made by the agent during reasoning, planning, action, observation, or verification that conflict with a reference. References fall into four categories:
Key design: every error trigger must have verbatim evidence—both the commitment text and the violated reference text. This prevents "hallucinated accusations."
Stage 2: Error State Classification—Four Fates
After finding error triggers, TrajDebug doesn't rush to conclusions. It aggregates triggers related to the same object into "error instances," then tracks each instance's subsequent development into one of four states:
Only Costly Resolution and Manifest Active states can be causally related to the final failure. It's like a doctor distinguishing a "self-healing cold," a "survivable chronic condition," and a "fatal acute episode"—not every anomaly needs to be prosecuted.
Stage 3: Candidate-Set-Guided Causal Attribution
Finally, TrajDebug hands all "Manifest Active" and "Costly Resolution" error instances to an LLM for final attribution. The key: the LLM no longer faces the entire trajectory, only a pre-filtered candidate set.
This reduces the task from "find the critical error among 120 steps" to "pick the most critical one among 3–5 candidates." The candidates come with complete evidence chains (trigger location, violated reference, subsequent development); the LLM only needs to make a causal judgment.
TrajErrBench: 486 "Accident Files"
To enable reproduction and comparison, the team built TrajErrBench—the first large-scale human-annotated benchmark of failed agent trajectories:
The annotation statistics reveal interesting domain differences:
| Dimension | τ²-Bench | SWE-Bench Pro | |------|----------|---------------| | Task conflict | 52.4% | 73.4% | | History conflict | 46.3% | 24.1% | | Reasoning-stage failures | 60.7% | 57.1% |
In τ²-Bench's customer service scenarios, agents more easily forget what they said before (more history conflicts); in SWE-Bench Pro's coding scenarios, agents more easily misunderstand task requirements (more task conflicts). But in both settings, the reasoning stage is the primary failure point—not wrong actions, but wrong thinking.
Results: Diagnosis Is Productivity
TrajDebug outperformed all baselines on TrajErrBench, with especially large advantages on long trajectories. But two application experiments are more persuasive:
Application 1: Diagnosis-guided re-execution, +10.80% success rate. Feeding TrajDebug's diagnosis (which step contained the critical error, what it violated, why it caused failure) back to the agent as feedback for re-executing the same task improved average success rate by 10.80 percentage points.
Application 2: Failure memory transfer, +5.70% success rate. Aggregating a small number of historical failure diagnoses into a "failure memory" and transferring it to completely unseen tasks improved success rate by 5.70 percentage points.
The second result is especially important. It proves TrajDebug's diagnoses are not just "hindsight" but transferable structured knowledge. Failure patterns can be abstracted and turned into agent immunity.
Engineering Insights: Errors Have Lifecycles, Debugging Needs a Time Dimension
TrajDebug offers three practical lessons for agent engineers:
1. Don't just look at the final failure point—trace the error's origin. An agent fails at step 95, but the root cause may be at step 18. Fixing step 95 directly treats symptoms, not the disease. TrajDebug's "error lifecycle" view lets you see the propagation chain.
2. Not every error needs fixing. 61.9% of local errors were self-repaired by the agent. If you spend effort fixing every error, you waste 60% of your engineering budget. Classify first, then decide what to fix—that's the value of the four-state classification.
3. Failed trajectories are gold mines, not garbage. Most teams throw away failed trajectories. TrajDebug proves 486 failed trajectories can yield transferable failure memories, boosting success on new tasks by 5.70%. Failure is a source of structured knowledge—provided you have tools to diagnose it clearly.
Conceptual Connections: Granularity Alignment and "Solving at a Different Level"
TrajDebug fits two cross-paper principles worth tracking:
The "granularity isomorphism" principle: optimization granularity should match the granularity of the object being optimized. Agent failure is not "whole-trajectory failure" but "a particular error instance, at some lifecycle state, causing failure." TrajDebug refines debugging granularity from "trajectory-level" or "step-level" to "error-instance-level"—the same principle behind Heddle and CodeRescue upgrading decision granularity from "single calls" to "trajectories/recovery actions."
"Solving at a different level": traditional methods ask the LLM to find critical errors over the whole trajectory; TrajDebug switches levels—first performing error lifecycle tracking, then downgrading the LLM's task from "multi-step reasoning" to "causal judgment within a candidate set." This parallels Euclid-MCP outsourcing reasoning to Prolog, or octopuses using RNA editing to bypass DNA rewriting: acknowledge that models are unreliable on certain tasks, and route around them at a different level.
Code and Data
Personal Reflection: From "Finding Bugs" to "Understanding a Bug's Life"
What struck me most is TrajDebug's redefinition of "error." We're used to treating errors as discrete events—"step 47 went wrong." But TrajDebug shows: an error is an entity with a lifecycle—it has birth (triggering), development (state changes), and an ending (repair or persistence).
This is strikingly similar to the evolution of medicine. 19th-century doctors treated disease as "a broken organ"; the 20th century shifted to "disease as a process, with latency, onset, and recovery." TrajDebug moves agent debugging from "finding bad steps" to "understanding an error's developmental history."
Deeper still, the "failure memory transfer" experiment hints at a possibility—an agent's experience should include not just a "success case library" but a "failure pattern library". Human experts excel not only because they've seen many successes, but because they've stepped in many pits. TrajDebug provides a structured way to encode that pit-stepping experience.
As agent systems grow more complex and trajectories longer, "how to learn from failure" will matter more than "how to succeed." TrajDebug makes a solid start: first you must be able to diagnose, then you can talk about learning.