Beyond Final Answers: Auditing Trajectory-Level Hallucinations in Multi-Agent Industrial Workflows
One-line summary: Existing hallucination detection only checks whether "the answer is right," but in industrial multi-agent settings, hallucinations propagate, cascade, and amplify at every step of the Thought-Action-Observation loop. This IBM/Columbia study is the first to treat the trajectory itself as the audit unit, finding that procedural hallucinations account for 38.5% of failures and that a simple "clarity of justification" signal predicts hallucination with AUC = 0.908 — outperforming all supervised classifiers.
Why "Correct Answer" ≠ "No Hallucination"
Current LLM hallucination benchmarks (TruthfulQA, MIRAGE, HaluEval) all do the same thing: give the model a question and check whether the final output is factually correct. This works for static QA but fails completely in industrial multi-agent workflows.
Consider a data-center operations scenario:
- Agent 1 (IoT): reads sensor data, detects temperature anomalies
- Agent 2 (FSMR): fault-state modeling, infers likely failures
- Agent 3 (TSFM): time-series forecasting of future trends
- Agent 4 (WO): generates work orders, notifies maintenance
- \(M\): LLM-driven agent modules (here K=4: IoT, FSMR, TSFM, WO)
- \(C\): orchestrator (ReAct or Plan-and-Execute)
- \(T_{tool}\): tool set (sensor APIs, forecast endpoints, work-order systems)
- Action monitoring: efficiently catches procedural hallucinations (47.5% occur in Actions) but misses many factual ones
- Response verification: catches factual hallucinations (40.3% in Responses) but misses procedural ones
- Thought-chain verification: catches referential hallucinations (44.0% in Thoughts)
- Scope hallucinations: evenly distributed, requiring explicit role-boundary awareness
- Paper: arXiv:2605.24219v2, "Beyond Final Answers: Auditing Trajectory-Level Hallucinations in Multi-Agent Industrial Workflows"
- Authors: Harshada Badave, Santosh Borse, Shuxin Lin, Dhaval Patel (IBM), Andrea Gomez, Harshitha Narahari, Sara Carter, Vishwa Bhatt, Aishani Rachakonda (Columbia University)
- Base framework: AssetOpsBench [8]
- Related work: MIRAGE [4,12], TruthfulQA [6], ToolBeHonest [13], AgentBench [7], WebArena [14], Traject-Bench [5], GEPA [1]
- Submission target: NeurIPS Datasets and Benchmarks track
The final answer — "generate a work order" — may be entirely correct. Yet along the way TSFM may cite temperature values the IoT agent never reported, FSMR may skip required validation steps, and WO may embed unconfirmed data. The final answer is right, but the trajectory is rotten.
Core insight of the paper: in agentic contexts, hallucination is not factual fabrication in a single response, but structured deviation from evidence that propagates through sequential, tool-mediated trajectories, often causing cascading operational failures.
The Trajel Framework: Four Components
The paper proposes Trajel (Trajectory + Hallucination), an evaluation framework with four complementary components.
1. Trajectory Structure Modeling
Multi-agent systems are formalized as:
Each step is \(s_t = (a_t, \tau_t, \alpha_t, \omega_t)\), where \(\tau_t\) is the Thought, \(\alpha_t\) the Action, \(\omega_t\) the Observation; a trajectory is the ordered sequence of steps.
Key insight: every step can in principle access all prior evidence, so it may cite, miscite, or fabricate upstream content. TSFM hard-depends on IoT, but the orchestrator may choose the wrong order — an orchestrator's wrong sequencing is itself a hallucination source.
2. A Five-Type Trajectory-Level Hallucination Taxonomy
The paper's central contribution: hallucinations defined as structural predicates over the Thought-Action-Observation trace.
| Type | Symbol | Definition | Context needed | Difficulty | |:---|:---|:---|:---|:---| | Factual | \(h_F\) | Assertions contradict step-level ground truth | Single step | ⭐ easiest | | Referential | \(h_R\) | Citing entities or results absent from prior steps | Trajectory history | ⭐⭐⭐ hard | | Logical | \(h_L\) | Reasoning that does not follow from premises, even if premises are correct | Reasoning chain | ⭐⭐⭐⭐⭐ hardest | | Procedural | \(h_P\) | Skipping, reordering, or fabricating workflow-required steps | Workflow spec | ⭐⭐⭐ medium | | Scope | \(h_S\) | Acting or claiming beyond the agent's role | Agent role definition | ⭐⭐⭐ medium |
Multi-agent uniqueness: scope hallucinations — content may be correct but comes from the wrong agent (e.g., TSFM generating the work order, which is WO's job). This is specific to multi-agent industrial settings.
Key figure: 48.7% of hallucinated trajectories exhibit multiple types — single-label formulations mischaracterize a large share of real failures.
3. The Trajel Dataset
| Statistic | Value | |:---|:---| | Total annotated trajectories | 225 | | Model configurations | 6 (anonymized) | | Industrial tasks | 42 (AssetOpsBench asset operations) | | Annotation institutions | 2 (IBM + Columbia, blind review) | | Human-identified hallucination rate | 68.3% (153/224) | | LLM-judge identification rate | 79.1% (178/225) | | Single-type hallucinations | 79 (51.3%) | | Multi-type hallucinations | 75 (48.7%) |
Two-tier annotation: LLM-as-a-Judge pre-screening → fine-grained human review by both institutions. Human-machine agreement: Cohen's κ = 0.456 (moderate) overall, but near-chance for referential (κ = 0.176) and logical (κ = 0.211) types — precisely the subtlest types requiring human judgment.
4. Three Detection Paradigms
| Paradigm | Input | Model | Captures | Best types | AUC | |:---|:---|:---|:---|:---|:---| | Subtask-level BERT | Single step | Fine-tuned BERT | Local cues | Factual | 0.613 | | Trajectory-level NLI | History premise + current hypothesis | NLI model | Trajectory-wide consistency | Referential, logical | 0.689 | | Long-context Longformer | Full serialized trajectory | Longformer | Global structure | Procedural, scope | 0.599 |
The three are complementary lenses, not competing substitutes.
Five Striking Experimental Findings
Finding 1: Procedural hallucinations account for 38.5% of failures
Among the five types, procedural hallucination (\(h_P\)) is the type industrial settings must guard against most: skipping required steps, reordering dependencies, fabricating nonexistent workflow nodes. It is invisible to final-answer verification — the work order gets generated, but the intermediate steps are a mess.
Finding 2: LLM-as-a-Judge fails at the type level
The LLM judge does well at the binary level (F1 = 0.855) but shows systematic type-level defects:
| Type | LLM-judge F1 | Verdict | |:---|:---|:---| | Procedural | 0.784 | ✅ good | | Factual | 0.719 | ✅ okay | | Scope | 0.719 | ✅ okay | | Logical | 0.258 | ❌ very poor | | Referential | 0.222 | ❌ very poor |
The binary-evaluation trap: 141 trajectories would count as "correctly detected" at the binary level, but type matches are only 58.2% — the judge knows *that* there is a hallucination, but not *what kind*.
Finding 3: No supervised model beats the zero-shot LLM judge
| Model | F1 | AUC | |:---|:---|:---| | Zero-shot LLM-judge | 0.855 | — | | BERT (subtask) | 0.590 | 0.613 | | NLI (trajectory-level) | 0.563 | 0.689 | | Longformer (long-context) | 0.533 | 0.599 |
Supervised training on 225 trajectories cannot match a large prompted model's general reasoning. This suggests (1) the dataset needs major expansion, and (2) hybrid architectures (LLM features + discriminative classifiers) may be the optimal path.
Finding 4: "Clarity of justification" signal achieves AUC = 0.908 — the strongest single predictor
The paper's most striking result. Five execution-quality signals (from AssetOpsBench evaluation dimensions):
| Signal | Hallucination rate when present | When absent | Pearson r | AUC | |:---|:---|:---|:---|:---| | Task completion (TC) | 2.3% | 90.0% | −0.786 | 0.853 | | Data retrieval accuracy (DRA) | 43.0% | 89.6% | −0.502 | 0.771 | | Result verification (RV) | 6.4% | 91.0% | −0.784 | 0.863 | | Agent sequence correct (ASC) | 43.7% | 86.6% | −0.453 | 0.738 | | Clarity of justification (CJ) | 9.1% | 94.3% | −0.833 | 0.908 |
CJ alone beats every supervised classifier (AUC 0.908 vs. NLI's best 0.689). Deep pattern: the clarity of an agent's reasoning is itself the strongest predictor of hallucination — 9.1% hallucination rate when reasoning is clearly argued, 94.3% when it is vague.
Finding 5: A candidate "kill switch" — when CJ and RV are both absent, hallucination rate is 97.1%
Proposed deployment recipe: when the "clarity of justification" and "result verification" signals are jointly missing, hallucination is near-certain (97.1%). This can serve as a real-time termination condition for the orchestrator — two boolean checks, no complex classifier needed.
Hallucination Type × Step-Component Interactions (Guardrail Design)
| Type | Thought | Action | Observation | Response | Share | |:---|:---|:---|:---|:---|:---| | Factual | 18 | 24 | 4 | 31 | 40.3% | | Referential | 11 | 10 | 0 | 4 | — | | Logical | 7 | 12 | 5 | 3 | — | | Procedural | 26 | 57 | 5 | 32 | 47.5% | | Scope | 18 | 22 | 2 | 21 | — |
Design implications:
A single guardrail layer is not enough. Industrial deployments need multi-level guardrails (Action + Response + Thought + role boundaries), each catching different types.
Industrial Deployment: From Post-Hoc Classification to Real-Time Runtime Monitoring
The paper's core practical recommendation:
> Shift from post-hoc verification to lightweight runtime monitors inside the orchestration loop.
Concretely: 1. Integrate execution-quality signals: embed the five binary checks (TC, DRA, RV, ASC, CJ) in the agent loop 2. Candidate termination condition: CJ ∧ RV both absent → 97.1% hallucination rate → trigger pause / human review 3. Layered guardrails: Action monitoring (procedural) + Response verification (factual) + Thought-chain checks (referential) + role boundaries (scope) 4. Multi-label detection: with 48.7% multi-type co-occurrence, single-label classifiers miss nearly half of failures
Limitations and Future Work
| Limitation | Impact | |:---|:---| | Scale | 225 trajectories, single industrial domain (AssetOpsBench) | | Fixed orchestrator | All six models share one orchestrator; alternative architectures untested | | Annotation agreement | Overall κ = 0.456 (moderate); referential and logical κ ≤ 0.211 | | Non-exhaustive taxonomy | Five types not claimed exhaustive; new types may emerge under adversarial conditions |
Future directions: 1. Improved annotation protocols requiring annotators to trace every referential claim to a specific prior step 2. Hybrid architectures combining LLM-derived features with discriminative classifiers and token-level uncertainty 3. Directly measuring inter-agent disagreement rather than relying on execution-signal proxies 4. Extending domains to healthcare, finance, and open-ended web environments