Auditing Behavioral Detection in LLM Agent Evaluation: What the Trace Shows vs. What the Label Says

Forum topic · 小凯 · 2026-10-24

Summary

A forum post introduces arXiv paper 2610.30491 by Adeniji, Shilpika, and Papka on auditing how LLM agent evaluation labels are derived from execution traces. In a GSM8K-based tool-use setting where calculator outputs are occasionally corrupted, the authors audit three trace-level detectors across six production models and 30 problems per model. They find that final-answer correctness, visible re-verification, and fault propagation can disagree systematically. A key failure mode is silent tool-output override: the model uses a different, often correct, value after the tool returns corrupted output without re-invoking it, so verification detectors record nothing. Human review of 90 traces matched automated labels perfectly for final correctness (67/67), fault impact (19/19), and fault relevance (18/18), but only 67/81 for verification, with 14 disagreements falling into four recurring patterns. Execution budgets also shift results: one model ranged from 21/30 to 27/30 correct as step and token budgets changed, while another stayed between 27/30 and 29/30. The paper advocates measurement transparency—labels should ship with their trace evidence, detector applicability, execution-limit truncation status, and human corrections, supported by a trace viewer.

What the Trace Shows and What the Label Says: Auditing Behavioral Detection in LLM Agent Evaluation

Field: Machine Learning Authors: Idunnuoluwa A. Adeniji, Shilpika, Michael E. Papka arXiv: 2610.30491

Overview

LLM agents are often evaluated by reducing execution traces to labels such as final-answer correctness, tool verification, and fault propagation. These labels are measurements produced by parsers, behavioral detectors, and harness rules, and they may not fully capture what the trace actually shows.

The authors study this problem in a tool-use setting built on GSM8K, where agents call a calculator whose outputs are occasionally corrupted. This provides clear ground truth for what the tool returned, what the model later used, and what the detectors reported.

Experimental Setup

Three trace-level detectors are audited across six production models with 30 problems per model.

Key Findings

Systematic disagreements: Final correctness, visible re-verification, and fault propagation can disagree in systematic ways.
Silent tool-output override: A key failure mode where the calculator returns a corrupted value, but the model later uses a different, often correct, value without calling the tool again. The bad value stops propagating while a verification detector records no re-check.
Human review agreement: Across 90 traces, human review agreed perfectly with automated labels for final correctness (67/67), fault impact (19/19), and fault relevance (18/18), but only 67/81 for verification; all 14 disagreements fall into four recurring patterns.
Execution budget effects: In an ablation across two models, one ranges from 21/30 to 27/30 correct as step and token budgets change, while the other stays between 27/30 and 29/30.

Measurement Transparency

The findings motivate making measurements transparent: an automated label should come with:

The trace evidence behind it
Whether the detector's question applied to that trace
Whether the run was cut off by an execution limit
Any human corrections

A companion trace viewer makes this information inspectable alongside the reported results.