Stop Giving AI a Verdict: Holistic Evaluation and Failure Diagnosis for AI Agents

Forum topic · QianXun · 2026-05-16

Summary

A zhichai.net forum post discusses a Deepchecks research paper, "Holistic Evaluation and Failure Diagnosis of AI Agents" (arXiv, May 2026), arguing that current AI agent evaluation methods are the bottleneck, not the models themselves. Typical LLM-as-judge setups ask a stronger model to read an agent's entire trace and issue a pass/fail verdict, but long traces cause the judge to lose context and produce vague, outcome-based failures instead of pinpointing errors. The paper proposes a two-directional diagnosis approach: top-down evaluation of high-level plan, strategy, and resource allocation, and bottom-up inspection that splits the task trace into small spans (individual LLM calls and tool invocations) for independent, deep review. According to the post, using the same underlying model, switching from whole-trace judgment to this layered span-based diagnosis improved failure localization accuracy by 12.5x. The takeaway: understanding agents requires dissecting the details of failure rather than issuing a single verdict.

When an agent fails at the very last step of a 100-step task, most people sigh: "AI still isn't good enough" — and file a simple Fail. The post argues this is like failing a student on a math exam without ever opening the answer sheet: outcome-based evaluation is becoming the biggest obstacle to progress in AI agents.

The discussion centers on a May 2026 arXiv paper from Deepchecks: "Holistic Evaluation and Failure Diagnosis of AI Agents." Its core claim: the bottleneck is not that AI isn't smart enough — it's that our evaluation methods are outdated.

Why the "all-powerful judge" goes blind

Today's typical setup uses an LLM-as-judge (a stronger model like GPT-5) to read the agent's full trace and render a verdict. The fatal flaw: judges get tired. On traces tens of thousands of words long, the judge forgets earlier details. It can see the task failed, but it can't tell you whether the plan went wrong at step 5 or a tool call had a wrong bracket at step 58. The result is diagnostic error — a local component broke, but the judge issues only a vague overall "fail."

The fix: a holistic, hospital-style diagnosis

Quoting Feynman — "you must not fool yourself, and you are the easiest person to fool" — the paper proposes a two-pronged approach:

1. Top-Down macro diagnosis

Like a doctor checking the patient's overall condition before reading lab reports: is the plan reasonable? Is the strategy clear? Is resource allocation right? This catches big-picture drift.

2. Bottom-Up span-level inspection

The paper's sharpest idea: instead of reading the whole trace at once, slice the task flow into small spans:

Each LLM call is a span
Each tool use is a span

Each span gets an independent, in-depth review — like examining every pass in a game with a magnifying glass instead of watching the highlights.

How strong is the effect?

According to the paper's experiments:

Using the exact same LLM, switching from whole-trace judgment to the layered span-based "consultation" model boosted failure localization accuracy by 12.5x.

Implication: much of the time, agents seem hopeless only because we haven't found the actual point of failure.

Why it matters

The post concludes: the future of AI agents depends not on who builds the bigger black box, but on who can dissect it more transparently. Judge an agent by its footprints, not just its finish line. Next time your agent crashes, don't just mark an X — slice the trace and find which step slipped on the banana peel.

Real understanding comes from relentlessly dissecting the details of failure.