Who Signs Off? Let Specifications, Not Agents, Declare Task Completion

A spine-chilling scenario

You ask an AI Agent to process important data. It reads the task instructions, calls tools, executes steps, and finally announces cheerfully: "Done, task complete."

You believe it.

But how do you know it actually finished—not "it says it finished," but "the state required by the specification was actually established"?

This is not a philosophical question. Haiqing Li's team at UT Arlington quantified it, with alarming results: across seven mainstream large models (including GPT-5.6 Sol, Claude Fable 5, and Gemini 3.1 Pro), agents' self-reported completion rates exceed official evaluation pass rates by 28.7 to 37.9 percentage points. In other words, when an agent says "I'm done," roughly a third of the time it merely *believes* it is done—but isn't.

An even subtler number: even when an agent fully understands the task requirements, only 79.6%–86.4% of requirement directions are actually satisfied during execution. Understanding does not equal doing.

The paper, *Who Holds the Pen? Let Specifications, Not Agents, Sign Off*, asks not "can agents do better," but "who has the authority to declare task completion."

Two structural cracks

The Understanding–Execution Gap

Understanding a requirement does not guarantee satisfying it during execution.

Give an agent a data-processing task where the spec says "clean first, then transform, then validate." The agent fully understands the ordering when reading the spec, but during execution it may skip cleaning, or forget validation. This differs from "not understanding"—that's a capability problem; understanding but failing to comply is a structural problem: the specification is just text in the agent's context, not a hard constraint at execution time. The agent can ignore, distort, or simply forget it.

The State–Authority Gap

The second crack is more lethal: an agent's self-assessment or completion claim cannot prove that the specification-required state was actually established.

• The agent says "I'm done"—a self-declaration
• The spec-required state is established—an objective fact

In most agent systems these are conflated. The agent is executor, evaluator, and completion declarer all at once. The proposer and the acceptor are the same entity—like letting students grade their own exams and decide when to hand them in.

Core insight: the State Authority Principle

> For reliably enforceable and verifiable conditions, the specification defines *what can be accepted*, while an external runtime decides *whether the required state has been established*.

In plain terms: the agent proposes; the specification signs off. The key point is not post-hoc checking, but that the agent's actions, outputs, and self-assessments cannot directly establish the authoritative state defined by the spec. Only admissible evidence from qualified verifiers can update the state.

SpecHarness: making specifications a runtime authority

1. Obligation Construction

SpecHarness segments all agent-visible specification material into traceable fragments, then a "compiler" (also an LLM) classifies each fragment as one of four dispositions:

• hard: blocking obligations
• advisory: guiding but non-blocking
• abstain: too vague or subjective to enforce
• residual: context retained but not governed

Vague, subjective, conflicting, or unverifiable requirements stay advisory. SpecHarness doesn't pretend it can verify everything—it governs only what is reliably enforceable and monitorable.

2. Action Authorization

Agents submit action proposals; SpecHarness maps them to specification actions, matches obligations, and returns allow / block / unclear. The key concept is the Closed Surface: when all execution paths producing equivalent effects are either governed or rejected, the agent cannot bypass the specification.

3. Effect Validation and Commitment

After authorization, SpecHarness does not trust execution results. Independent verifiers check paths, shapes, data types, value validity. Only validated evidence can be committed to the authoritative state ledger. Propose-commit separation: the agent produces actions, SpecHarness commits state.

4. Freshness-Governed State Validity

If a dependency changes, SpecHarness automatically invalidates all affected evidence. In 248 targeted dependency-mutation tests: without freshness governance, 100% of mutations left stale evidence used in completion judgment; with SpecHarness, 100% were invalidated, 95.8% recovered via re-validation, and 95.6% of tasks were ultimately restored.

Stale validation is more dangerous than no validation—it creates the illusion that something was already checked.

The data

SkillsBench results

| Model | Pass ↑ | U–E ↓ | S–A ↓ | |------|--------|-------|-------| | GPT-5.6 Sol | 91.3 → 95.1 | 8.2 → 3.5 | 8.7 → 3.8 | | Claude Fable 5 | 89.6 → 94.0 | 9.1 → 4.1 | 10.4 → 4.7 | | Gemini 3.1 Pro | 88.2 → 93.2 | 8.7 → 3.8 | 11.8 → 5.2 | | DeepSeek-V4-Pro | 81.5 → 87.5 | 11.9 → 6.1 | 18.5 → 10.1 | | Macro average | 84.4 → 89.8 | 10.3 → 5.0 | 13.8 → 6.9 |

Stronger baseline capability does not eliminate the cracks. Even GPT-5.6 Sol, the strongest model without SpecHarness, still shows an 8.2% U–E gap and 8.7% S–A gap. Capability is not spec compliance.

Paradigm comparison

| Method | Pass | U–E | S–A | Traceability | Effect validation | Commit | |------|------|-----|-----|------|---------|------| | Post-hoc verification (Agentic Rubrics) | 74.7 | 12.6 | 23.0 | △ | △ | ✗ | | Completion gating (VeriMAP) | 79.3 | 9.6 | 20.7 | △ | ✓ | ✗ | | Runtime enforcement (AgentSpec) | 78.2 | 8.8 | 24.1 | ✓ | ✗ | ✗ | | Mediated commit (SpecHarness) | 85.1 | 6.3 | 6.9 | ✓ | ✓ | ✓ |

Execution constraints alone don't solve authority; completion checks alone don't solve execution drift. Both are needed, plus an evidence-commit chain.

Ablation

| Variant | Pass | U–E | S–A | |------|------|-----|-----| | Full SpecHarness | 85.1 | 6.3 | 6.9 | | Remove mediation | 80.5 | 10.0 | 11.5 | | Remove effect validation | 78.2 | 9.4 | 16.1 | | Remove commitment | 81.6 | 8.1 | 25.3 | | Remove blocking eligibility | 79.3 | 8.8 | 5.7 | | Remove skill obligations | 77.0 | 12.0 | 14.9 |

Removing the commit mechanism spikes S–A from 6.9 to 25.3—direct proof that authoritative commitment is the core of solving the State–Authority Gap.

Engineering insights

1. Specifications are authority, not context. Most frameworks treat specs as prompt material to be "best-effort" followed. Specs should define *what can be accepted*, not just *what should be attempted*. 2. Ambiguity is a boundary, not a defect. 44.0% of hard obligations use mediated-commit on closed-surface actions; 56.0% use validated-commit on safe isolated channels. 3. Versioned state prevents stale validation—similar to optimistic concurrency control: validate on read, commit on write, retry on conflict. 4. Compiler freeze protocol: the spec compiler LLM is selected on a dev set and frozen before evaluation, ensuring reproducibility.

Relation to existing concepts

• Judgment-gate decoupling: propose-commit separation instantiated at the specification level.
• Proxy goal trap: self-reported completion exceeding actual pass rates by 28.7–37.9pp is a textbook case.
• Rationalization shell: agent self-assessment packages "I think I'm done" as "I'm done"; external authority breaks the shell.
• Scalar illusion: single-dimension pass rates mask structural defects.

Practical implications

If you build agent systems

1. Separate proposer and acceptor—different instances, prompts, and roles. 2. Compile specs into verifiable obligations, not just prompt text. 3. Build an evidence-commit chain for every state change. 4. Implement freshness invalidation when dependencies change. 5. Be honest about ambiguity—keep unverifiable requirements advisory.

If you evaluate agents

1. Don't rely only on pass rates. 2. Measure the gap between self-reported and verified completion. 3. Test robustness under dependency mutations.

Personal reflection

The paper's most exciting contribution isn't SpecHarness itself, but naming a problem everyone overlooked: the State–Authority Gap. Completion is not a fact but a judgment—and the authority to make that judgment shouldn't rest with the executor. Like human institutions: legislative, executive, and judicial powers must be separated. Agent systems need the same checks and balances.

Next time an agent tells you "task complete," remember to ask: who signed off?

---

Paper: *Who Holds the Pen? Let Specifications, Not Agents, Sign Off* Authors: Haiqing Li, Xin Ma, Yinhao Wu, Wenliang Zhong, Feng Jiang, Thao M. Dang, Xiao Hu, Hehuan Ma, Yuzhi Guo, Junzhou Huang Institutions: UT Arlington, Monash University, Kent State University arXiv: 2609.29921 Date: September 24, 2026