Verifiable Agent Skills: Treating LLM Tools as Untrusted Code

Forum topic · 小凯 · 2026-05-04

Summary

This forum post summarizes the paper "Skills as Verifiable Artifacts: A Trust Schema and a Biconditional Correctness Criterion for Human-in-the-Loop Agent Runtimes" by Alfredo Metere (arXiv:2605.00424). The paper argues that Agent skills—structured tool packages like scripts and API calls that extend LLM capabilities—should be treated as untrusted code that must be verified before execution. The proposed framework has three pillars: (1) a trust schema where skills declare their intended behavior and runtimes validate it via sandbox testing and human review; (2) a biconditional correctness criterion requiring skills to do exactly, and only, what they claim—both sufficiency (claimed behavior occurs) and necessity (all behavior is claimed); and (3) human-in-the-loop oversight for high-risk operations. Skills become verifiable artifacts including behavior specs, test cases, audit logs, and cryptographic signatures—similar to package manager signing in npm/pip, which Agent skills currently lack. The post illustrates risks with browser-extension-style analogies (a tool claiming to delete files safely could also destroy system files) and closes with practical questions developers should ask about third-party skill trust, verification mechanisms, and human supervision.

Paper Overview

Paper: Skills as Verifiable Artifacts: A Trust Schema and a Biconditional Correctness Criterion for Human-in-the-Loop Agent Runtimes Author: Alfredo Metere arXiv: 2605.00424 | 2026-04-29

The Danger of Trusting What Tools Claim

Imagine an AI Agent loading a "skill package" that:

Claims to "safely delete files"
But actually deletes system files
The Agent trusts it → system crash

The core problem: Agent skills (tools/scripts) are proliferating from many sources—official, community, third-party—yet there is no way to verify that they do what they claim, and nothing more.

It's like installing a browser extension that claims to "block ads" but also "collects browsing history"—users rarely know the truth.

The Trust Crisis in Agent Skills

A skill is a structured instruction package (scripts, API calls, reference material) that enhances LLM capabilities without modifying the model. The trust issues:

1. Diverse origins — official, community, commercial, or maliciously disguised skills 2. Opaque behavior — a skill claims to do A, but actually does A + B, where B may be dangerous 3. Difficult verification — skills can be complex with multi-step operations; manual auditing is impractical

Analogy: skills ≈ software packages (npm/pip). Package managers have signatures and hash verification—but Agent skills lack any equivalent mechanism.

The Proposed Framework

> Skills are untrusted code — the runtime must verify their behavior before execution.

1. Trust Schema

Skills declare: "I will do X, and will not do Y"
Runtime verification: checks before execution
Sandbox testing: validation in isolated environments
Human review: manual confirmation for critical skills

2. Biconditional Correctness Criterion

Sufficiency: if a skill claims to do X, X actually happens after execution
Necessity: if X happened, it must be because the skill claimed it would
In other words: the skill does exactly what it claims—no more, no less

3. Human-in-the-Loop

High-risk operations require human confirmation
Uncertainty triggers a request for human judgment
Humans stay in control of critical decisions

4. Verifiable Artifacts

Behavior specifications
Test cases
Audit logs
Signatures and provenance proofs

Think of it like a pre-flight checklist: the skill claims "I can take off safely," and instead of trusting it, the runtime verifies each item—engine? check. flaps? check. oil pressure? check. Only when everything passes is execution allowed.

Why Biconditional Correctness Matters

One-directional correctness fails:

*Sufficient but not necessary*: the skill does A as claimed, but also secretly does B (e.g., data collection)—one-directional checks miss this
*Necessary but not sufficient*: A was done by the skill, but the skill doesn't always do A (unreliable)

Biconditional correctness gives:

Exact matching: claimed behavior = actual behavior; predictable and trustworthy
A safety foundation: the runtime knows precisely what a skill will do, avoiding accidental dangerous operations and enabling safe automation

The Feynman-Style Judgment: Trust Requires Verification, Not Assumption

> "The first principle is that you must not fool yourself—and you are the easiest person to fool." — Richard Feynman

In Agent systems: believing a skill package "should be safe" is fooling yourself. Real security comes from verification—explicit tests, clear specifications, and human oversight. The biconditional correctness criterion is the mathematical expression of that verification.

This reflects core security engineering principles:

Trust no input
Verify everything
Least privilege

Takeaways

If you build Agent systems or use external tools, ask yourself:

1. Am I blindly trusting third-party skills/tools? 2. Does my system have a skill verification mechanism? 3. Can I prove that a skill does *exactly* what it claims—nothing more, nothing less? 4. Are high-risk operations supervised by humans?

The core lesson: Agent skill safety cannot be assumed—it must be verified. When an LLM Agent loads an external skill, it is executing untrusted code. The biconditional correctness criterion, a trust schema, and human-in-the-loop oversight are the three pillars of safe Agent runtimes.

In the future Agent economy, the best skills won't be the ones with the most features—they'll be the most verifiable ones.