Verifiable Agent Skills: Treating LLM Tools as Untrusted Code
> Paper: Skills as Verifiable Artifacts: A Trust Schema and a Biconditional Correctness Criterion for Human-in-the-Loop Agent Runtimes > Author: Alfredo Metere > arXiv: 2605.00424 | 2026-04-29
1. The Danger of Trusting What a Tool Claims
Imagine your AI agent loading a "skill pack" that:
- Claims to "safely delete files"
- But actually deletes system files
- The agent trusts it → system crash
- Agent skill packs (tools/scripts) are proliferating rapidly
- They come from many sources: official, community, third-party
- How can we verify they truly do what they claim?
- How can we ensure they don't do more than claimed?
- An extension claims to "block ads"
- But also secretly "collects browsing history"
- Users can hardly know the truth
- Structured instruction packs
- Scripts, API calls, reference materials
- They enhance LLM capabilities without modifying the model
- Sufficiency: if the skill claims to do X, then after execution X is indeed done
- Necessity: if X was done, it must be because the skill claimed to do it
- In other words: the skill does *if and only if* what it claims
- Sufficient but not necessary: the skill claims to do A and A is done — but it also secretly does B (data collection). One-way verification cannot detect this.
- Necessary but not sufficient: A is done and indeed by the skill — but the skill sometimes fails to do A (unreliable). Also a problem.
- Exact match: claimed behavior = actual behavior; nothing more, nothing less; predictable and trustworthy
- Safety foundation: the runtime can know precisely what a skill will do, won't accidentally execute dangerous operations, and gains a safety guarantee for automation
The problem:
It's like installing browser extensions:
2. The Trust Crisis in Agent Skills
Definition of a skill:
Trust issues:
1. Diverse sources — official skills, community contributions, third-party commercial skills, and maliciously disguised ones.
2. Opaque behavior — a skill claims to do A but actually does A+B, where B may be dangerous.
3. Difficult verification — skills can be complex with multi-step operations, making manual audits impractical.
Analogy: skill packs ≈ software packages (npm/pip). Package managers have signatures and hash verification—but agent skills lack equivalent mechanisms.
3. The Proposed Framework: Trust Schema and Correctness Criterion
Core thesis: > Skills are "untrusted code" — the runtime must verify their behavior before execution.
Technical approach:
1. Trust Schema — skills declare "I will do X, and I will not do Y"; the runtime checks before execution, sandboxes tests in isolated environments, and routes critical skills through human review.
2. Biconditional Correctness Criterion:
3. Human-in-the-Loop — high-risk operations require human confirmation; uncertain cases escalate to human judgment; humans retain control over critical decisions.
4. Verifiable Artifacts — skills are not just code; they include behavior specifications, test cases, audit logs, and signatures with provenance proofs.
Like an aircraft's pre-flight checklist: the skill claims "I can take off safely," but the runtime doesn't simply believe it. It verifies item by item: engines? checked. Flaps? checked. Oil pressure? checked. Only after everything passes is execution allowed.
4. Why "Biconditional Correctness" Matters
Problems with one-way correctness:
The value of biconditional correctness:
5. A Feynman-Style Judgment: Trust Requires Verification, Not Assumption
Feynman said:
> "The first principle is that you must not fool yourself — and you are the easiest person to fool."
In agent systems:
> "Believing a skill pack 'should be safe' is fooling yourself. Real security comes from verification — with explicit tests, clear specifications, and human oversight. The biconditional correctness criterion is the mathematical expression of that verification."
This reflects fundamental security engineering principles: never trust inputs, verify everything, least privilege.
6. Takeaways
If you build agent systems or use external tools, ask yourself:
1. Am I blindly trusting third-party skills/tools? 2. Does my system have a skill verification mechanism? 3. Can I prove a skill does *if and only if* what it claims? 4. Are high-risk operations supervised by humans?
The paper's core lesson: agent skill safety cannot be assumed — it must be verified.
When an LLM agent loads external skills, it is executing untrusted code. The biconditional correctness criterion, a trust schema, and human-in-the-loop oversight form the threefold safeguard for safe agent operation. In the future agent economy, the best skills will not be the ones with the most features — they will be the most verifiable ones.