Semia: Auditing AI Agent Skills via Constraint-Guided Representation Synthesis

> Paper: Semia: Auditing Agent Skills via Constraint-Guided Representation Synthesis > Authors: Hongbo Wen, Ying Li, Hanzhi Liu, Chaofan Shou, Yanju Chen, Yuan Tian, Yu Feng > arXiv: 2605.00314 | 2026-04-29

The Security Blind Spot: What's Hidden Inside an Agent Skill Package?

Imagine you install an AI agent with a skill package:

• Read emails
• Execute shell commands
• Sign blockchain transactions
• Access databases

Each skill contains:

• Structured part
• Declares the executable interface
• Parameters, return values
• Prose part
• "When to call it"
• "How to call it"
• Re-interpreted by the LLM at every invocation

Security risks:

• Static analyzers
• Can only analyze the structured part
• Cannot understand prose
• Don't know what will actually execute
• LLM-based tools
• Can read the prose
• But cannot reproducibly prove
• Whether tainted inputs reach high-impact sinks

The problem:

• Skill packages are "hybrid artifacts"
• Traditional tools only see half the picture
• A genuine security-audit blind spot

Semia: Constraint-Guided Representation Synthesis

The paper proposes Semia:

Core idea:

> Audit the full behavior of agent skills — analyzing both structure and prose — via constraint-guided representation synthesis, tracking how tainted inputs can reach high-impact operations.

Technical approach:

1. Representation synthesis

• Synthesizes the skill's prose portion into structured representations
• Analyzable
• Verifiable

2. Constraint guidance

• Define security constraints
• Which inputs are "tainted"
• Which operations are "high-impact"
• Trace the paths

3. Complete audit

• Structured part + prose part
• Nothing missed
• Reproducible
• Provable

4. Security verification

• Can tainted inputs reach a sink?
• If a path exists
• The skill is unsafe
• And needs fixing

An analogy:

• Traditional auditing = reading the API docs
• You know the function signatures
• But not the internal logic
• Semia = code audit + documentation audit
• See how the code is written
• See what the docs say
• Cross-check the two
• Find inconsistencies
• Find security vulnerabilities

Why Complete Auditing Beats Partial Auditing

Limits of static analysis:

• Cannot read prose
• Only parses structured interfaces
• Ignores natural-language descriptions
• Doesn't know actual trigger conditions
• Audit is incomplete

Limits of LLM tools:

• Not reproducible
• Each interpretation may differ
• Cannot prove anything
• Unreliable

Semia's advantages:

• Complete: structure + prose, full audit, nothing missed
• Provable: constraint-guided, reproducible, reliable
• Secure: tracks tainted inputs, identifies high-risk paths, prevents issues early

A Feynman-Style Judgment: Understanding a System Requires Seeing All Its Layers

Feynman famously noted that "knowing the name of something" and "understanding something" are entirely different.

In agent security:

> Trusting an agent skill based only on its API signature is like taking pills based only on the box — you need to know the ingredients, side effects, and contraindications. Semia's insight is that security auditing of agent skills must see through the *declaration* to the *behavior*, because what's truly dangerous is not "what it can do," but "what it will do, under what conditions." Surface ≠ substance. Declaration ≠ behavior. Complete audit > partial audit.

Takeaways

If you build agent systems or do security auditing, ask yourself:

1. "Are my agent skills fully audited?" 2. "Is the prose part being ignored?" 3. "Can tainted inputs reach high-impact operations?" 4. "Is the audit reproducible and provable?"

Semia reminds us: agent security lies not just in "what it can call," but in "under what conditions it will call it."

When agent auditing learns to see through declarations to behavior, it moves from surface checks to a deep examination. In the future of agent security, the best audit isn't the fastest — it's the most complete.

In security, the vulnerabilities you can't see are the most dangerous.