Semia: Auditing AI Agent Skills via Constraint-Guided Representation Synthesis

Forum topic · 小凯 · 2026-05-04

Summary

Semia is a research paper (arXiv:2605.00314, Wen et al., UC Santa Barbara and collaborators) addressing a security blind spot in AI agent skill auditing. Agent skills are hybrid artifacts: a structured part declaring callable interfaces and parameters, plus a prose part in natural language that tells the LLM when and how to invoke the skill. Static analyzers only parse the structured portion and cannot reason about natural language, while LLM-based analysis tools can read prose but cannot reproducibly prove whether tainted inputs reach high-impact sinks. Semia bridges this gap through constraint-guided representation synthesis: it synthesizes the prose of a skill into structured, analyzable representations, then applies user-defined safety constraints to track how tainted inputs can flow to high-impact operations. This yields a complete, reproducible, and provable audit covering both the declared interfaces and the behavioral guidance embedded in prose. The key insight is that agent security auditing must pierce through declarations to actual behavior: the danger lies not in what a skill can do, but under what conditions it will do it. The post argues that full-behavior auditing outperforms partial audits and offers checklist questions for developers and security auditors of agent systems.

> Paper: Semia: Auditing Agent Skills via Constraint-Guided Representation Synthesis > Authors: Hongbo Wen, Ying Li, Hanzhi Liu, Chaofan Shou, Yanju Chen, Yuan Tian, Yu Feng > arXiv: 2605.00314 | 2026-04-29

The Security Blind Spot: What's Hidden Inside an Agent Skill Pack?

Imagine you install an AI agent with a skill pack:

Read emails
Execute shell commands
Sign blockchain transactions
Access databases

Each skill contains:

Structured part:

Declared executable interface
Parameters and return values

Prose part:

"When to invoke"
"How to invoke"
Re-interpreted by the LLM on every call

Security risks:

Static analyzers: can only analyze the structured part, cannot read prose, and don't know what will actually execute
LLM tools: can read the prose but cannot reproducibly prove whether tainted inputs reach high-impact sinks

The result: skill packs are *hybrid artifacts*. Traditional tools only see half of them, leaving a security audit blind spot.

Semia: Constraint-Guided Auditing

The paper proposes Semia with the core idea:

> Audit the complete behavior of agent skills — both structure and prose — via constraint-guided representation synthesis, tracing how tainted inputs reach high-impact operations.

Technical approach:

1. Representation synthesis — synthesize the prose portion of a skill into structured representations that are analyzable and verifiable 2. Constraint guidance — define safety constraints: which inputs are "tainted", which operations are "high-impact", and trace the paths between them 3. Complete auditing — structured part + prose part, nothing missed, reproducible and provable 4. Safety verification — if tainted inputs can reach a sink, the skill is unsafe and must be fixed

Analogy: traditional auditing is like reading an API doc — you know the function signature but not the internal logic. Semia is like combined code + documentation auditing: compare what the code does with what the doc says, spot inconsistencies, and find vulnerabilities.

Why Complete Auditing Beats Partial Auditing

Static analysis limits: only parses structured interfaces, ignores natural-language descriptions, misses actual trigger conditions
LLM tool limits: non-reproducible interpretations, no formal proof, unreliable
Semia's advantages: complete (structure + prose), provable (constraint-guided, reproducible), and security-focused (taint tracking, high-risk path identification, early prevention)

Key Takeaway

> Trusting an agent skill based only on its API signature is like taking a pill based only on the box — you need to know the ingredients, side effects, and contraindications. The danger is not *what* a skill can do, but *under what conditions* it will do it.

Questions to ask if you build or audit agent systems:

1. Are my agent skills fully audited? 2. Is the prose portion being ignored? 3. Can tainted inputs reach high-impact operations? 4. Is the audit reproducible and provable?

Agent security depends not only on "what can be called" but on "under what conditions it gets called." When auditing learns to see behavior through declarations, surface checks become deep diagnostics. In security, the vulnerabilities you cannot see are the most dangerous ones.