SkillWrapper: Generative Predicate Invention for Task-level Planning
A joint effort from Brown University and the Allen Institute for AI (AI2), SkillWrapper lets robots "invent" their own causal logic, enabling them to solve long-horizon tasks they have never seen before.
- Paper: SkillWrapper: Generative Predicate Invention for Task-level Planning
- Authors: Ziyi Yang, Benned Hedegaard, Ahmed Jaafar, Yichen Wei, Skye Thompson, Shreyas S. Raman, Haotian Fu, Stefanie Tellex, George Konidaris, David Paulius, Naman Shah
- arXiv: 2511.18203 (v6, 2026-04-13)
- Project page: https://yzylmc.github.io/skill-wrapper/
- Soundness: learned models never produce unsound plans (skills are not executed when preconditions are unmet).
- Completeness: if a task is solvable, the learned model guarantees a solution is found.
- Franka Emika Panda (single arm): manipulating cups, plates, sponges, teapots; learned model with 6 predicates and 1111 grounded instances; 86.7% VLM classification accuracy; failures traced to
gripper_emptyandholdingon sponges whose color matches the background. - Bimanual Kuka: sandwich making with peanut butter, knife, and bread; 6 black-box skills (LeftArmPick, RightArmPick, Open, Scoop, Spread, Drop); multiple dead ends (a dropped knife cannot be picked up again; bread coated in peanut butter cannot be cleaned); improves with iterations and surpasses all baselines; 98.5% VLM classification accuracy.
- Domain shift tests: swapping object instances/configurations dropped Franka
stacked_onaccuracy from 100% to 73.3% (small plates misclassified as saucers); Kuka predicates stayed at 96–100%. - Open-source alternative: with Qwen3-VL-235B, performance is comparable — Qwen3 is more stable on
holding(100% vs GPT-5's 60%) but slightly weaker oncoated(90% vs 100%). SkillWrapper is not tied to closed models. - arXiv: https://arxiv.org/abs/2511.18203
- Project page: https://yzylmc.github.io/skill-wrapper/
- RSS 2025 Workshop version: https://rss25-roboreps.github.io/papers/26_SkillWrapper_Autonomously_L.pdf
- ICRA 2025 Workshop version: https://dyalab.mines.edu/2025/icra-workshop/16.pdf
The Core Problem
Current robots are like construction workers who can carry bricks but cannot build a house: they have skills (pick, place, open) but no causal model of how those skills relate.
1. No causal logic model: a robot knows it can grasp a cup, but not that grasping requires the cup to be unobstructed from above. Without high-level symbolic representations, multi-step manipulation planning is impossible. 2. Long-term dependence on expert-written rules: traditional PDDL planning domains require experts to hand-define all predicates and operators, and they must be rewritten for every new environment.
SkillWrapper's approach: let a vision-language model distill semantic predicates directly from raw RGB images, converting pixel-level perception into high-level symbolic logic.
Core Innovation: Generative Predicate Invention
Instead of human-defined predicates like OnTable(cup) or GripperEmpty(), the system invents them itself. It observes one successful and one failed skill execution, then asks the VLM: "What visual feature explains this difference?" The VLM might propose ClearAbove(object). Each invented predicate has human-readable semantics and is verified for planning usefulness — a generative pixel-to-symbol process, not mere feature extraction.
System Architecture: A Three-Step Loop
1. Active data collection: a foundation model proposes exploratory skill sequences that maximize information gain, targeting boundary cases the current model does not yet understand. 2. Contrastive predicate invention: find a pair of transitions with the same skill but different outcomes; feed both RGB images and the skill description to a VLM; score the proposed predicate on whether it satisfies operator preconditions for successes and not for failures. Prompts (Appendix J) require predicates to be purely visually decidable, banning inferred properties like affordance or reachability. 3. Operator learning: learn PDDL-style operators (preconditions + effects) from data using the invented predicates, usable directly with classical planners such as MetricFF.
The loop iterates: new operators → new exploration → more data → more predicates → better operators.
Theoretical Guarantees: Sound & Complete
Unlike direct VLM-planning methods (e.g., ViLa), SkillWrapper offers formal proofs of:
The guarantee holds given a coverage property on the predicate set, which the invention algorithm progressively approximates.
Experiments
Simulation: Robotouille (hamburger making)
| Method | Easy solve rate | Hard solve rate | Impossible-task recognition | |--------|----------------|-----------------|------------------------------| | Expert Ops (oracle) | 81.0% | 58.1% | 100% | | SkillWrapper | 74.0% | 40.0% | 100% | | System Predicates (8 fixed) | 79.0% | 22.0% | 42.0% | | No Heuristic | 76.0% | 24.0% | 80% | | FM Invent (no filtering) | 23.3% | 1.7% | 63.3% | | ViLa (direct VLM planning) | 46.0% | 13.9% | 20.0% | | Random Exp. | 4.0% | 0% | 100% |
Key findings: on hard tasks SkillWrapper beats all baselines (even the expert-designed fixed predicate set); it recognizes 100% of impossible tasks, directly demonstrating soundness; and it needs a lower planning budget.
Real robots
Key insight: "Foundation models may more effectively ground and interpret predicate semantics when those abstractions are self-generated rather than manually defined by human experts."
VLM Reliability Analysis
Honest Failure Analysis
1. Predicate over-invention: an extra predicate like on_cutting_board(item) fragments data and causes operator overfitting.
2. Semantic redundancy: synonyms/antonyms are not filtered, burdening the VLM classifier.
3. Perception error propagation: a single predicate misclassification can doom a whole plan due to symbolic planning's rigidity.
4. VLM limits: objects deviating from semantic appearance expectations (a tiny plate → saucer) are misidentified.
Why It Matters
1. Bridges the perception–symbol gap with an automatic, interpretable, theoretically guaranteed conversion.
2. Cuts manual engineering cost: no more expert-written PDDL domains — robots figure out what their own skills can do.
3. Fundamentally different from direct VLM planning (ViLa, SayCan): SkillWrapper learns a model first, then plans with classical planners, inheriting decades of domain-independent heuristics and correctness guarantees, while direct VLM action output lacks soundness/completeness and accumulates errors on long horizons.
4. Interpretability: learned predicates are human-readable — you can ask why the robot thinks it cannot pick up an object and get "because ClearAbove(Mug) is false."
Limitations and Future Directions
1. Assumes full observability; partially observable scenes (occluded objects) are unhandled. 2. Balancing predicate invention against data acquisition to avoid overfitting. 3. VLM call costs can be high on long-horizon tasks. 4. Discrete symbols only; continuous parameters (e.g., grasp positions) are out of scope. 5. Exploration could be strengthened with information-theoretic methods.
Related Work Positioning
| Direction | Representative work | Difference from SkillWrapper | |-----------|--------------------|------------------------------| | Direct VLM planning | ViLa, SayCan | No symbolic model, no correctness guarantees | | Handcrafted TAMP | PDDL + motion planner | Requires expert-defined domains | | Automatic domain learning | PDDLLM | Derives from demonstrations, not active exploration | | Neuro-symbolic | Neural Logic Machines | Representations not interpretable | | Contrastive learning | Various concept learning | Not robot-skill focused, no planning guarantees |