Cortex: A Bidirectionally Aligned Embodied Agent Framework for Long-Horizon Manipulation

Forum topic · 小凯 · 2026-07-08

Summary

This paper introduces Cortex, a bidirectionally aligned embodied agent framework designed to overcome the limitations of Markovian vision-language-action (VLA) models on long-horizon manipulation tasks. While hierarchical dual-system approaches combine high-level planning and low-level execution, a semantic gap often separates VLM-generated plans from VLA-executable motions. Cortex addresses this by introducing a customized planning interface that transmits executable, manageable sub-task plans from the high-level VLM to the low-level VLA. The framework standardizes manipulation sub-tasks into 32 canonical skill primitives and embeds manageability principles, including representative object attributes and improved trajectory reachability, directly into the data generation pipeline. Empirically, Cortex outperforms monolithic baselines by 3.1 percent on Libero-long and 4.1 percent on RoboTwin. Additionally, Cortex's general-purpose VLM, when paired with a fine-tuned VLA, completes unseen real-world long-horizon tasks such as multi-stage chemistry experiments in a zero-shot manner, demonstrating strong cross-domain generalization.

Paper Overview

Research Field: Computer Vision (CV)
Authors: Jiaqi Peng, Xiqian Yu, Delin Feng, Yuqiang Yang, Wenzhe Cai, Jing Xiong, Ganlin Yang, Jinliang Zheng, Jiafei Cao, Xueyuan Wei, Jiangmiao Pang, Yuan Shen, Tai Wang
Release Date: 2026-07-06
arXiv: 2607.05377

Abstract

This paper proposes Cortex, a bidirectionally aligned embodied agent framework featuring a customized planning interface that conveys executable and manageable sub-task plans from a high-level VLM to a low-level VLA. The framework standardizes manipulation sub-tasks into 32 canonical skill primitives and injects manageability principles, such as representative object attributes and improved trajectory reachability, into the data generation pipeline.

Key Results

Libero-long: outperforms monolithic baselines by 3.1%.
RoboTwin: outperforms monolithic baselines by 4.1%.
Zero-shot real-world generalization: by simply pairing Cortex's general-purpose VLM with a fine-tuned VLA, the system completes previously unseen real-world long-horizon tasks, including multi-stage chemistry experiments, without additional training.

Core Contributions

1. Bidirectional alignment framework: bridges the semantic gap between high-level VLM planning and low-level VLA execution. 2. Skill primitive standardization: decomposes manipulation into 32 canonical skill primitives to constrain the planning space. 3. Manageability-aware data generation: incorporates object-level attributes and trajectory reachability into training data. 4. Empirical validation: consistent gains on Libero-long and RoboTwin benchmarks. 5. Zero-shot long-horizon transfer: demonstrated on real-world multi-stage chemistry experiments.

*Auto-collected on 2026-07-06*