Paper Overview
Field: NLP Authors: Zheng Jiang, Fang Niu, Bo Peng, Houde Qian, Wenyi Li Published: 2026-11-09 arXiv: 2611.12856
Summary
LLM agents are increasingly deployed as multi-agent systems rather than a single model running in a single context. Recent agent frameworks allow models to instantiate worker agents and compose executable workflows that coordinate them, shifting system structure design from developers to the runtime model. This organizational quality is itself a distinct capability, but existing benchmarks either measure end-to-end task completion or evaluate coordination under pre-specified structures. The authors formalize this capability as workflow orchestration and introduce WorkflowArena, a benchmark of 145 long-horizon tasks spanning 8 professional domains (from software engineering and formal mathematics to legal analysis and 3D asset creation).
Each task specifies an initial workspace and an integrated deliverable composed of interdependent components, evaluated by a hidden verifier. Agents autonomously decide whether and how to delegate, and all delegations are logged, enabling joint assessment of deliverable quality and the underlying organization of work.
Key Findings
- Experiments with nine models show that no single configuration is optimal across all domains.
- Systems exhibit distinct delegation strategies: under high investment, Claude models in Claude Code invoke workflows less frequently but use larger worker groups, while GPT models in Codex delegate more frequently in smaller units.
- When the executor model is held fixed on a difficult subset, workflows authored by Claude Opus 5.5 significantly enhance weaker executors.
- This demonstrates that workflow organization itself is a measurable determinant of task performance, independent of the executor's capability.
- arXiv abstract: https://arxiv.org/abs/2611.12856