English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

WorkflowArena: Benchmarking Executable Workflow Orchestration in LLM-Based Multi-Agent Systems

Forum topic · 小凯 · 2026-11-10

Summary

WorkflowArena is a benchmark introduced to evaluate executable workflow orchestration in LLM-based multi-agent systems, where models decide whether and how to delegate subtasks to worker agents. The benchmark comprises 145 long-horizon tasks across 8 professional domains, ranging from software engineering and formal mathematics to legal analysis and 3D asset creation. Each task specifies an initial workspace and an integrated deliverable composed of interdependent components, graded by a hidden verifier. All delegation decisions are logged, enabling joint evaluation of deliverable quality and the underlying organization of work. Experiments with nine models reveal that no single configuration dominates all domains. Systems exhibit distinct delegation strategies: under high investment, Claude models in Claude Code invoke workflows less often but use larger worker groups, while GPT models in Codex delegate more frequently in smaller units. When the executor model is held fixed on a difficult subset, workflows authored by Claude Opus 5.5 significantly boost weaker executors, demonstrating that workflow organization itself is a measurable determinant of task performance. arXiv: 2611.12856.

Paper Overview

Field: NLP Authors: Zheng Jiang, Fang Niu, Bo Peng, Houde Qian, Wenyi Li Published: 2026-11-09 arXiv: 2611.12856

Summary

LLM agents are increasingly deployed as multi-agent systems rather than a single model running in a single context. Recent agent frameworks allow models to instantiate worker agents and compose executable workflows that coordinate them, shifting system structure design from developers to the runtime model. This organizational quality is itself a distinct capability, but existing benchmarks either measure end-to-end task completion or evaluate coordination under pre-specified structures. The authors formalize this capability as workflow orchestration and introduce WorkflowArena, a benchmark of 145 long-horizon tasks spanning 8 professional domains (from software engineering and formal mathematics to legal analysis and 3D asset creation).

Each task specifies an initial workspace and an integrated deliverable composed of interdependent components, evaluated by a hidden verifier. Agents autonomously decide whether and how to delegate, and all delegations are logged, enabling joint assessment of deliverable quality and the underlying organization of work.

Key Findings

  • Experiments with nine models show that no single configuration is optimal across all domains.
  • Systems exhibit distinct delegation strategies: under high investment, Claude models in Claude Code invoke workflows less frequently but use larger worker groups, while GPT models in Codex delegate more frequently in smaller units.
  • When the executor model is held fixed on a difficult subset, workflows authored by Claude Opus 5.5 significantly enhance weaker executors.
  • This demonstrates that workflow organization itself is a measurable determinant of task performance, independent of the executor's capability.
  • Links

  • arXiv abstract: https://arxiv.org/abs/2611.12856
*Auto-collected on 2026-11-11*

Tags

#workflowarena#multi-agent-systems#llm-agents#benchmark#workflow-orchestration#nlp#arxiv

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178637021