Key Points
- The core problem: Coordinating many AI coding agents is harder than making agents write code. Orchestration cost scales worse than linearly.
- Eight-slot architecture: Runtime (tmux/process), Agent (claude-code/codex/aider/cursor/opencode/kimicode), Workspace (worktree/clone), Tracker (github/linear/gitlab), SCM (github/gitlab), Notifier (desktop/slack/discord/webhook/openclaw), Terminal (iterm2/web), Lifecycle (Reactions + core). All slots are swappable via the
PluginModuleinterface inpackages/core/src/types.ts. - Session lifecycle: Tracker pulls an issue → Workspace creates a git worktree → Runtime starts a tmux session → Agent works autonomously → Terminal exposes observation → SCM opens a PR → Reactions listen for external events → Notifier alerts humans only on judgment calls.
- Reactions close the feedback loop: YAML-defined handlers for
ci-failed(auto-inject logs back into the agent session with retry limit),changes-requested(route reviewer comments to the right agent with optional escalation), andapproved-and-green(notify human or auto-merge). - Activity detection bypasses agent self-reporting: Claude Code writes structured JSONL event files while running. Orchestrator reads them directly to classify states as active, waiting, idle, or finished. The author notes "agents lie, or at least get confused."
- Self-improvement loop: The system records which prompts produce clean PRs, which need guardrails, which CI failures are flaky vs. real, and which review comments are substantive vs. style nits. Over time it can tighten prompts for agents that tend to over-engineer specific task types.
- Worktree isolation: Uses
git worktree(shared object store, independent working directories) instead of full clones—nearly instant creation, minimal disk overhead, natural branch-to-PR mapping. Limitation: it isolates code, not processes, so port collisions still need port injection or containers. - "30 agents built themselves" is overstated. The developer made all architectural decisions (8-slot design, session lifecycle, config schema); agents filled in implementations. Without the blueprint, agents would not independently invent a plugin architecture.
- "100% AI co-authored" ≠ "0% human involvement." The human role shifted from typing to judgment: architecture, issue triage, PR review, cross-agent conflict resolution, and accepting/rejecting approaches.
- The 41 self-corrected CI failures were execution errors (type errors, lint, test regressions) on a project with strong CI and 3,288 tests. Design-level or architectural defects likely cannot be auto-healed.
- Idle-time limits: Third-party evaluation notes agents idle 90+ seconds between steps waiting for tool approval. 30 concurrent agents ≠ 30 simultaneously productive.
- GitHub: https://github.com/ComposioHQ/agent-orchestrator
- npm: @aoagents/ao
- Author blog: https://pkarnal.com/blog/open-sourcing-agent-orchestrator
- Composio blog: https://composio.dev/blog/the-self-improving-ai-system-that-built-itself
- Third-party evaluation: https://www.augmentcode.com/tools/open-source-agent-orchestrators
Reported Build Metrics
| Metric | Value | |---|---| | TypeScript code | ~40,000 lines | | Test cases | 3,288 | | Plugin packages | 17 | | Total PRs | 102 (61 merged) | | AI-created PRs | 86 (84%) | | Commits (all branches) | 722 | | AI co-authored commits | 100% | | Peak concurrent agents | 30 | | CI failures self-corrected | 41 of 41 (100%) | | CI success rate | 84.6% |
Model split: Claude Opus 4.6 made 512 commits (architecture, cross-package integration), Claude Sonnet 4.5 made 373 (plugins, tests, docs), Claude Sonnet 4.6 made 124. Peak day: 27 PRs merged on Feb 14.
Review Distribution
| Reviewer | Reviews | Inline Comments | Share | |---|---|---|---| | Cursor Bugbot (automated) | 377 | 700 | 69% | | AI agents | 316 | 303 | 30% | | Humans | 13 | 13 | 1% |
Bugbot caught real bugs including shell injection via exec(), path traversal, unclosed intervals, and missing null checks. Agents fixed ~68%, explained ~7% as intentional, and deferred ~4%.
Critical Examination (Feynman-style)
Competitive Positioning
| Dimension | Claude Code Teams | Agent Orchestrator | |---|---|---| | Scope | Multiple Claude instances in one session | Cross-repo, cross-project, cross-agent-type | | Session persistence | Dies with terminal | Survives crashes, SSH drops, reboots | | External integrations | Terminal-bound | GitHub reactions, Linear, Slack, webhooks | | Scale | Small | 30 concurrent, 40+ worktrees | | Plugins | None | 8 swappable slots | | Feedback loop | Manual | Reactions auto-handle CI + review |
Other alternatives: Claude Squad (AGPL, TUI with per-edit approval), Bernstein (Apache, deterministic scheduler with Janitor cleanup), Intent (spec-driven verification), Microsoft Conductor (YAML workflow for Azure DevOps teams). Agent Orchestrator sits between human-in-the-loop and full-auto: milestone gates plus automatic CI retry.
Fit and Limits
Best fit: Medium-to-large codebases with solid CI and test coverage, a clear issue backlog, parallelizable independent tasks, and an existing code-review culture.
Poor fit: Projects without CI (no feedback signal), high-stakes code needing per-edit human approval (financial core, security modules), small projects (orchestrator overhead beats 2–3 manual agents), or fully offline environments requiring GitHub/Linear API access.
Closing Insight
The project illustrates a transition in AI-assisted engineering: from "how well can a single agent code" to "how do you coordinate many agents without human attention becoming the bottleneck." Execution has commoditized; orchestration is the next ceiling. Self-improvement loops raise that ceiling with each cycle. The engineer's role shifts from typing to architecture, judging, and conflict resolution.
> The tool is improving itself through the agents it manages. — pkarnal