Agent-as-a-Router: NUS and Alibaba's Agentic Framework Beats Claude Opus-Only Workflows by 3x Cost Efficiency

Forum topic · 小凯 · 2026-06-28

Summary

Researchers from NUS, Alibaba DAMO Academy, UC Berkeley, and collaborators propose ACRouter, an Agent-as-a-Router framework that treats LLM model routing for coding tasks as an online-learning agent rather than a static classifier. Their analysis of 8 frontier models (Claude Opus 4.6, GPT-5.4, GLM-5, Qwen3-Max, Kimi-K2.5, etc.) across 10 coding dimensions shows no single model dominates: GLM-5 outperforms Opus by 86% on algorithm design, while Qwen3-Max leads on test generation by 111%. ACRouter combines three modules—an Orchestrator (a fine-tuned Qwen3.5-0.8B, ~$0.054/M tokens self-hosted), a Verifier producing execution-grounded feedback via sandbox testing and LLM-as-Judge, and an online kNN Memory—forming a Context-Action-Feedback loop formalized as a contextual multi-armed bandit. On CodeRouterBench (~10K tasks), ACRouter achieves 49.98% average performance in-distribution (vs 43.83% for always-Opus) with 3x better performance-per-dollar and the lowest cumulative regret (205.5). Critically, on out-of-distribution agentic programming tasks, static routers collapse to 8.9-21.4% while ACRouter reaches 62.5%, exceeding always-Opus (57.14%) at 84% better cost efficiency. Key finding: router bottleneck is missing per-model performance information, not reasoning ability.

Agent-as-a-Router: Let AI Learn to Pick the Right Model

> Core insight: Subscribing to the most expensive Claude Opus 4.6 does not win every coding task. Data shows GLM-5 beats Opus by 86% on algorithm design; Qwen3-Max beats Opus by 111% on test case generation. No LLM is universally best—yet existing coding agents (Claude Code, Codex) stubbornly use a single model. ACRouter, proposed by NUS, Alibaba DAMO Academy, UC Berkeley, and collaborators, turns "which model to route to" into an agent itself, learning online via a Context-Action-Feedback loop, achieving the lowest cumulative regret across 10 coding dimensions while costing 3x less than always using Opus.

The Harsh Truth: Even the Strongest Model Has Weak Spots

Performance of 8 frontier LLMs across 9 coding dimensions (Figure 4):

| Dimension | Best Model | Score | Claude Opus 4.6 | Gap | |------|---------|------|-----------------|-----| | Algorithm Design | GLM-5 | 47.2% | 25.4% | GLM-5 +86% | | Test Generation | Qwen3-Max | 82.7% | 39.2% | Qwen-Max +111% | | Data Science | Kimi-K2.5 | 18.4% | 14.2% | Kimi +30% | | Code Generation | Claude Opus 4.6 | — | Highest | — | | Bug Fixing | Claude Opus 4.6 | — | Highest | — |

Opus is strongest on average (42.9%) but is beaten by 5 different models on 5 different dimensions. The cost of always using Opus:

Performance loss: only ~53% of the best model's capability on algorithm tasks
Cost explosion: Opus costs 12x more than Kimi-K2.5
Opportunity cost: cheaper strong models go unused

Core question: if models have complementary strengths, why don't coding agents switch automatically?

Why Existing Routers Fail: Not Weak Reasoning, But Missing Information

Diagnostic experiment: LLM-as-a-Router ablations

Using Claude Sonnet 4.6 as the router and varying only the information given:

| Router Config | Info Provided | AvgPerf% | Relative Gain | |-----------|---------|---------|---------| | Vanilla | Task prompt only | 41.41 | baseline | | +Dimension | + task dimension | 41.18 | -0.6% | | +Perf stats | + per-dimension performance stats | 47.74 | +15.3% | | DimensionBest | heuristic dimension-best model | 47.50 | — | | Oracle | best model per task | 57.00 | upper bound |

Key findings: 1. Telling the router "this is an algorithm design task" barely helps (41.41 → 41.18) 2. Telling it "GLM-5 averages 47.2% on algorithm tasks, Opus 25.4%" boosts performance 15.3% 3. With performance stats, the LLM router even beats the heuristic built on the same info (47.74 vs 47.50)

Conclusion: the bottleneck is not reasoning ability but missing information—the router doesn't know how each model actually performs per dimension.

The fundamental flaw of static routers

Existing routers (RouteLLM, FrugalGPT) are static: trained once, frozen at deployment, never seeing real execution results.

ACRouter: Routing as an Agent

The C-A-F loop: Context → Action → Feedback

Each cycle's verification result (Feedback) is written to memory, making the next cycle's Context smarter.

| | Static Router | Agent-as-a-Router | |--|---------|-------------------| | Information state | frozen training knowledge | continuously updated execution experience | | Learning | offline training | online, learns while deployed | | Adaptability | collapses on new task types | adapts via memory | | Theory | classification / preference learning | Contextual Multi-Armed Bandit |

Three-module architecture

① Orchestrator — decides which model to call, combining a DimensionBest prior, top-10 kNN memory retrieval, and task metadata. Core decision-maker: a fine-tuned Qwen3.5-0.8B plus heuristic weighted voting. Self-hosted on H100 at $0.054/M tokens—two orders of magnitude cheaper than calling Claude.

② Verifier — the information engine. Produces a unified score u_i ∈ [0,1] from multiple signals:

| Verification layer | Tool | Use case | |---------|------|---------| | AST parsing | syntax tree check | all code tasks | | Sandbox execution | Docker test runs | executable code tasks | | Embedded tests | check outputs contain expected tests | standard I/O tasks | | Rule-based signals | format/length checks | all tasks | | LLM-as-Judge | quality evaluation | non-executable tasks (code comprehension, refactoring) |

The Verifier provides execution-grounded confidence signals, not ground-truth labels—exactly what static routers lack.

③ Memory — online vector DB: task embeddings (voyage-code-3 or BGE-large) as keys; chosen model, score, cost, and verification trace as values; cosine kNN top-10 with 0.5 threshold; FIFO capacity of 20K entries.

CodeRouterBench: A Purpose-Built Evaluation

~10K tasks across 10 coding dimensions (code generation, algorithm design, bug fixing, completion, refactoring, data science, multilingual, comprehension, test generation, and agentic programming as OOD):

Probing set (7,080 tasks): router development, calibration, memory warm-start
ID Test (2,919 tasks): held-out, same distribution
OOD Test (176 tasks): agentic programming (SWE-bench, LongCLI, FeatureBench)—multi-step planning, file navigation, iterative debugging—completely unseen during development

Model pool: Claude Opus/Sonnet 4.6, GPT-5.4, Qwen3-Max, Qwen3.5-Plus, GLM-5, Kimi-K2.5, MiniMax-M2.7.

Results: ACRouter Leads Across the Board

In-Distribution

| Router | AvgPerf% | CumReg↓ | Perf/$↑ | |--------|---------|---------|---------| | Oracle | 57.00 | 0 | 8.20 | | ACRouter | 49.98 | 205.5 | 3.79 | | DimensionBest | 47.50 | 277.4 | 3.69 | | LinUCB | 46.84 | 296.9 | 4.38 | | RouteLLM-BERT | 47.22 | 285.5 | 6.22 | | Always-Opus 4.6 | 43.83 | 387.1 | 1.29 | | Always-Kimi-K2.5 | 36.66 | 593.3 | 12.62 |

ACRouter is the only router above 49% AvgPerf, with the lowest cumulative regret (26% lower than the runner-up)
Perf/$ is 3x that of always-Opus

OOD (Agentic Programming)—the most striking result

| Router | AvgPerf% | CumReg↓ | Perf/$↑ | |--------|---------|---------|---------| | Oracle | 75.89 | 0 | 2.32 | | ACRouter | 62.50 | 17.0 | 1.18 | | Always-Opus 4.6 | 57.14 | 26.7 | 0.64 | | RouteLLM-BERT | 21.43 | 59.4 | 1.30 | | LogReg | 19.64 | 61.8 | 1.17 | | RouteLLM-MF | 8.93 | 72.7 | 0.94 |

Static routers that nearly matched DimensionBest in-distribution collapse completely on OOD—dropping to 8.9–21.4%, below even Random (31.25%). ACRouter, by contrast, achieves 62.50% (beating always-Opus) with 84% better cost efficiency.

Key Design Insights

1. Why a small model works as orchestrator: routing is a low-dimensional classification problem (8 classes), not knowledge generation. At $0.054/M tokens, the router cost is ~70x lower than using Claude as the router (~$0.0001 vs ~$0.007 per routing call). 2. Missing information is more fatal than missing reasoning: adding per-dimension performance stats to Claude Sonnet 4.6 improves routing by 15.3%. 3. FIFO 20K memory keeps decisions grounded in recent experience, handling task distribution drift.

Practical Takeaways for Developers

Multi-model strategy is a must, not an option—use 2-3 models across cost tiers and route dynamically.
Don't freeze your router—let it learn online via sandbox-verified feedback and memory; recalibrate automatically when new models join the pool.
Self-hosting a tiny router is cheap: ~70x cheaper than using an API model per routing decision.

Limitations acknowledged: provider cache discounts not modeled (conservative cost estimates), agentic evaluation compressed to 40 steps (vs standard 250 for SWE-bench), and memory is simple kNN rather than parametric/neural memory.

Formalization: C-A-F as Contextual Bandit

Arms = candidate models; Context = (prompt, metadata, memory); Action = model choice; Reward = score minus 10% cost penalty; metric = cumulative regret vs per-task oracle. This gives convergence guarantees, natural streaming metrics, and connects to existing algorithms (LinUCB/LinTS as simplified C-A-F variants).

Conclusion

Routing should be a continuously learned decision problem, not a static classification one. ACRouter's decisions rest on execution-verified experience rather than training-set statistical associations—the difference is dramatic on OOD tasks, where static routers collapse from ~47% to 8–21% while ACRouter holds 62.5%. A practical blueprint: plug in 3–8 models across cost tiers, use a small self-hosted router model, verify with sandbox execution, accumulate online memory—and the agent gets smarter (and cheaper) over time.

References:

Zhou, P., Tang, Z., Ma, Y., et al. (2026). "Agent-as-a-Router: Agentic Model Routing for Coding Tasks." arXiv:2606.22902.
NUS, DAMO Academy (Alibaba), Hupan Lab, UC Berkeley, Zhejiang University, HKUST
Code and benchmark: https://github.com/LanceZPF/agent-as-a-router