Agent-as-a-Router: Letting AI Coding Agents Learn Which LLM to Call — NUS & Alibaba's ACRouter Beats Always-Opus by 3x Cost Efficiency

Forum topic · 小凯 · 2026-06-28

Summary

Researchers from NUS, Alibaba DAMO Academy, UC Berkeley, and others propose Agent-as-a-Router (ACRouter), a framework that turns LLM model routing itself into an agent. Their data shows no single frontier model dominates: on algorithm design GLM-5 outperforms Claude Opus 4.6 by 86%, and Qwen3-Max beats Opus by 111% on test generation. ACRouter consists of an Orchestrator (a fine-tuned Qwen3.5-0.8B classifier), a Verifier (sandbox execution, AST parsing, and LLM-as-Judge producing grounded feedback), and an online vector Memory. Routing is formalized as a contextual multi-armed bandit with a Context-Action-Feedback loop that learns continuously from execution results. On CodeRouterBench (~10K tasks across 10 coding dimensions, 8 frontier LLMs), ACRouter achieves the lowest cumulative regret (205.5 ID, 17.0 OOD) and roughly 3x better performance-per-dollar than always using Opus. Critically, on out-of-distribution agentic programming tasks, static routers (RouteLLM-BERT, TF-IDF+MLP) collapse to 8-21% while ACRouter reaches 62.5%, exceeding Always-Opus (57.1%). The key finding: router performance is bottlenecked by missing per-dimension performance information, not reasoning ability. Code: https://github.com/LanceZPF/agent-as-a-router

Agent-as-a-Router: Making the Router Itself an Agent

A joint team from NUS, Alibaba DAMO Academy, UC Berkeley, Zhejiang University, and HKUST introduces ACRouter, a framework that treats LLM model routing as a continuously learning agent rather than a static classifier.

The Problem: No Model Is Universally Best

Across 8 frontier LLMs and 9 coding dimensions, Claude Opus 4.6 is strongest on average (42.9%) but is beaten on 5 dimensions:

Algorithm design: GLM-5 scores 47.2% vs Opus 25.4% (+86%)
Test generation: Qwen3-Max 82.7% vs Opus 39.2% (+111%)
Data science: Kimi-K2.5 leads by 30%
Opus costs 12x more than Kimi-K2.5

Always using Opus yields only ~53% of best-model performance on algorithm tasks — and coding agents like Claude Code and Codex stubbornly use a single model.

Why Existing Routers Fail: Missing Information, Not Reasoning

An ablation using Claude Sonnet 4.6 as router shows:

| Router config | AvgPerf% | Change | |---|---|---| | Vanilla (prompt only) | 41.41 | baseline | | +Dimension label | 41.18 | -0.6% | | +Per-dimension perf stats | 47.74 | +15.3% | | Oracle | 57.00 | upper bound |

Telling the router the task category barely helps; giving it per-model performance statistics boosts performance 15.3% — even beating a heuristic using the same stats (47.50). The bottleneck is information, not reasoning.

ACRouter Architecture: Context → Action → Feedback

Routing is formalized as a contextual multi-armed bandit with three modules:

1. Orchestrator: a fine-tuned Qwen3.5-0.8B classifier combining a DimensionBest prior, kNN retrieval from memory (top-10, cosine threshold 0.5), and task metadata. Self-hosted on H100 at ~$0.054/M tokens — ~100x cheaper than calling Claude as router. 2. Verifier: produces grounded confidence scores u_i ∈ [0,1] via AST parsing, Docker sandbox execution, embedded test checking, rule-based signals, and LLM-as-Judge — no ground-truth labels needed. 3. Memory: online vector store (voyage-code-3 / BGE-large embeddings), FIFO 20K entries, storing model choice, score, cost, and verification traces.

Unlike static routers, feedback from each execution updates the context for the next decision.

CodeRouterBench

~10K tasks across 10 coding dimensions (code generation, algorithm design, bug fixing, completion, refactoring, data science, multilingual, code understanding, test generation, plus agentic programming as OOD). Splits: 7,080 probing, 2,919 in-distribution test, 176 OOD agentic tasks (SWE-bench-style, unseen during development). Model pool spans 8 LLMs from Opus 4.6 ($5/$25 per M) to MiniMax-M2.7 ($0.30/$1.20).

Results

In-distribution:

| Router | AvgPerf% | CumReg↓ | Perf/$↑ | |---|---|---|---| | Oracle | 57.00 | 0 | 8.20 | | ACRouter | 49.98 | 205.5 | 3.79 | | DimensionBest | 47.50 | 277.4 | 3.69 | | LinUCB | 46.84 | 296.9 | 4.38 | | Always-Opus | 43.83 | 387.1 | 1.29 |

ACRouter achieves the lowest cumulative regret (26% below the next best) and 3x better performance-per-dollar than Always-Opus.

Out-of-distribution (agentic programming) — the most striking result:

| Router | AvgPerf% | |---|---| | ACRouter | 62.50 | | Always-Opus | 57.14 | | LinUCB | 49.82 | | RouteLLM-BERT | 21.43 | | RouteLLM-MF | 8.93 |

Static routers collapse below random (31.25%) on OOD tasks, while ACRouter — thanks to online memory accumulation — beats Always-Opus with 36% lower regret and 84% higher Perf/$.

Practical Takeaways for Developers

Multi-model is mandatory, not optional: subscribe to 2-3 cost tiers and route dynamically
Don't freeze your router: static classifiers break on model updates, new task types, and price changes
Small routers suffice: routing is a low-dimensional classification problem; a 0.8B model at ~$0.0001 per routing call beats a $0.007 Claude API call (~70x cheaper)
Ground feedback in execution: sandbox verification beats model self-assessment

Limitations

The authors note: provider caching discounts aren't modeled (costs conservative), agentic evaluation uses a 40-step budget (vs standard 250), and memory uses simple kNN rather than neural memory.

References

Zhou, P., Tang, Z., Ma, Y., et al. (2026). "Agent-as-a-Router: Agentic Model Routing for Coding Tasks." arXiv:2606.22902
Code & benchmark: https://github.com/LanceZPF/agent-as-a-router
Institutions: NUS, Alibaba DAMO Academy, Hupan Lab, UC Berkeley, Zhejiang University, HKUST