BAMAS: Cutting 86% of API Cost in Multi-Agent LLM Systems Without Losing Performance

Forum topic · 小凯 · 2026-07-06

Summary

This article explains BAMAS, the first budget-aware multi-agent architecture for LLM workflows, presented in arXiv paper 2511.21572. Existing multi-agent frameworks such as AutoGen, MetaGPT, and ChatDev optimize for capability but offer no built-in cost control, making enterprise deployment expensive. BAMAS tackles this with two coupled optimizers: an integer linear program (ILP) that selects an optimal mix of LLMs under a hard budget constraint while maximizing weighted performance, and an offline reinforcement learning policy (REINFORCE with entropy regularization) that picks the best collaboration topology from four options: Linear, Star, Feedback, and Planner-Driven. Experiments on GSM8K, MBPP, and MATH use DeepSeek-V3 and GPT-4.1 nano as the candidate model pool. On MBPP the framework achieves 82.6% accuracy at a cost of 529, against MetaGPT with DeepSeek-V3 at 82.2% and cost 3,735, an 86% reduction. GSM8K shows a 62% cost cut at equal accuracy, and MATH delivers both lower cost and higher accuracy than baselines. Budget overruns stay below 3%. The authors also report that the Planner-Driven topology is never selected by the learned policy, while Feedback dominates math tasks and Linear dominates code generation.

Overview

BAMAS (Budget-Aware Multi-Agent Systems), introduced in arXiv:2511.21572 by Liming Yang and colleagues, is the first multi-agent LLM framework that treats budget as a first-class design constraint rather than an after-the-fact cap. The authors report up to 86% API cost reduction on standard benchmarks with no loss in task accuracy.

The Problem: Capability Without Cost Discipline

Frameworks like AutoGen, MetaGPT, and ChatDev let multiple LLM agents collaborate on complex tasks, but a single run can involve 10–50 API calls, each consuming hundreds to thousands of tokens. The authors estimate that running MetaGPT on a coding task with DeepSeek-V3 (priced at $0.27 / $1.10 per million tokens) can exceed $3 per run, scaling to roughly $90,000 per month for 1,000 daily runs. None of these systems ship native budget control, so enterprise teams either pay the bill or manually truncate runs and break task completion.

BAMAS Architecture: Three Components in Series

1. Budget-Constrained LLM Selection via Integer Linear Programming

Given a budget B and a pool of available LLMs ranked by capability tiers (proxied by LMSys Chatbot Arena standings), BAMAS assigns each tier a decision weight such that any higher-tier model outweighs any affordable lower-tier combination. It then solves an ILP with two constraints: total cost stays at or below B, and at least two LLMs are chosen. The solver (PuLP with CBC) returns a lexicographically optimal model pool that prefers the strongest affordable models. Cost estimates use 500 input tokens (double the typical 128–256 to absorb multi-agent context) and the maximum output length observed across a 50-instance training sample.

2. Topology Selection via Offline Reinforcement Learning

With the LLM pool fixed, BAMAS picks a collaboration topology from a curated library:

Linear: sequential reasoning, suited to multi-step inference.
Star: parallel hypothesis generation followed by aggregation, suited to decomposable or multi-view problems.
Feedback: generate–critique–revise loops for iterative refinement.
Planner-Driven: a central planner dynamically coordinates other agents, suited to open-ended tasks.

The policy network takes a task description embedded by MiniLM (384-dim) plus the budget scalar and outputs one of the four topologies. Training uses REINFORCE with entropy regularization over a pre-collected dataset of task, budget, topology, and outcome tuples. The composite reward combines task success (+C_succ or −C_fail), a heavy overflow penalty (−C_overflow), and a savings bonus g(1 − actual_cost / budget) when the run succeeds under budget. A notable empirical result: the Planner-Driven topology is never selected because the policy learns that its cost and instability outweigh its benefits.

3. Agent Instantiation and Execution

The chosen pool and topology are instantiated into roles: high-weight LLMs take critical roles such as Critic or Planner, while low-weight LLMs serve as Executors. Linear and Star topologies use a templated scheduler, Feedback runs a generate–critique–revise loop in which the Critic only audits rather than reasons over the full problem, and Planner-Driven supports dynamic stepwise orchestration. A runtime monitor terminates any task whose accumulated cost exceeds the budget.

Experimental Results

Datasets: GSM8K (1,319 math word problems), MBPP (500 Python tasks), MATH (1,000 competition problems).
Candidate LLMs: DeepSeek-V3 (high performance) and GPT-4.1 nano (low cost).

Key findings:

GSM8K: BAMAS at budget 1,625 reaches 95.3% accuracy at average cost 542.9, matching AutoGen + DeepSeek-V3 at 95.4% and cost 1,425.3, a 62% cost cut.
MBPP: BAMAS at budget 1,250 reaches 82.6% accuracy at cost 529.2, versus MetaGPT + DeepSeek-V3 at 82.2% and cost 3,735.1, an 86% cost reduction with slightly higher accuracy.
MATH: BAMAS at budget 2,000 reaches 81.2% at cost 646.0, beating AutoGen + GPT-4.1 nano at 77.6% and cost 797.2 in both metrics.
Budget adherence: 0/1,319 GSM8K tasks, at most 5/500 MBPP tasks, and at most 30/1,000 MATH tasks exceed their budget, all under 3%.

Ablations confirm that the ILP + RL joint optimization beats a naive cost-aware greedy strategy: at the same 95.3% accuracy on GSM8K, greedy selection costs 1,650.8, three times BAMAS's 542.9. The learned topology policy is task-driven and budget-sensitive. Feedback dominates math (40.1% on GSM8K, 69.8% on MATH), where iterative refinement helps, while Linear dominates MBPP code generation, where step-by-step construction fits better. Low budgets favor simple Linear and Star topologies; high budgets shift selection toward Feedback.

Why It Matters

BAMAS shifts multi-agent design from post-hoc cost caps to proactive budget-aware planning. The ILP guarantees an upper bound on per-run cost at configuration time, and the RL policy continuously rebalances performance against spending. The framework is tunable: at budget 500 on GSM8K it costs 222 with 87.9% accuracy, while at budget 1,625 the cost is 543 with 95.3% accuracy. The ILP frequently mixes strong and weak models rather than picking all top-tier ones, showing that critical roles benefit from strong LLMs while executor roles do not.

Limitations and Outlook

The candidate pool in the experiments contains only two models; richer pools with Claude, Gemini, or open-source models would enlarge the ILP search space but not change the principle. The topology selector depends on pre-collected offline data and needs retraining for new task types. The four topologies are hand-designed and do not cover adaptive mid-run restructuring. Latency is not modeled, although parallel topologies like Star could trade tokens for wall-clock time. Finally, budgets are per-task; streaming or continuously running systems would need a more dynamic allocator.

Takeaway

BAMAS reframes multi-agent cost as a design variable that can be systematically optimized. By coupling an ILP-based LLM selector with an RL-based topology selector, the framework delivers the same accuracy as capability-first baselines while spending a fraction of the budget, an important step toward enterprise-grade deployment of multi-agent LLM systems.

Reference Information

Paper: BAMAS: Structuring Budget-Aware Multi-Agent Systems, arXiv:2511.21572
Authors: Liming Yang, Junyu Luo, Xuanzhe Liu, Yiling Lou, Zhenpeng Chen
Code: https://github.com/chunfenri/BAMAS
Datasets: GSM8K, MBPP, MATH
Candidate LLMs: DeepSeek-V3, GPT-4.1 nano
Topology library: Linear, Star, Feedback, Planner-Driven
Headline numbers: 86% cost cut on MBPP, 62% on GSM8K, lower cost and higher accuracy on MATH, budget overruns under 3%