AgentFloor: How Far Up the Tool Use Ladder Can Small Open-Weight Models Go?

Forum topic · 小凯 · 2026-05-04

Summary

AgentFloor is a deterministic 30-task benchmark that maps agent tool-use capabilities onto a six-tier ladder, measuring how well small open-weight models can replace large language models in AI agent pipelines. The tiers range from basic instruction following (Tier 1) and single-tool invocation (Tier 2) to multi-step coordination (Tier 3), conditional branching (Tier 4), error handling and recovery (Tier 5), and long-horizon planning under persistent constraints (Tier 6). The core motivation is cost: agent systems typically route every API call through expensive frontier models, even though many calls involve simple, structured, routine operations that smaller models can handle. By identifying which tiers a given small model can reliably reach, AgentFloor enables intelligent routing between models, cutting costs, reducing latency, and improving scalability. The benchmark's framing compares agent design to a construction crew: routine tasks go to junior workers, complex design to senior engineers. The takeaway for builders is that agent system intelligence lies not in using the most powerful model at every step, but in knowing how much capability each step actually requires, balancing capability, cost, and latency.

AgentFloor: A Six-Tier Ladder for Agent Tool Use

> Paper: AgentFloor: How Far Up the Tool Use Ladder Can Small Open-Weight Models Go? > Authors: Ranit Karmakar, Jayita Chatterjee > arXiv: 2605.00334 | 2026-04-29

The Cost Problem in Agent Systems

Consider building an AI agent that handles a request like: "Check tomorrow's weather in Beijing, then book a flight to Shanghai." The agent parses the intent, calls a weather API, calls a flight-booking API, and integrates the results — with every single call typically routed through a large model.

The problems:

Large model APIs are expensive
Many calls are simple, structured, and routine
They don't require GPT-4-level intelligence
Small models may suffice — but it's unclear which steps can be downgraded

The Six-Tier Capability Ladder

AgentFloor is a deterministic 30-task benchmark built around the idea that not every agent task needs a large model. The ladder:

1. Tier 1 — Instruction Following: Understanding simple instructions; basic language comprehension; nearly all models can do this. 2. Tier 2 — Tool Use: Calling a single tool, passing correct arguments, handling returned results. 3. Tier 3 — Multi-Step Coordination: Chaining multiple tools, passing intermediate results, managing state. 4. Tier 4 — Conditional Judgment: Deciding next steps based on results; if/then logic and branching. 5. Tier 5 — Error Handling: What to do when a tool call fails — retry, substitute, or degrade gracefully. 6. Tier 6 — Long-Horizon Planning: Planning under persistent constraints across multi-turn, goal-directed interactions.

The analogy is a construction crew: you don't assign a senior engineer to carry bricks. AgentFloor evaluates what "level of worker" each task requires.

Why Tiered Evaluation Matters

Using large models for everything causes:

Cost explosion — every call burns expensive API tokens
High latency — large model inference is slow, hurting user experience
Wasted resources — overkill for simple tasks

Tiered routing instead offers:

Cost optimization — small models for easy tasks, large models for complex ones
Speed — small models respond quickly on routine operations
Scalability — an economical system that can serve more users

The Key Insight

Giving every task the biggest model is laziness, not cleverness. AgentFloor's insight is that an agent system's intelligence lies not in using the strongest brain at every step, but in knowing how much capability each step actually needs — a higher form of system design.

Questions for Agent Builders

1. Does my agent use a large model for every call? 2. Which steps could be downgraded to a small model? 3. Is there an evaluation framework for tiering? 4. Is cost optimization being considered?

AgentFloor reminds us: agent intelligence is not only about *what a system can do*, but *how many resources it uses to do it*. Once an agent learns intelligent routing, it goes from a resource black hole to an economically efficient service. Knowing which hammer to use is the true craftsman's way.