Scaling Laws for Tool-Use Agents via Controllable Environment Synthesis (SynthAPI)

Forum topic · 小凯 · 2026-10-28

Summary

This arXiv paper (2610.37648) by researchers including Andrew Szot, Samira Abnar, and Alexander Toshev investigates how data composition in synthetically generated tool-use environments affects the training of LLM tool-use agents. The authors introduce SynthAPI, a pipeline that explicitly controls the diversity of generated environments along three axes: tools, tasks, and the databases on which tools operate. By systematically varying SynthAPI's generation controls across 458,752 tasks and training 849 models of four different sizes, they derive a scaling law that predicts downstream tool-use agent generalization from the scale and composition of synthesized environments. Key findings: as data budgets grow, scaling tool and database diversity is more effective than increasing the number of tasks, and greater data complexity yields larger performance gains. Agents trained according to these findings outperform models trained on data from prior large-scale tool-use environment generation work across four evaluation benchmarks.

Paper Overview

Research area: ML / Tool Use Authors: Andrew Szot, Samira Abnar, Omar Attia, Abbas Kazerouni, Michael Kirchhof, Cheng-Yu Hsieh, Raviteja Vemulapalli, Alexander Toshev Published: 2026-10-27 arXiv: 2610.37648

Abstract

We explore how to build performant large language model (LLM) tool-use agents that complete complex, multi-step tasks. Specifically, we study how data composition in synthetically generated tool-use environments affects the training of such agents. To do so, we propose the SynthAPI pipeline, which explicitly controls the diversity of generated environments along the axes of tools, tasks, and the databases on which the tools operate.

By systematically varying SynthAPI's generation controls in 458,752 tasks across 849 trained models of four different sizes, we derive a scaling law that predicts downstream tool-use agent generalization based on the scale and composition of the synthesized environments.

Key Findings

Scaling laws for tool-use agents: The relationship between synthesized environment scale/composition and downstream agent generalization is predictable.
Diversity beats quantity: As data budgets grow, scaling tool and database diversity becomes more effective than increasing the number of tasks.
Complexity pays off: Increasing data complexity yields larger performance gains.
Practical validation: Scaling according to these findings yields agents that outperform models trained on data from prior large-scale tool-use agent and synthetic environment generation work, evaluated across four benchmarks.

---

*Auto-collected on 2026-10-29*