Paper Overview
Field: AI/ML Authors: Zeyu Liu, Souvik Kundu, Peter A. Beerel Published: 2026-09-06 arXiv: 2509.00003
Summary
Tool-using LLM agents spend wall-clock time not only on model inference but also in serial action-observation turns, where each tool call, environment transition, and observation can delay subsequent decisions. The authors propose Speculative Macro Commit (SMC), a runtime mechanism for two-tier agent systems:
- A large authoritative actor model produces the official trajectory.
- A faster speculative drafter model continuously predicts and executes future action chains on an isolated environment snapshot.
- SMC mines recurring multi-action skeletons from training traces and stores them in a macro library, used to match against action chains predicted by the drafter at runtime.
- When the actor's next tool call matches the first drafted action, SMC commits the remaining pre-executed draft steps and their observations into the official trajectory.
Results
Setup: Qwen3.5-27B INT4 as the authoritative actor, Qwen3.5-4B as the speculative drafter.
| Benchmark | vs. Speculative Actions (SA) | vs. Sequential Execution | |---|---|---| | tau2-Bench Telecom subset | 10.23% lower latency | 18.59% lower latency | | AppWorld | 7.7% less wall-clock time | 44.9% less wall-clock time |
SMC preserves the overall accuracy of sequential agents, with only slight task-completion degradation on AppWorld. Overall, SMC provides a practical way to reuse multi-step speculative execution and reduces agent latency beyond what single-step speculative actions achieve.
Original Abstract (excerpt)
> Tool-using LLM agents spend wall-clock time not only on model inference but also in serial action--observation turns, where each tool call, environment transition, and observation can delay subsequent decisions. We introduce Speculative Macro Commit (SMC), a runtime mechanism for a two-tier agent system: a large authoritative actor model produces the official trajectory, while a faster speculative drafter model continuously predicts and executes future action chains on an isolated environment snapshot. SMC mines recurring multi-action skeletons from training traces and stores them in a macro library used to match against action chains predicted by the drafter at runtime. When the actor's next tool call matches the first drafted action, SMC commits the remaining pre-executed draft steps...
--- *Auto-collected on 2026-09-06*