LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling

Forum topic · 小凯 · 2026-05-12

Summary

This paper introduces an agentic framework in which large language models (LLMs) autonomously improve other LLMs by discovering novel test-time scaling (TTS) methods. Published on arXiv as 2505.05128 in May 2025, the work treats the design of TTS strategies as a search problem that an LLM-driven agent explores through hypothesis generation, experimentation, and refinement. The agent iteratively proposes candidate scaling techniques, evaluates them on downstream tasks, and updates its search policy based on observed gains. The paper reports that discovered methods outperform hand-designed baselines such as self-consistency, best-of-N, and reward-model reranking across reasoning, math, and code benchmarks. Results suggest that automated, agent-based discovery can yield scalable, task-adaptive inference strategies, reducing reliance on human-engineered heuristics and offering a practical path toward self-improving LLM systems.

Overview

Research area: Natural Language Processing (NLP)

Authors: Tong Zheng, Haolin Liu, Chengsong Huang

Released: 2025-05-07

arXiv: 2505.05128

---

Abstract (Original)

Test-time scaling (TTS) has become an effective approach for improving the performance of large language models (LLMs) by allocating additional inference-time computation. Existing TTS techniques, such as self-consistency, best-of-N, and reward-model reranking, are largely hand-designed by human researchers. This work proposes an agentic framework in which an LLM-powered agent autonomously discovers new TTS strategies by formulating hypotheses, running controlled experiments, and iteratively refining candidate methods. The agent treats TTS method design as a search problem over the space of prompting, sampling, and aggregation policies.

Key Points

Problem framing: TTS method design is reformulated as a programmatic search problem that an LLM agent can navigate.
Agentic loop: The agent generates candidate scaling strategies, evaluates them on downstream benchmarks, and updates its internal search policy using observed performance feedback.
Search space: Includes prompting templates, sampling configurations (temperature, top-p, number of samples), aggregation/reranking logic, and reward-model usage.
Benchmark coverage: Evaluation spans reasoning, mathematical problem solving, and code generation tasks.
Baselines compared: Self-consistency, best-of-N sampling, and reward-model reranking.
Findings: Agent-discovered TTS methods outperform strong hand-designed baselines on multiple benchmarks, indicating that automated discovery can match or exceed human-engineered heuristics.
Implications: Reduces manual effort in designing inference strategies and provides a route toward self-improving LLM systems whose inference behavior adapts to task distribution.