Strategically Diverse Sampling for Self-Training (arXiv 2609.31571)

Forum topic · 小凯 · 2026-09-29

Summary

This paper introduces strategic diversity—substantive variation among approaches to solving a problem—as an alternative principle for constructing self-training data for large language models. Standard self-training pipelines sample responses independently (IID) and filter for correctness, which overrepresents strategies the model already favors. The authors propose two sampling methods: GROOT, which builds a hierarchical tree of solution approaches and samples distinct paths, and Verbalized Sampling (VS), adapted to produce an unstructured set of approaches. On competitive programming and next-chapter prediction tasks, models trained on strategically diverse data outperform IID-trained counterparts on difficult problems and provide strong initializations for reinforcement learning and test-time scaling. Notably, self-training on strategically diverse but incorrect trajectories from Qwen3-4B surpassed IID distillation from a 235B teacher model, suggesting that diversity of approaches may matter more than correctness or teacher scale. Paper: arXiv:2609.31571.

Paper Overview

Field: NLP Authors: Alexander Gurung, Esmeralda S. Whitammer, Mirella Lapata arXiv: 2609.31571

Summary

Many LLM training and inference methods, including RL and test-time scaling, depend on repeated sampling, but they only benefit when responses meaningfully differ. Self-training faces the same challenge: training data is typically constructed by sampling IID responses and filtering primarily for correctness, thereby overrepresenting strategies a model already favours.

This paper investigates strategic diversity—substantive variation among approaches to a problem—as an alternative principle for constructing self-training data. Two sampling methods are proposed:

GROOT: a new method that constructs a hierarchical tree of approaches and samples distinct paths.
Verbalized Sampling (VS): adapted to produce an unstructured set of approaches.

Findings

On competitive programming and next-chapter prediction tasks, models trained on strategically diverse data outperform IID-trained counterparts on difficult tasks.
Strategically diverse data provides a strong initialization for RL and test-time scaling.
Most strikingly, self-training on strategically diverse but incorrect trajectories from Qwen3-4B outperformed IID distillation from a 235B teacher model.

These results challenge popular assumptions about what constitutes useful self-training data, showing that diversity of approaches may matter more than correctness or teacher model size.

Strategically Diverse Sampling for Self-Training (arXiv 2609.31571)

Summary

Paper Overview

Summary

Findings

Links

Tags