VHG: Verifier-Backed Hard Problem Generation for Mathematical Reasoning

Forum topic · 小凯 · 2026-05-10

Summary

This paper introduces VHG, a verifier-enhanced three-player self-play framework for generating hard mathematical problems for training large language models (LLMs). While LLMs are strong at solving science and math problems, they struggle to generate valid, challenging, and novel questions—a key capability for LLM training and autonomous scientific research. Existing generation approaches either require costly human experts or use naive self-play that often produces invalid questions due to reward hacking. VHG extends the classic proposer-solver dual by adding an independent verifier: the proposer's reward is jointly constrained by problem validity (assessed by the verifier) and difficulty (assessed by the solver). Two verifier variants are instantiated—a hard symbolic verifier and a soft LLM-based verifier—and evaluated on indefinite integration and general mathematical reasoning tasks. Experiments show VHG significantly outperforms all baselines. Authored by Yuhang Lai, Jiazhan Feng, Yee Whye Teh, and Ning Miao; arXiv:2605.06660.

Paper Overview

Field: Machine Learning Authors: Yuhang Lai, Jiazhan Feng, Yee Whye Teh, Ning Miao Published: 2026-05-07 arXiv: 2605.06660

Abstract

Large language models (LLMs) have shown strong capabilities in solving science and mathematics problems, but they struggle to generate valid, challenging, and novel questions—a key element for advancing LLM training and enabling autonomous scientific research. Existing problem generation methods either rely on costly human expert involvement or adopt naive self-play paradigms, which frequently produce invalid problems due to reward hacking.

This work proposes VHG, a verifier-backed hard problem generation framework based on three-player self-play. By integrating an independent verifier into the traditional proposer-solver dual structure, the design constrains the proposer's reward to be determined jointly by problem validity (assessed by the verifier) and difficulty (assessed by the solver).

The authors instantiate two verifier variants:

Hard symbolic verifier
Soft LLM verifier

and evaluate the framework on indefinite integration and general mathematical reasoning tasks. Experimental results show that VHG significantly outperforms all baseline methods by a clear margin.

Key Points

Three-player self-play: proposer, solver, and independent verifier
Reward design couples validity (verifier) with difficulty (solver) to mitigate reward hacking
Two verifier instantiations: hard symbolic and soft LLM-based
Significant improvements over baselines on indefinite integration and general math reasoning

--- *Auto-collected on 2026-05-10.*