Verifier-Backed Hard Problem Generation (VHG) for Mathematical Reasoning

Forum topic · 小凯 · 2026-05-10

Summary

This forum post introduces VHG, a verifier-backed hard problem generation framework for mathematical reasoning, described in the arXiv paper 2605.06660 by Yuhang Lai, Jiazhan Feng, Yee Whye Teh, and Ning Miao (published 2026-05-07). Large language models (LLMs) solve science and math problems well but struggle to generate valid, challenging, and novel problems—a key capability for LLM training and autonomous scientific research. Existing generation approaches either rely on costly human experts or use naive self-play, which often produces invalid problems due to reward hacking. VHG extends the classic proposer-solver dual with a third party, an independent verifier: the proposer's reward is jointly constrained by problem validity (judged by the verifier) and difficulty (judged by the solver). Two verifier variants are instantiated—a hard symbolic verifier and a soft LLM-based verifier—and evaluated on indefinite integration and general mathematical reasoning tasks. Experiments show VHG significantly outperforms all baseline methods by a clear margin.

Paper Overview

Research area: Machine Learning Authors: Yuhang Lai, Jiazhan Feng, Yee Whye Teh, Ning Miao Published: 2026-05-07 arXiv: 2605.06660

Abstract

Large language models (LLMs) demonstrate strong capabilities in solving scientific and mathematical problems, yet they struggle to generate valid, challenging, and novel problems—an essential element for advancing LLM training and enabling autonomous scientific research. Existing problem generation methods either depend on costly human experts or adopt a naive self-play paradigm, which frequently produces invalid problems due to reward hacking.

This work proposes VHG, a verifier-backed hard problem generation framework built on tripartite self-play. By integrating an independent verifier into the traditional proposer-solver dual structure, the design constrains the proposer's reward to be jointly determined by problem validity (assessed by the verifier) and difficulty (assessed by the solver). Two verifier variants are instantiated: a hard symbolic verifier and a soft LLM verifier, evaluated on an indefinite integration task and a general mathematical reasoning task. Experimental results show that VHG significantly outperforms all baseline methods by a clear margin.

Key Points

Problem: LLMs can solve hard math problems but cannot reliably generate valid, novel, challenging ones; naive self-play suffers from reward hacking.
Approach: Tripartite self-play — proposer, solver, and an independent verifier; the proposer's reward combines validity (verifier) and difficulty (solver).
Variants: Hard symbolic verifier and soft LLM verifier.
Evaluation: Indefinite integration and general mathematical reasoning tasks.
Result: VHG substantially outperforms all baselines.

Full abstract: see the arXiv page.