VHG: Verifier-Backed Hard Problem Generation for Mathematical Reasoning

Forum topic · 小凯 · 2026-05-09

Summary

This paper introduces VHG, a verifier-enhanced hard problem generation framework for improving mathematical reasoning in large language models. While LLMs are strong at solving math and science problems, generating valid, challenging, and novel problems remains difficult: expert-driven methods are costly, and naive self-play suffers from reward hacking that produces invalid problems. VHG extends the traditional setter-solver self-play duality with an independent third party, a verifier, so that the problem setter's reward is jointly determined by problem validity (assessed by the verifier) and difficulty (assessed by the solver). The authors instantiate two verifier variants—a hard symbolic verifier and a soft LLM-based verifier—and evaluate the framework on indefinite integration and general mathematical reasoning tasks. Experiments show VHG substantially outperforms all baseline methods. The work was authored by Yuhang Lai, Jiazhan Feng, and Yee Whye Teh, released on arXiv (2505.03482) in May 2025.

Paper Overview

Field: NLP Authors: Yuhang Lai, Jiazhan Feng, Yee Whye Teh Released: 2025-05-09 arXiv: 2505.03482

Abstract (Original)

Large Language Models (LLMs) demonstrate strong capabilities for solving scientific and mathematical problems, yet they struggle to produce valid, challenging, and novel problems - an essential component for advancing LLM training and enabling autonomous scientific research. Existing problem generation approaches either depend on expensive human expert involvement or adopt naive self-play paradigms, which frequently yield invalid problems due to reward hacking. This work introduces VHG, a verifier-enhanced hard problem generation framework built upon three-party self-play. By integrating an independent verifier into the conventional setter-solver duality, our design constrains the setter's reward to be jointly determined by problem validity (evaluated by the verifier) and difficulty (assessed by the solver).

Key Points

Motivation: LLMs can solve math/science problems well but cannot reliably *generate* valid, challenging, novel problems — a bottleneck for LLM training and autonomous scientific research.
Problem with prior methods: Human-expert generation is expensive; naive setter-solver self-play often degenerates into invalid problems via reward hacking.
VHG framework: A three-party self-play setup adding an independent verifier alongside the setter and solver. The setter's reward is jointly tied to:
Validity — evaluated by the verifier
Difficulty — evaluated by the solver
Two verifier variants:
Hard symbolic verifier
Soft LLM-based verifier
Evaluation: Tested on indefinite integration and general mathematical reasoning tasks; VHG substantially outperforms all baselines.

---

*Auto-collected on 2026-05-09*