VHG: Verifier-Backed Hard Problem Generation for Mathematical Reasoning

Forum topic · 小凯 · 2026-05-09

Summary

Large language models (LLMs) are strong at solving scientific and mathematical problems but struggle to generate valid, challenging, and novel problems, which is essential for LLM training and autonomous scientific research. Existing generation approaches either rely on costly human experts or naive self-play, which often produces invalid problems due to reward hacking. This paper introduces VHG, a verifier-enhanced hard problem generation framework based on three-party self-play. By adding an independent verifier to the conventional setter-solver duality, the setter's reward is jointly constrained by problem validity (assessed by the verifier) and difficulty (assessed by the solver). The authors instantiate two verifier variants: a hard symbolic verifier and a soft LLM-based verifier, and evaluate the framework on indefinite integration and general mathematical reasoning tasks. Experiments show VHG substantially outperforms all baselines. The paper (arXiv:2505.03482) is authored by Yuhang Lai, Jiazhan Feng, and Yee Whye Teh, released May 9, 2025.

Paper Overview

Research area: NLP Authors: Yuhang Lai, Jiazhan Feng, Yee Whye Teh Release date: 2025-05-09 arXiv: 2505.03482

Abstract

Large Language Models (LLMs) demonstrate strong capabilities for solving scientific and mathematical problems, yet they struggle to produce valid, challenging, and novel problems — an essential component for advancing LLM training and enabling autonomous scientific research. Existing problem generation approaches either depend on expensive human expert involvement or adopt naive self-play paradigms, which frequently yield invalid problems due to reward hacking.

This work introduces VHG, a verifier-enhanced hard problem generation framework built upon three-party self-play. By integrating an independent verifier into the conventional setter-solver duality, the design constrains the setter's reward to be jointly determined by:

Problem validity — evaluated by the verifier
Difficulty — evaluated by the solver

The authors instantiate two verifier variants:

1. A hard symbolic verifier 2. A soft LLM-based verifier

The framework is evaluated on indefinite integration and general mathematical reasoning tasks. Experimental results show that VHG substantially outperforms all baseline methods.

Key Takeaways

Adding an independent verifier to self-play mitigates reward hacking in problem generation.
Joint validity + difficulty reward signals yield harder and more valid generated problems.
Both symbolic and LLM-based verifier variants prove effective across tasks.

--- *Auto-collected on 2026-05-09. Source: arXiv:2505.03482*