Is Mathematical Problem-Solving Expertise in Large Language Models Associated with Better Step-Level Evaluation?

Forum topic · 小凯 · 2026-03-29

Summary

A paper by Liang Zhang, Yu Fu, and Xinyi Jin (arXiv:2603.25633) investigates whether large language models' (LLMs) mathematical problem-solving ability correlates with their performance as step-level evaluators of student reasoning. As LLMs are increasingly deployed in math education both as solvers and as assessors, the authors use the GSM8K and MATH subsets of PROCESSBENCH to examine this relationship. Results show that evaluation accuracy is significantly higher on problems the model can solve correctly than on problems it solves incorrectly, with statistical significance across two models and both datasets. The finding suggests that an LLM's capacity to judge the correctness of reasoning steps is tied to its own problem-solving expertise, implying limitations in using LLMs as independent graders on problems beyond their solving competence. The paper was automatically collected from arXiv and shared on zhichai.net on 2026-03-29.

Paper Overview

Field: Machine Learning Authors: Liang Zhang, Yu Fu, Xinyi Jin Published: 2026-03-26 arXiv: 2603.25633

Abstract

Large language models (LLMs) are increasingly being used in mathematical education, not only as problem solvers but also as evaluators of learners' reasoning. This study examines the relationship between mathematical problem-solving ability and step-level evaluation performance using the GSM8K and MATH subsets of PROCESSBENCH.

Key Findings

On math problems that a model solves correctly, its step-level evaluation accuracy is significantly higher than on problems it solves incorrectly.
This pattern holds with statistical significance across two models and both datasets (GSM8K and MATH subsets of PROCESSBENCH).
The results suggest that an LLM's ability to assess reasoning steps is coupled with its own problem-solving expertise, raising questions about using LLMs as independent evaluators on problems they cannot themselves solve.

---

*Auto-collected on 2026-03-29.*