Multi-Step Reasoning in Large Language Models: A Survey

This post summarizes the survey "Multi-Step Reasoning in Large Language Models: A Survey" by Aske Plaat, Annie Wong, Suzan Verberne, et al. from Leiden University, published on arXiv (arXiv:2407.11511v2, updated 2025-08-13).

Paper Overview

Although LLMs have revolutionized natural language processing, they show clear limitations on tasks requiring complex, multi-step logical reasoning. Grade-school math word problems—trivial for humans—remain a major challenge, because LLMs generate text by predicting the next most likely token rather than following strict logical rules.

To address this, the research community proposed Chain-of-Thought (CoT) prompting (Wei et al., 2022), which guides models to produce explicit, step-by-step intermediate reasoning. CoT and its variants have brought significant gains on math and logic benchmarks.

Key Contribution: A Three-Stage Taxonomy

The survey's core contribution is a three-stage taxonomy: Generate – Evaluate – Control.

1. Step Generation

How intermediate reasoning steps are produced:

• Hand-crafted prompts (e.g., "Let's think step by step") — intuitive and controllable, but poorly generalized and labor-intensive.
• External knowledge-augmented prompts using knowledge graphs or databases — improves accuracy and reliability, but retrieval and integration are complex.
• Model-generated prompts (e.g., Auto-CoT) — highly automated and generalizable, but data- and compute-hungry.

2. Step Evaluation

Reviewing, verifying, and scoring generated steps to prevent error propagation:
• Self-evaluation: the model critiques its own reasoning; limited by the model's own blind spots.
• Tool-based evaluation: external deterministic tools such as Python interpreters, calculators, and logic solvers; highly reliable and precise.
• External model verification: an independent model scores the steps; more objective, but requires extra data and compute.

3. Control of Reasoning Steps

Managing the overall reasoning flow based on evaluation feedback:
• Greedy selection: pick the best-looking step at each point; fast and simple, but short-sighted.
• Ensemble strategies: self-consistency, path ensembling, model ensembling; robust but computationally expensive.
• Reinforcement learning and tree search: framing reasoning as sequential decision-making with algorithms like Monte Carlo Tree Search, PPO, and A3C; can find globally optimal paths, but at very high cost and with difficult reward design.

Findings and Significance

• On the GSM8K benchmark (~8,500 grade-school math word problems requiring 2–8 solution steps), CoT methods improve accuracy by about +39% over direct-answer baselines.
• Unlike surveys that merely enumerate techniques, this taxonomy offers a process-oriented framework, emphasizing the completeness and systematics of the reasoning pipeline and enabling unified comparison of methods.
• Practically, it points the way toward more reliable and explainable LLM reasoning systems and encourages co-development across all three stages.

Conclusion

The survey argues that robust multi-step reasoning requires treating generation, evaluation, and control as an integrated pipeline. Its Generate–Evaluate–Control taxonomy provides both a clear theoretical framework and a practical roadmap for future research on reasoning in LLMs.

Source: arXiv:2407.11511v2