Are Large Reasoning Models Good Everyday Reasoners? Benchmark Study Finds Gaps vs. Formal Logic

Forum topic · 小凯 · 2026-10-19

Summary

This paper (arXiv:2610.22538) by Ozeki et al. examines whether large reasoning models (LRMs) can handle everyday logical reasoning, which often departs from textbook formal logic. The authors build a benchmark based on linguistic and pragmatic reasoning phenomena and compare reasoning models with non-reasoning models under controlled prompting conditions. Three key findings emerge: (1) reasoning models show degraded performance when everyday reasoning deviates from formal logical rules; (2) some non-reasoning models with chain-of-thought prompting outperform comparable reasoning models on everyday reasoning; and (3) additional instructive prompts provide only limited control over reasoning behavior. The results suggest current reasoning models do not reliably capture everyday reasoning, pointing to limitations in post-training and motivating improved training objectives and broader evaluation across diverse reasoning forms.

Paper Overview

Field: NLP Authors: Kentaro Ozeki, Risako Ando, Takanobu Morishita, Hirohiko Abe, Koji Mineshima, Mitsuhiro Okada Published: 2026-10-19 arXiv: 2610.22538

Abstract

Large reasoning models (LRMs) that leverage test-time computation achieve strong performance on mathematical and scientific benchmarks and are increasingly deployed in real-world applications. However, their ability to perform everyday logical reasoning, which often departs from textbook formal logic, remains underexplored. In this work, the authors systematically evaluate how well reasoning models align with everyday reasoning patterns by introducing a benchmark based on linguistic and pragmatic reasoning phenomena. They also compare reasoning models with non-reasoning models under controlled prompting conditions.

Key Findings

Degradation on non-formal reasoning: In many cases, reasoning models show degraded performance in everyday reasoning when such reasoning deviates from formal logical rules.
CoT prompting can suffice: Some non-reasoning models with chain-of-thought prompting outperform comparable reasoning models in everyday reasoning.
Limited prompt control: Prompting with additional instructions provides limited control over reasoning behavior.

Implications

These findings suggest that current reasoning models do not reliably capture everyday reasoning and point to limitations in post-training, motivating improved training objectives and evaluation across diverse forms of reasoning.

--- *Auto-collected on 2026-10-19*