Discovering a Shared Logical Subspace in LLMs via Cross-View Canonical Correlation Analysis

Forum topic · 小凯 · 2026-04-23

Summary

This paper investigates whether large language models (LLMs) contain an internal, shared logical subspace that aligns natural-language and symbolic-language views of multi-step reasoning. The authors hypothesize that such a subspace captures logical reasoning capabilities that are independent of surface form. To test this, they apply Canonical Correlation Analysis (CCA) to paired residual activations from natural-language and symbolic-language reasoning chains, learning a low-dimensional subspace with maximum cross-view correlation. They further propose a training-free steering method that guides LLM reasoning along this logical subspace, leveraging complementary reasoning signals from both views. Experiments on four logical reasoning benchmarks show accuracy improvements of up to 11 percentage points, with strong generalization to out-of-domain problems. The work offers a new perspective on LLM logical reasoning by bridging neural and symbolic representations.

Overview

Field: NLP Authors: Feihao Fang, My T. Thai, Yuanyuan Lei Published: 2026-04-21 arXiv: 2604.19716

Abstract (English Translation)

Large Language Models (LLMs) still struggle with multi-step logical reasoning. Existing approaches either purely refine the reasoning chain in natural language form or attach a symbolic solver as an external module.

In this work, the authors instead ask whether LLMs contain a shared internal logical subspace that simultaneously aligns natural-language and symbolic-language views of the reasoning process. Their hypothesis is that this logical subspace captures logical reasoning capabilities in LLMs that are shared across views while remaining independent of surface forms.

To verify this hypothesis, they employ Canonical Correlation Analysis (CCA) on the paired residual activations from natural-language and symbolic-language reasoning chains, learning a low-dimensional subspace with maximum cross-view correlation. Furthermore, they design a training-free steering method that guides LLM reasoning chains along this logical subspace, thereby exploiting the complementary reasoning signals of both views.

Experiments on four logical reasoning benchmarks demonstrate the effectiveness of the approach, with accuracy improvements of up to 11 percentage points and good generalization to out-of-domain problems.

Key Points

Problem: LLMs struggle with multi-step logical reasoning; existing methods either refine natural-language reasoning chains or rely on external symbolic solvers.
Hypothesis: LLMs contain a shared internal logical subspace that captures cross-view reasoning capabilities independent of surface form.
Method: Canonical Correlation Analysis (CCA) is applied to paired residual activations from natural-language and symbolic-language reasoning chains to learn a low-dimensional subspace with maximum cross-view correlation.
Steering Mechanism: A training-free method guides LLM reasoning along the discovered logical subspace, leveraging complementary signals from both natural-language and symbolic views.
Results: Achieves up to 11 percentage point accuracy improvements on four logical reasoning benchmarks, with strong out-of-domain generalization.
Contribution: Bridges neural and symbolic perspectives on logical reasoning by revealing a shared internal subspace within the model.

Source: arXiv:2604.19716