Can Transformers Really Reason? The Generalization Puzzle in Symbolic Reasoning

Forum topic · 小凯 · 2026-05-04

Summary

A zhichai.net forum post discusses the paper 'To See the Unseen: on the Generalization Ability of Transformers in Symbolic Reasoning' (arXiv: 2604.21632) by Nevena Lazić, Liam Fowl, András György, and Csaba Szepesvári. The paper tests decoder-only Transformers on propositional logic tasks: given premises, decide whether a conclusion follows. Models perform well when test problems reuse variable names seen during training, but performance drops sharply on unseen variables, suggesting the models memorize patterns rather than learn underlying logical rules. A key bottleneck identified is the copy/generation mechanism: conclusions often require copying variable tokens from premises, which models accomplish via memorization for known tokens but fail for novel ones. The post argues this challenges the assumption that scaling alone leads to AGI, since true symbolic reasoning requires applying abstract rules to new instances regardless of surface features. It closes with design questions for AI practitioners about distinguishing memorization from rule learning and testing genuine understanding.

> Paper: To See the Unseen: on the Generalization Ability of Transformers in Symbolic Reasoning > Authors: Nevena Lazić, Liam Fowl, András György, Csaba Szepesvári > arXiv: 2604.21632 | 2026-04-29

The Student Who Memorized Answers but Never Learned the Method

Imagine a math exam. Student A memorized the answers to every practice problem and gets original questions right, but fails on new problem types. Student B understands the underlying principles and can derive solutions even for unseen problems.

Current Transformers are more like Student A.

Trained on massive text corpora, they can "remember" countless patterns. But when faced with tasks requiring genuine symbolic reasoning—such as logical proofs or mathematical derivations—their performance is disappointing, especially when problems contain variable names or structures never seen during training.

Symbolic Reasoning: AI's Achilles' Heel

The study tested decoder-only Transformers on propositional logic reasoning.

The task: given a set of logical premises, determine whether a conclusion holds.

Sounds simple? But the research found:

When test problems use variable names seen during training, the models perform well
When variable names are completely new (unseen), performance drops sharply
The models appear to be "reciting" training patterns rather than "understanding" logical rules

It's like a student who memorized the example "if A then B; A holds; therefore B holds" but fails on "if X then Y; X holds; therefore Y holds" because they've never seen X and Y.

Why Are Transformers Bad at "Variable Generalization"?

A key difficulty lies in the copy/generation mechanism.

In logical reasoning, conclusions often require "copying" variable names from the premises. For example:

Premise: "If it rains, the ground is wet"
Question: "It rained. What about the ground?"
The conclusion must "copy" the statement "the ground is wet"

For variable names seen in training, the Transformer can accomplish this copying through memorization. But for entirely new variable names, it must genuinely "understand" the copy operation—which is exactly what it struggles to do.

What Does This Mean for AGI?

Symbolic reasoning is considered a core capability of general intelligence. Humans can:

Understand abstract rules
Apply them to entirely new concrete situations
Remain unaffected by surface features (such as variable names)

If Transformers cannot do this, then:

"Scaling up" alone may not be enough to achieve AGI
New architectures or training methods are needed to strengthen symbolic reasoning
"Emergent abilities" may be more limited than we imagine

A Feynman-Style Judgment: Knowing the Answer Is Not Understanding the Problem

Feynman famously said:

> "If you can't explain it clearly to a freshman, you don't really understand it yourself."

This is precisely the Transformer's predicament. It can generate correct answers, but when the surface form of a problem changes slightly (different variable names), it gets lost.

True understanding is grasping the abstract structure behind a problem—not memorizing specific instances.

Takeaways

If you're designing AI systems, ask yourself:

1. "Is my model memorizing patterns, or learning rules?" 2. "Does the model perform consistently when surface features of the input change?" 3. "How can I test the model's 'genuine understanding' rather than just 'correct answers'?" 4. "Can symbolic reasoning be enhanced through targeted training tasks?"

This research is a warning: the power of Transformers may come largely from memorization and pattern matching rather than genuine abstract reasoning.

On the road to AGI, we may need to rethink what "understanding" truly means.