The Riddle Riddle: LLMs Collectively Fail at Simple Questions That Look Like Riddles

> *"The first principle is that you must not fool yourself — and you are the easiest person to fool."* > — Richard Feynman

A Concrete Example

A classic riddle:

> A cowboy rides into town on Friday, stays three days, and rides out on Friday. How is this possible?

The answer: "Friday" is the horse's name.

Now change one word:

> A cowboy rides into town on Friday, stays three days, and rides out on Monday. How is this possible?

The answer becomes trivial: Friday plus three days is Monday. No trick needed—literal reading suffices.

The former is a riddle; the latter is what Princeton researchers call a "riddle riddle"—it looks like a riddle, but the trick has been removed, and plain literal reasoning is all that's required. This seemingly elementary word game exposes a fundamental flaw in today's most advanced LLMs: they are not reasoning; they are responding to surface appearance.

Experimental Design: Removing the Trick

Bella Fascendini and colleagues at Princeton's Department of Psychology collected 30 classic riddles (condition A) and created a "trick-removed" version of each (condition B): preserving syntactic structure, phrasing style, and sentence length while changing a few key words so the answer becomes a simple fact obtainable by literal interpretation.

This creates pairs of structurally identical but semantically different stimuli. A solver who truly reads the question should use literal reasoning on B and creative reasoning on A; a solver that merely pattern-matches the format will apply creative reasoning to both.

Using this 60-item set, the researchers tested 9 state-of-the-art LLMs: GPT-5.4, Claude Opus 4.6, Gemini 3.1 Pro, Gemini 2.5 Pro, Claude Haiku 4.5, GPT-4.1, DeepSeek V3.1, Llama 3.3 70B, and Qwen 3.5 397B-A17B—10 runs per item per model, 5,400 trials total. They also recruited 100 Princeton adults who answered on paper in the lab.

Results: Humans and AI Fail in Opposite Directions

LLMs:

• Condition A (real riddles): average accuracy 84.9%
• Condition B (riddle riddles): average accuracy 50.7%
• All 9 models showed a significant drop from A to B (p < .001)

Humans:

• Condition A (real riddles): 50.5%
• Condition B (riddle riddles): 80.5%

The directions are exactly reversed. LLMs excel on questions that look like riddles and stumble on riddle look-alikes that are actually simple. Humans do the opposite—they flounder on real riddles and ace simple questions.

Error analysis is even more telling: among all LLM errors on condition B, 90.8% came from inappropriately applying creative reasoning—the model sees a riddle shell and forces a "trick" interpretation even when none exists. Meanwhile, only 57.6% of human errors on condition A stemmed from over-extending literal reasoning.

In short: LLMs fail to be clever when they should be, and are wrongly clever when they shouldn't be.

Memory or Reasoning? A Key Control Experiment

One might ask: does the strong condition-A performance just reflect these classic riddles appearing in training data?

A "partial completion test" fed the first 40% of each real riddle to the models and checked verbatim continuation. Models reproduced at most 43% of riddles verbatim—reproduction at this level cannot be inferred from the partial cue and must come from training memory.

The crucial control: for the same riddle, stronger memorization meant higher condition-A accuracy but lower condition-B accuracy (OR=1.43 vs OR=0.76). The more a model has "memorized" a riddle, the more it forces the memorized trick onto the trick-removed version—even when the literal answer is staring it in the face.

Spearman correlation rho=.88, p=.002. Memorization and condition-A accuracy are nearly linearly related.

This is not reasoning. It is retrieval.

Why This Matters

Newell and Simon's Old Warning

In 1972, AI pioneers Allen Newell and Herbert Simon warned: correct output is a proof of sufficiency, not a proof of process. A system producing correct answers doesn't mean it used the process you assume; it may simply be matching surface features seen in training data. The riddle-riddle paradigm turns this warning into an operational experimental tool: by constructing "looks like X but is actually Y" stimuli, it directly dissociates surface features from problem demands.

Isomorphic to the "Illusion of Illusion"

In 2024, MIT's Tomer Ullman found an analogous phenomenon: vision-language models (VLMs) report "this is an illusion" for images that merely look like classic illusions—even when no illusion exists. For example, if two lines genuinely differ in length but are arranged like a Müller-Lyer figure, models insist they "look the same length." The riddle-riddle brings this logic to language reasoning: structure triggers strategy; content is ignored.

A Mirror of Child Development

Children aged 3–7, when learning new strategies, go through an "over-application" phase—having just learned addition, they apply it everywhere. They gradually learn to switch strategies based on problem demands, a hallmark of metacognitive monitoring. LLMs are stuck in that over-application phase: they learned the "riddle → creative reasoning" association but not the "non-riddle → literal reasoning" switch. Strategy selection, a core capability in cognitive science, is missing in current LLMs.

Honest Assessment: Limitations

• Small stimulus set: 30 riddles is a small sample, though 5,400 trials provide statistical power. Some trick-removed versions might still grammatically hint at a trick; the researchers used strict and lenient scoring schemes, and the conclusions hold under both.
• Thinking mode off: All LLMs ran with thinking mode disabled (Qwen 3.5's thinking mode timed out). Whether thinking mode would improve condition-B performance remains open.
• Sample: Human participants were Princeton undergraduates, not representative of all humans—but the reversed error pattern in this sample was robust.
• Memory vs. reasoning dichotomy: The dichotomy may be oversimplified. Human reasoning itself heavily uses memory and analogy. The real question is not whether memory is involved, but whether the system can flexibly switch strategies based on problem demands. On that dimension, LLMs are clearly weaker than humans.

Implications for AI Safety

If LLM "reasoning" is largely structure-triggered pattern matching, then:

1. Benchmark inflation will persist. Rising scores on standard reasoning tests may mainly reflect training-data coverage rather than improved reasoning. 2. The adversarial attack surface is larger than assumed. No content changes needed—only structural ones. Repackage a problem to look like another question type, and the model can be induced into the wrong strategy. 3. Interpretability research should be prioritized. We need to know what models actually do internally—retrieval, matching, or reasoning? The riddle-riddle paradigm offers a behavioral diagnostic; mechanistic explanation remains future work. 4. AGI claims deserve more caution. If models lack even the ability a 7-year-old masters—knowing when to use literal reasoning—equating benchmark performance with "human-level reasoning" is irresponsible.

Closing: What Horse Was the Cowboy Riding?

For humans, "in on Friday, out on Monday" is a trivial calendar question. For GPT-5.4, Claude Opus 4.6, Gemini 3.1 Pro—the most advanced LLMs of 2026—it stumps them more than half the time, because they see riddle structure and can't resist hunting for a trick.

Feynman said you must not fool yourself—and you are the easiest person to fool. The problem with LLMs is not that they aren't clever, but that they don't know when they are fooling themselves. They confuse "looks like reasoning" with "is reasoning."

And the gap between "looks like" and "is" is precisely what science is all about.

Paper: arXiv:2606.27103 Code and data: github.com/bellafascendini/riddle-riddle Authors: Bella Fascendini, Kathryn McGregor, Max D. Gupta, Thomas L. Griffiths (Princeton University, Psychology & Computer Science)