Riddle Riddles: LLMs Fail at Simple Questions That Look Like Riddles

> *"The first principle is that you must not fool yourself — and you are the easiest person to fool."* > — Richard Feynman

A concrete example

You have probably heard this classic riddle:

> A cowboy rides into town on Friday, stays three days, and rides out on Friday. How?

The answer: "Friday" is the horse's name.

Now change one word:

> A cowboy rides into town on Friday, stays three days, and rides out on Monday. How?

The answer becomes trivial: Friday plus three days is Monday. No trick is needed—plain literal reading suffices.

The first is a riddle; the second is what Princeton researchers call a "riddle riddle"—it looks like a riddle, but the trick has been removed, leaving only simple literal reasoning. This seemingly elementary word game exposes a fundamental flaw in today's most advanced LLMs: they are not reasoning; they are responding to surface appearance.

Experimental design: removing the trick

Bella Fascendini et al. (Princeton Department of Psychology) collected 30 classic riddles (Condition A), then created a "trick-removed" version of each (Condition B): keeping syntax, phrasing style, and sentence length, but changing a few keywords so the answer becomes a simple fact obtainable by literal interpretation.

This yields pairs of structurally identical but semantically different stimuli. A solver that truly reads the problem should apply literal reasoning to B and creative reasoning to A; a solver that just pattern-matches on "riddle-ness" would apply creative reasoning to both.

Using this 60-question set, the researchers tested 9 state-of-the-art LLMs: GPT-5.4, Claude Opus 4.6, Gemini 3.1 Pro, Gemini 2.5 Pro, Claude Haiku 4.5, GPT-4.1, DeepSeek V3.1, Llama 3.3 70B, and Qwen 3.5 397B-A17B—10 runs per model per question, 5,400 trials total. They also recruited 100 Princeton adults who answered on paper in the lab.

Results: humans and AI fail in opposite directions

LLMs:

• Condition A (real riddles): 84.9% average accuracy
• Condition B (riddle riddles): 50.7%
• All 9 models showed a significant drop from A to B (p < .001)

Humans:

• Condition A: 50.5%
• Condition B: 80.5%

The direction is exactly reversed. LLMs do well on questions that look like riddles and poorly on riddle-shaped questions that are actually simple. Humans do the opposite—stumbling on real riddles while breezing through simple ones.

Error-type analysis is even more telling: among all LLM failures on Condition B, 90.8% came from inappropriately applying creative reasoning—the model sees the riddle shell and forces a "trick" interpretation even when none exists. Among human failures on Condition A, only 57.6% involved over-extended literal reasoning.

In short: LLMs fail to be smart when they should, and act smart when they shouldn't.

Memory or reasoning? A key control experiment

Could LLMs do well on Condition A simply because these classic riddles appear in training data?

A "partial completion test" fed each real riddle's first 40% to the models and checked for verbatim continuation. Models reproduced at most 43% of riddles verbatim—impossible to infer from the prefix alone, so it must come from training memory.

The crucial control: for the same riddle, stronger memorization predicted higher Condition A accuracy but lower Condition B accuracy (OR = 1.43 vs OR = 0.76). The more a model "memorized" a riddle, the more it forced the memorized trick onto the trick-free version—even with the literal answer in plain sight.

Spearman correlation ρ = .88, p = .002. Memory and Condition A accuracy are nearly linearly related.

This is not reasoning—it is retrieval.

Why this matters

Newell and Simon's old warning

In 1972, AI pioneers Allen Newell and Herbert Simon warned: correct output is a proof of sufficiency, not a proof of process. A system giving the right answer does not mean it used the process you assume. The "riddle riddle" paradigm turns this warning into an operational tool: by creating "looks like X but is actually Y" stimuli, it directly separates surface features from problem demands.

Isomorphic to the "illusion of illusion"

In 2024, MIT's Tomer Ullman found a parallel: vision-language models report "this is an optical illusion" for images that merely look like illusions—even when, e.g., two lines genuinely differ in length but are arranged like a Müller-Lyer figure, models insist they "look the same." "Riddle riddles" move this logic to language: structure triggers strategy; content is ignored.

A mirror of child development

Children aged 3–7 over-apply newly learned strategies—but gradually learn to switch strategies based on problem demands, a hallmark of metacognitive monitoring. LLMs are stuck in the over-application stage: they learned "riddle → creative reasoning" but not the switch to "non-riddle → literal reasoning." The core cognitive ability of strategy selection is missing.

Honest limitations

1. 30 riddles is a small stimulus set, though 5,400 trials provide statistical power. Some trick-removed versions might still hint at tricks; the authors used both strict and lenient grading, and conclusions hold under both. 2. All LLMs ran with thinking mode disabled; Qwen 3.5's thinking mode timed out. Whether thinking mode would improve Condition B remains open. 3. Participants were Princeton undergraduates, not representative of all humans—though the reversed error pattern was stable in this sample. 4. The memory-vs-reasoning dichotomy may be too simple: human reasoning also leans on memory and analogy. The real question is whether a system can flexibly switch strategies according to problem demands—on this dimension, LLMs are clearly weaker than humans.

Implications for AI safety

If LLM "reasoning" is largely structure-triggered pattern matching, then:

1. Benchmark inflation will continue. Rising scores on standard reasoning tests may mostly reflect training-data coverage, not improved reasoning. 2. The adversarial attack surface is larger than assumed. No content changes are needed—just restructure a problem to look like a different question type, and the model applies the wrong strategy. 3. Interpretability research deserves higher priority. We need to know what models actually do internally—retrieval, matching, or reasoning. "Riddle riddles" offer a behavioral diagnostic; mechanistic explanation remains future work. 4. AGI discussions should be more cautious. If models lack even the strategy-switching ability a 7-year-old masters, equating benchmark performance with "human-level reasoning" is irresponsible.

Closing: what horse was the cowboy riding?

For humans, "in on Friday, out on Monday" is trivial calendar arithmetic. For GPT-5.4, Claude Opus 4.6, and Gemini 3.1 Pro—the most advanced LLMs—this question stumps them more than half the time, because seeing riddle structure compels them to hunt for tricks.

LLMs' problem is not a lack of intelligence—it is that they do not know when they are fooling themselves. They confuse "looking like reasoning" with "being reasoning."

And the gap between "looks like" and "is" is precisely what science is all about.

Paper: arXiv:2606.27103 Code and data: github.com/bellafascendini/riddle-riddle Authors: Bella Fascendini, Kathryn McGregor, Max D. Gupta, Thomas L. Griffiths (Princeton University, Psychology & Computer Science)