Humans vs. LLMs as Scientists: Active Exploration in Causal Reasoning

Forum topic · 小凯 · 2026-06-06

Summary

A new study compares human adults and large language models on the classic blicket detector causal reasoning task. While passive observation has long shown adults struggle with conjunctive (AND) rules—the 'conjunctive handicap'—this research found that active exploration dramatically eliminates this deficit. In three experiments, adult participants who could freely test a five-block causal machine inferred conjunctive rules nearly as flexibly as disjunctive ones, though requiring more trials (9.6 vs 6.4 on average). Critically, 'passive proposers' who suggested tests but saw others' results performed far worse, showing that the key to learning is observing the outcomes of one's own interventions. When frontier LLMs including GPT-5, DeepSeek-Reasoner, and Gemini-2.5-Flash ran the same task, top human testers remained the most efficient explorers: humans averaged 31.5 seconds per test versus 44.3 for GPT-5 and 65.2 for DeepSeek-Reasoner. AI models also showed a conjunctive handicap and explored less systematically than humans. The paper argues active exploration lets learners gather the evidence they need, and that humans remain the most effective scientists when designing experiments themselves.

Humans vs. LLMs as Scientists: Who Explores Better?

Imagine a mysterious machine with several blocks placed on it. Your task: figure out the machine's rule. Does it light up when *any* block is placed on it (an OR rule), or only when *two specific blocks* are placed together (an AND rule)?

You can only find out by experimenting yourself.

This is the classic "blicket detector" task—the gold-standard paradigm for studying causal reasoning in cognitive science. Decades of research show that adults perform reasonably well on OR rules but frequently fail on AND rules, a phenomenon known as the conjunctive handicap.

But there's a catch: most past studies had participants passively observe an experimenter's demonstration rather than explore on their own. What happens when adults take the initiative?

The Experiments: Humans and AI as Scientists

The research team designed three key experiments:

Experiment 1: Active Exploration vs. Passive Observation

Fifty adult participants faced the "Nexiom detector"—a causal reasoning machine with 5 blocks. They could freely choose which blocks to place on it, observe whether it lit up, and infer the causal rule.

Result: Active exploration dramatically improved adults' conjunctive reasoning. The classic conjunctive handicap appeared in the passive observation condition but nearly vanished under active exploration.

That said, conjunctive rules still required more tests to infer (9.6 trials on average vs. 6.4 for OR rules). Active exploration didn't make the problem easier—it gave learners the means to collect the evidence they needed.

Experiment 2: Passive Proposers

A subtler control: one group of participants could *propose* which tests to run, but couldn't see the results of their own proposals—they only observed another participant's test sequence and outcomes.

Result: Passive proposers performed much worse than active explorers. This shows the advantage of active learning cannot be reduced to "proposing hypotheses" or "choosing tests"—the key is whether you can see the results of your own choices.

Experiment 3: LLMs on the Same Task

The researchers had GPT-5, GPT-5-mini, Gemini-2.5-Flash, DeepSeek-Reasoner, o4-mini, and other models perform the exact same task.

The results were surprising:

| Metric | Human avg. | Human top | GPT-5 | DeepSeek-Reasoner | |--------|-----------|-----------|-------|-------------------| | Tests (conjunctive) | 9.6 | 8.5 | 10.2 | 8.2 | | Tests (disjunctive) | 6.4 | 3.8 | 7.7 | 6.8 | | Time per test (sec) | 31.5 | 23.9 | 44.3 | 65.2 |

Key findings:

1. Top humans remain the most efficient explorers. Humans averaged 31.5 seconds per test, while GPT-5 needed 44.3 and DeepSeek-Reasoner 65.2.

2. AI exploration strategies are less efficient. Although some SOTA models approach human-level accuracy in hypothesis inference, they need more tests to reach the same conclusions.

3. AI also has a "conjunctive handicap." All models needed more tests in the conjunctive condition, just like humans.

4. AI exploration is more "wandering." Humans tend to systematically shrink the hypothesis space—ruling out simple possibilities first, then progressively testing more complex ones. AI exploration paths are more random, lacking this incremental strategy.

Why Does Active Exploration Matter So Much?

The paper's core insight in one sentence: Active exploration doesn't make the problem easier—it enables learners to collect the evidence they need.

Under passive observation, you only see the evidence the experimenter chose to show you. If the experimenter never demonstrates "two blocks placed together," you can never distinguish an OR rule from an AND rule. Your prior—"probably an OR rule, since it's simpler"—never gets challenged.

But active exploration lets you design the critical experiment: place two blocks together and see if the machine lights up. A single test can eliminate the OR hypothesis.

This also explains why passive proposers failed—even if you can propose "place two blocks together," the proposal is meaningless if you can't see the outcome. Causal knowledge is knowledge about how actions change the world, not knowledge about which tests look good.

Implications for Research on Children's Cognition

The paper also resolves a long-standing puzzle: why do young children sometimes outperform adults on certain causal reasoning tasks?

Traditional explanation: children have weaker priors and aren't bound by the OR-rule bias.
New explanation: children are more inclined toward active exploration (they are natural little scientists), while adults tend to rely on prior knowledge for quick inferences. When adults are also allowed to explore actively, they become just as flexible as children.

It's not that adults can't reason—it's that passive observation paradigms constrain them.

Honest Limitations

Small sample size (50 human participants), all Western, educated adults
LLM prompt engineering may influence results—different prompts could yield different exploration strategies
The environment is relatively simple (5 blocks); real-world causal reasoning is far more complex
AI models' "thinking time" (65 seconds per test) may reflect verbose reasoning chains rather than genuine "thinking"

But the study does something important: it compares humans and AI under exactly the same experimental conditions, rather than running separate tests and comparing scores. This "fair competition" paradigm is the right way to understand cognitive differences between humans and AI.

Next time you think AI is "already very smart," remember this experiment: in tasks requiring self-designed experiments, self-collected evidence, and self-drawn inferences, top humans remain the most effective scientists. AI can approach human performance, but its exploration paths are longer, more scattered, and less principled.

Perhaps the core of "doing science" isn't knowing the answer—it's knowing which questions to ask.

---

Paper link: https://arxiv.org/abs/2606.06464