Paper Information
| Attribute | Content | |:---|:---| | Title | Probing Embodied LLMs: When Higher Observation Fidelity Hurts Problem Solving | | Authors | Oussama Zenkri, Oliver Brock | | Affiliation | Robotics and Biology Laboratory, Technische Universität Berlin | | arXiv ID | 2605.20072 | | Published | May 19, 2026 | | Category | cs.AI | | Core claim | Higher-fidelity observations degrade problem-solving performance of embodied LLM agents; moderate noise improves performance by breaking repetitive behavior loops |
Imagine this scenario: you walk into a room and face a locked steel treasure chest with four push-pull metal rods. Your task is to crack it open — but all you can do is flip the rods one by one. You don't know which rod connects to which, or in what order. You can only experiment.
Now suppose you're offered three ways to play:
Option 1: A perfect X-ray device. You pull rod A — *ding* — the display precisely shows A at position 0, B at 1, C at 0, D at 1. The world is fully transparent to you.
Option 2: An ordinary color camera. After each flip, it takes a photo sent to your phone. You must compare consecutive images to infer what happened.
Option 3: Option 2 plus a depth sensor — the photos become立体-like, adding distance information.
Which condition lets you open the chest fastest? Like most people, you'd pick the X-ray. More knowledge, better decisions — that's common sense, right?
Wrong. The group with the perfect view was the slowest.
This is the result of a new study by Zenkri and Brock at TU Berlin. They built a physical lockbox, had a real robotic arm manipulate it, used GPT o1 as the brain, and provided three different observation channels. When I read the conclusion, my first thought was: wait, that can't be right. After examining their methodology carefully, I realized it's not only plausible — it's fascinating, because it forces us to rethink why we believe "more information is always better."
1. A Detective in a Lockbox
The Lockbox isn't a sophisticated ML benchmark. It's a wooden board with four toggleable levers, each moving between two positions. Hidden mechanical linkages connect them — for example, whether lever A can move depends on the current positions of B and C. These rules are completely hidden and must be inferred through trial and error.
Cognitive biologists have used such lockboxes for years to study how parrots, raccoons, and mice solve problems. It compresses the core challenge of real-world problem solving to its essence: the effect of actions depends on hidden states, and you must reverse-engineer the dependency structure through interaction history.
On the robotics side: a Franka Emika Panda arm with a soft gripper, an RGB-D camera, and a force sensor. At each step the robot picks a lever to flip, a photo is taken, and GPT o1 decides the next move.
2. Three Eyes, Three Minds
The three observation channels:
- RGB: A standard color photo from a fixed viewpoint. The LLM must compare consecutive images — which lever moved, and how much? Non-trivial reasoning.
- RGB-D: RGB plus an aligned depth map. Theoretically, extra 3D geometry should make inferring state changes easier.
- Ground-truth: Text output of all lever states: "Joint A: position 0, Joint B: position 1..." No camera, just sensors. Theoretically perfect — no visual reasoning needed.
- Action-loop frequency vs. success rate: strongly negative.
- Noise level vs. loop frequency: non-monotonic, with the minimum at p ≈ 40%.
Each condition ran 10 independent trials of up to 20 steps. Metric: unlocking the target lever within the step budget.
3. The Counterintuitive Result
RGB performed best, ground-truth worst, RGB-D in between.
Specifically: RGB reached 80% success in 11 steps; ground-truth needed 15 steps for the same rate — 36% more steps with a cleaner signal. Both plateaued at 80%.
Even stranger: in the ground-truth condition, on roughly 5% of steps the model claimed to observe a state change that never happened — a hallucination despite perfect textual input. GPT o1 occasionally invents a state change when reading "Joint A: position 1."
The RGB group had no such phantom readings — but its visual misjudgment rate was 21.7%. The group with worse perception made more efficient decisions overall.
4. Pouring Garbage into Perfect Observations
The researchers followed up: they added noise directly. Moving to simulation (the robot arm was too expensive to run at scale), they removed vision entirely and kept only text state descriptions. Then, with probability p, they randomly flipped the reported action outcome — the lever actually moved but the model is told it didn't, or vice versa.
P ranged from 0% to 60%, 30 trials per condition.
The truly counterintuitive result: success rate is not monotonically decreasing. It peaked at p = 40% — 2.85x the zero-noise rate.
The curve is an inverted U: 0% noise → 23.3% success; rising with noise until 40%, then falling again at 60%.
You spend an afternoon upgrading your AI's camera, quintupling sensor precision, eliminating all uncertainty — and it turns out letting its perception lie randomly 40% of the time nearly triples problem-solving efficiency.
Something deeper must be going on.
5. The Broken Gyroscope: Repetitive Action Loops
A clue emerged from the trajectories: LLMs repeatedly cycle through action sequences. They pick A, B, C, then a few rounds later pick A, B, C again — despite nothing changing. The paper defines a repetitive action loop as an action subsequence of length ≥ 3 appearing at least twice in one trial.
Two correlations:
Inject random erroneous feedback — "you flipped A, and A didn't move" — and the model is forced to stop and reconsider. That interruption breaks the loop.
In the paper's words: "Accurate feedback may sustain repetitive action loops, whereas erroneous observations can disrupt them."
Some degree of perceptual failure turns out to be a precondition for decision flexibility.
6. What Does This Mean?
Honest caveats first:
First, only OpenAI models were tested. GPT o1 for the physical experiment, GPT-4o for simulation. How DeepSeek, Claude, or Gemini would perform is unknown — the paper states this as a limitation.
Second, only one lockbox instance was tested. Simulation experiments with visual layout variations initially support layout-independent conclusions, but real-world replication is lacking.
Third, "moderate noise helps" cannot be generalized to all tasks. The lockbox has a tiny state space (4 levers × 2 positions = 16 states), fixed dependencies, and low exploration cost. In a large state space with costly errors — autonomous driving, say — injecting noise would likely be catastrophic.
Even with these boundaries, there's a hard insight here for anyone building embodied AI:
Performance metrics themselves can be a trap.
Success rate and step count may reflect the *fortuitous interplay* between perceptual errors and reasoning failures, not genuine reasoning ability. An AI may make the "right" decision because it misperceived — luck and task structure, not intelligence.
In the real world, perception is always noisy, reasoning always imperfect, environments always changing. If you evaluate your AI only under clean conditions — tidy rooms, fixed cameras, perfect annotations — what exactly are you measuring: your AI's capability, or the artificiality of your test environment?
7. A Broader Lesson
Physics has a concept called noise-induced order: in some nonlinear systems, small random perturbations help the system escape local optima and find better global states. Simulated annealing works on this principle.
The pattern found here resembles annealing. An LLM stuck in a repetitive loop is like a system trapped in a local energy minimum. Perfect feedback prevents it from "seeing" alternatives — every step returns the expected answer, so it keeps walking the same path. Randomly flipped observations inject thermal motion; occasional misreports push it off course, giving it a chance to discover genuinely useful action sequences.
But let's not overreach. This is not "noise is good." Noise isn't good — it merely happens, under this specific task structure, to pull the model out of a worse failure mode. The real solution is a model that doesn't fall into loops even with clean feedback. That's the hard problem.
From this angle, the paper is a mirror revealing a structural weakness of LLMs in closed-loop embodied reasoning: they lack an intrinsic exploration drive. When observations say everything is fine, their default is repetition, not variation. Nature gave animals curiosity — a cat will poke at something novel even without explicit reward. LLMs currently have no such thing.
Closing
The authors write that in embodied scenarios where perception and reasoning are simultaneously imperfect and hard to disentangle, success rates may reflect the chance interaction of these imperfections rather than true problem-solving ability.
That sentence deserves to be pinned above the monitor of everyone working on robots + large models.
We tend to treat observation as a passive input to decisions — clearer input → better decisions, linear thinking. This paper, with a physical lockbox and a series of elegant experiments, shows that observation actively shapes decision dynamics. Give a model different eyes, and it becomes a different thinker.
Sometimes "knowing a bit less" makes you more flexible. That sounds anti-intellectual, but it isn't. It's a reminder: don't mistake measured success for genuine capability.
---
*Hashtags from the original post: CrushAI, EmbodiedAI, LLM, Embodied Intelligence, Behavior Probing, Zhichai Systems Lab.*