Illusions Awareness in CPS Design: Acknowledging That Assumptions Fail Is Safer Than Pretending They Won't

> Paper: *Towards Illusions Awareness in Cyber-Physical System's Design* > arXiv: 2609.17260 (Université Côte d'Azur / I3S/INRIA Kairos)

A Drone Story

A Crazyflie drone flies perfectly in the lab. It follows walls, uses an optical flow sensor to keep distance, and an IMU to maintain attitude. All tests pass.

Deployed in a real warehouse — it crashes.

Why? The lab walls were white and matte; the warehouse walls were reflective metal. The optical flow sensor fails on reflective surfaces, but the drone's avoidance logic still assumes "optical flow readings are reliable." That assumption holds in the lab, not in the warehouse.

The paper gives this phenomenon — an assumption failing at runtime — a precise name: design illusion.

What Is a "Design Illusion"

An illusion is not a bug and not a failure. It is a state of persistently relying on an assumption falsified by reality, while maintaining false confidence in the system's correctness.

The definition has three key parts:

1. The assumption was reasonably held: the designer had grounds to believe "optical flow is reliable" at the time 2. Runtime has falsified it: the real environment proves the assumption wrong 3. The design model still depends on it: the system has no mechanism to recognize "the assumption has failed" and keeps running on the original logic

Together, these form the illusion. What the designer "sees" is perfect lab performance; what is "real" is warehouse failure. The gap between them is the reality gap.

The paper's core argument: the mainstream CPS engineering approach is to "shrink the reality gap" — more realistic simulation, more precise sensors, more robust models. But that is not enough. We also need the ability for a system to recognize it is in an illusion when assumptions inevitably fail.

Four Steps of Illusions Awareness

The paper proposes a methodology called Illusions Awareness, in four steps:

1. Identify

Detect "candidate illusions" through runtime monitoring by comparing design-model expectations with actual observations. If the drone expects optical flow readings within [0.5, 2.0] and reads 0.01, that's a candidate illusion.

But identification is only the first step — a single deviation may be noise. Repeatedly occurring deviation patterns are needed to confirm an illusion.

2. Classify

Classify confirmed illusions along two dimensions:

• Explicit vs. implicit assumptions: explicit assumptions are documented by designers ("the sensor works at 0–40°C"); implicit ones are dependencies designers never realized ("walls are matte")
• Architectural layer: perception (sensors), computation (decision-making), actuation (control)

Classification makes illusions reasonable to reason about. Different illusion types call for different strategies.

3. Characterize

For each illusion, characterize four dimensions:

• Origin: how did the assumption arise — deduction (from theory) or induction (from data)?
• Validity: under what conditions does the assumption hold? Where are the boundaries?
• Confidence: how reliable is it? Based on how much data? What sensor quality?
• Impact: if the assumption fails, what happens? Drift? Crash? Mission failure?

Together these form an illusion's "ID card" — the basis for deciding how to handle it.

4. Leverage

The most crucial step: illusions are not enemies to eliminate but resources to exploit. Every identified and characterized illusion is design knowledge — it tells you "this assumption fails under these conditions."

The paper's drone example: once the optical flow illusion is identified, you can:

• Add an "optical flow confidence" input to the avoidance logic, switching to ultrasonic mode when readings are anomalous
• Document "the optical flow assumption only holds on matte surfaces"
• Run dedicated pre-deployment tests on reflective surfaces

These aren't bug fixes — they convert illusions into design assets.

Why "Shrinking the Reality Gap" Isn't Enough

The mainstream CPS path is finer simulation, more robust models, more comprehensive testing — all shrinking the reality gap.

But the paper points out a fundamental problem: the reality gap cannot be fully eliminated:

1. Open world: CPS operate in open environments; scenarios cannot be exhaustively enumerated 2. Sensor limits: every sensor has blind spots and failure conditions 3. Temporal change: environments change (day/night, summer/winter, new/old buildings)

If the reality gap is irremovable, illusions awareness is a necessary complementary capability. Not preventing illusions entirely, but enabling the system to recognize and respond to them.

This matches what human experts do. An experienced pilot never assumes "instruments are always accurate" — they cross-validate instruments and enter a cautious mode when readings disagree. Experts aren't error-free; they can recognize when they err.

Parallels with LLM Hallucinations

CPS "design illusions" and LLM "hallucinations" are strikingly similar in structure:

| Dimension | CPS design illusion | LLM hallucination | |-----------|--------------------|--------------------| | Root cause | Design assumptions fail at runtime | Training distribution fails at inference | | Manifestation | System acts on wrong assumptions | Model generates from wrong distribution | | Detection | Runtime monitoring of deviations | Output confidence estimation | | Response | Mode switching / degradation | Refusal / asking for help |

Both face the same fundamental question: how can a model trained in a closed world recognize its limits in an open world.

LLM practice today — alignment training (RLHF) and inference-time guardrails — resembles CPS's "shrink the reality gap." What LLMs lack is "runtime illusions awareness" — recognizing "I am hallucinating" during generation and switching modes.

Limitations

The paper honestly acknowledges:

1. The framework is conceptual: no concrete implementation, no quantitative evaluation 2. The taxonomy may be incomplete — more illusion types likely exist 3. The "leverage" step is vague — direction given, method not 4. Only the Crazyflie drone case is discussed; applicability to other CPS (autonomous driving, industrial control) is unverified

My Take: Assumptions as Debt, Engineered

Reading this, an analogy came to mind: design assumptions are like technical debt.

• Assumptions are reasonable when they hold (like borrowing to build is reasonable)
• Environments change and assumptions fail (like interest accrues)
• Unmanaged, assumptions "default" (like debt explodes)
• Illusions awareness is the assumption's financial statement — how much you owe, when it's due, what default costs

The problem in CPS engineering isn't having assumptions — all engineering relies on them. The problem is that assumptions aren't explicitly managed. Designers write assumptions into documents, deploy, and forget. When they fail at runtime, the system keeps running the original logic until something breaks.

The essence of illusions awareness is making assumptions first-class citizens — identified, classified, characterized, leveraged. Not eliminating assumptions, but making them manageable.

This applies beyond CPS. Any AI system operating in the open world — LLMs, robots, autonomous vehicles — needs some form of illusions awareness. Acknowledging that assumptions fail is safer than pretending they won't.

---

Paper link: https://arxiv.org/abs/2609.17260 Crazyflie project: https://github.com/geraldrocher/Crazyflie2.1