Sutton's "Betrayal": The Father of Reinforcement Learning Dismantles His Own "Bitter Lesson"

> Richard Sutton—2024 Turing Award co-winner and father of reinforcement learning—co-authored a philosophical position paper in May 2026 that systematically critiques mainstream 'passive representation' AI from symbolic systems to LLMs, proposing that AI should move toward 'Enactive Cognition.' Ironically, the paper itself runs into two of Sutton's own iron rules: The Bitter Lesson (less human knowledge, more compute) and the Reward Hypothesis (all goals = reward maximization). With a reported $1.1B bet from Sequoia/NVIDIA/Google on this route, the question is: is Sutton revising his legacy or overturning it?

Key points

• In May 2026, arXiv published "Toward Enactive Artificial Intelligence" (arXiv:2605.24238) by Banafsheh Rafiee and Richard S. Sutton—not a technical paper but a philosophical manifesto.
• The paper critiques nearly all mainstream paradigms—symbolic AI, deep learning, and LLMs—for treating cognition as internal processing detached from embodied interaction and intrinsic normativity.
• It proposes four pillars from cognitive science and phenomenology: Experience, Action-Perception Inseparability, Autonomy, Embodiment.

1. Representation vs. Enaction

| Dimension | Representationism (mainstream AI) | Enactive Cognition (new claim) | |---|---|---| | Perception | Passive receipt of inputs, internal processing | Active, skillful world-engagement | | Perception-action | Linear: perceive → process → act | Mutually constitutive, inseparable | | World model | Build internal representations | "The world is its own best model" (Brooks) | | Evaluation | Representation fidelity | Skillful interaction with environment |

Core distinction: representationism requires an agent to "look in a mirror"—copy the world internally and reason over the copy. Enactivism requires the agent to "dance"—be shaped by the world through action while shaping it back.

The four pillars

• Experience: Data ≠ experience. Supervised data is a byproduct of experience, not experience itself. LLMs learn patterns of human-generated data but never "lived through" any interaction. Deepens Sutton & Silver's 2025 "Era of Experience" framing.
• Action-perception inseparability: Perception is action—eye movements, head motion, and manual exploration structure sensory input. The paper's sharpest LLM critique: a generative video model can predict green-yellow-red traffic light sequences, but that is pattern-tracking, not the ability to cope when the pattern breaks.
• Autonomy: Introduces autopoiesis (self-producing, self-maintaining systems). A harsh rating table gives supervised learning, LLMs, and symbolic planning no self-evaluation; RL is closer but its reward function is still externally specified. Verdict: "full enactive autonomy—normativity emerging from the agent's own organization—remains unrealized."
• Embodiment: Affordances (Gibson) exist only relative to an agent's bodily capacities. LLMs, per the paper, reduce perception to pattern recognition on static datasets with no sensorimotor participation.

2. RL: closest to enactive cognition, but not enough

The paper describes "structural resonance" between RL and enactivism—experience generation, action-centrality, feedback-driven adaptation—but lists critical missing elements: external evaluation standards, perception still prior to action, embodiment as mere implementation detail, incomplete autonomy (no autopoiesis).

> "This resonance should not be viewed as theoretical equivalence... RL only approximates some enactive insights, but key elements remain missing or weakly developed."

RL is *closer* to enactive cognition than other paradigms—but it is not enactive cognition.

3. Sutton steps on his own two iron rules

• Bitter Lesson (2019) said the long-term value of human knowledge is being surpassed by general computation. Yet the 2026 paper imports Merleau-Ponty's "intentional arc," Husserl's phenomenology, Gibson's ecological psychology, and Maturana's autopoiesis—deep human knowledge, not compute-and-search. Is this not exactly the human-knowledge engineering the Bitter Lesson warned against?
• Reward Hypothesis (2018): "all goals can be conceived as maximization of expected cumulative reward." But the paper argues normativity should arise from the agent's own organization—implying externally specified reward functions are fundamentally flawed. The paper never explicitly resolves this contradiction; it says RL "resonates but is insufficient" without declaring the Reward Hypothesis wrong.

Possible reconciliations: treat enactive principles as an *extension* of RL rather than replacement; treat them as organizational principles rather than hand-crafted features; or hope computation spontaneously gives rise to autopoiesis through sustained interaction. But the paper does not argue for the third path—it only asserts resonance and lists gaps.

4. The $1.1B bet

According to industry reports cited in the post, Sequoia, NVIDIA, and Google jointly invested $1.1B at a $5B valuation in a zero-product company betting on the enactive AI route. The likely bets: embodied AI (robots learning in the physical world), world models, continual learning, and intrinsic motivation—mapping directly onto the paper's four pillars.

Risks: no product validation, unclear engineering path (how do you turn "intentional arc" and "autopoiesis" into a trainable loss function?), and unclear product-market fit while LLM commercialization accelerates.

5. Four critical questions

1. "Resonance" ≠ realizability: Can the missing elements be achieved within existing RL frameworks, or do they require a replacement architecture Sutton, as RL's father, cannot publicly concede? 2. Phenomenology vs. compute scale: If enactive AI needs human philosophical knowledge as design principles, it violates the Bitter Lesson. 3. Do LLMs really "only track patterns"? Recent LLMs already show some capacity to cope with pattern breaks in tool use, debugging, and reasoning. Did Sutton underestimate emergent capabilities? 4. Investment bubble risk: $1.1B into a zero-product company is unprecedented—even early OpenAI had GPT-2/3 as validation. If enactive AI's product-market fit takes 5–10 years, this could mark an AI bubble.

Historical caution: the first two AI winters (1970s symbolic AI, 1980s–90s expert systems) both stemmed from human-knowledge-engineering routes hitting ceilings. Sutton's likely reply: this time human knowledge enters as architecture-level inductive bias (sensorimotor coupling, embodiment), not hand-coded rules—like deep learning's architectural priors rather than hand-crafted features.

6. Conclusion: revision, not overturning—but near paradigm shift

| Legacy | Original stance | Revised stance | Magnitude | |---|---|---|---| | Bitter Lesson | Less human knowledge, more compute | Human knowledge as architectural bias, but ultimately compute-validated | Moderate | | Reward Hypothesis | All goals = reward maximization | Rewards insufficient; intrinsic normativity and autopoiesis needed | Major | | Era of Experience | AI should learn from its own experience | Deepened: skillful, normative, embodied interaction | Moderate |

The paper's real value is not algorithms (it offers none) but the questions it raises: does the passive-representation route have a fundamental ceiling? When the father of RL starts critiquing LLMs, citing phenomenology, and implicitly questioning his own Reward Hypothesis, that itself is a signal:

> "The victory of scale is real, but it is not everything. True intelligence requires experience, action, embodiment, and autonomy—things that do not emerge from more parameters and data alone, but demand new architectures, new learning paradigms, and new ways of interacting with the world."

At 70, questioning his own 40-year legacy may be Sutton's most valuable contribution yet.

---

> "The world is its own best model." — Rodney Brooks > > "But to use the world as its model, you need a body in it." — Enactive Cognition > > "And to have a body in it, you need to be born into it." — The question Sutton leaves open.

References

Source paper: Rafiee, B., & Sutton, R. S. (2026). *Toward Enactive Artificial Intelligence*. University of Alberta, Amii. arXiv:2605.24238.

Sutton's legacy:

• Sutton, R. S. (2019). *The Bitter Lesson*. http://www.incompleteideas.net/IncIdeas/BitterLesson.html
• Sutton, R. S., & Barto, A. G. (2018). *Reinforcement Learning: An Introduction* (2nd ed.). MIT Press.
• Silver, D., & Sutton, R. S. (2025). *Welcome to the Era of Experience*.

Enactive cognition classics:

• Varela, F. J., Thompson, E., & Rosch, E. (1991). *The Embodied Mind*. MIT Press.
• O'Regan, J. K., & Noë, A. (2001). A sensorimotor account of vision and visual consciousness. *Behavioral and Brain Sciences*, 24(5), 939-973.
• Noë, A. (2004). *Action in Perception*. MIT Press.
• Brooks, R. A. (1991). Intelligence without representation. *Artificial Intelligence*, 47(1-1), 139-159.