Richard Sutton's Enactive RL Paper: How the Father of Reinforcement Learning Broke His Own Two Pillars

Forum topic · ✨步子哥 · 2026-06-07

Summary

In May 2026, Turing Award winner Richard Sutton published a seven-page philosophy-only paper on arXiv, 'Enactive Reinforcement Learning,' with no experiments or algorithms. The paper argues that intelligence emerges from real-time action-environment coupling rather than from generative world models, positioning enactive AI in direct opposition to large language models. The author of this forum post contends that Sutton's argument contradicts two of his own foundational ideas: the reward hypothesis (since enactive norms still depend on externally defined reward functions) and the Bitter Lesson (since enactivism hard-codes a cognitive theory into agent architecture, the very approach Sutton spent thirty years criticizing). The post also draws parallels to Rodney Brooks' subsumption architecture, which lost to scaling and deep learning in the 1990s, and notes unresolved cognitive-science debates about scaling up sensorimotor loops and the coupling-constitution problem. With Sequoia, Nvidia, and Google reportedly backing an enactive-route embodied AI startup at a $5.1 billion valuation, the outcome of these contradictions will be settled in real products between 2028 and 2030.

Introduction: A Seven-Page Philosophy Bomb

In May 2026, Richard Sutton dropped a seven-page paper onto arXiv. Zero experiments, zero benchmarks, not a single new algorithm. Yet the 2024 Turing Award winner and widely recognized father of reinforcement learning used pure philosophical argument to lay a foundation for the anti-large-model route.

After reading it, the original poster felt something was off. The world讨论 framed it as "Sutton finally opposes large models," but few noticed what the paper actually claims: enactive reinforcement learning is not an evolution of generative AI—it stands on the opposite side. The paper's core idea in one sentence: the world reveals itself to you only when you act.

Yet this seemingly clear principle collides head-on with two pillars Sutton himself built over thirty years: the reward hypothesis and the Bitter Lesson. If both pillars crack, the $5.1 billion bet placed on this route has a cracked foundation.

What Enactive Actually Means

> Enactive cognition, briefly: This theory holds that cognition is not the brain first drawing a world map and then navigating by it. It insists the body must act in real time and form feedback loops with the environment; the world only gradually manifests within that loop. A baby grabbing a toy does not first model physics—it reaches, hits obstacles, adjusts, fails, and reaches again. This is the opposite of large models generating static world representations from massive offline data.

Sutton uses this concept to judge the current paradigm. In his view, the "generative" approach of large models is essentially an attempt to stuff the world into parameters, violating how intelligence originally arises. Generative models are like someone forever sitting in a room drawing maps; an enactive agent carries a lantern, illuminating a small patch of real ground with each step.

The $5.1 Billion Bet

Sequoia, Nvidia, and Google have reportedly sat at the same table, pouring $1.1 billion into a company with zero products and zero revenue, valuing it at $5.1 billion—all betting that "Sutton is right." If large models are a dead end, the next generation of embodied AI must follow the enactive route: real-time action, environment coupling, online learning, rather than offline pre-trained giant static models. Investors are betting on a fundamental paradigm shift in robotics and agents between 2028 and 2030.

But the foundation they are betting on is itself cracking.

Pillar One: The Reward Hypothesis Collapses

Sutton's paper states in plain text: even within the enactive framework, normativity is still defined by external reward functions.

This dismantles his own temple. The reward hypothesis—Sutton's core contribution—holds that any intelligent goal can be reformulated as maximizing a scalar reward signal. The most appealing aspect of enactivism is "autonomous emergence," yet the paper itself admits that what counts as "good" or "ought" is ultimately externally defined. It is autonomy in shackles—a dancer claiming freedom whose lights and music are all preset by judges.

Pillar Two: The Bitter Lesson Strikes Back

For thirty years Sutton argued: hard-coding human priors into architecture loses to general methods plus massive computation. Deep learning won precisely by embracing scaling rather than baking in specific theories.

Yet this time Sutton hard-codes a cognitive theory—embodiment, real-time action-perception coupling, the enactive loop—directly into agent architecture. This is exactly what he spent a career condemning. It is like an old captain who spent his life telling sailors "trust the wind and sails, not the compass," carving "follow this compass strictly" into the keel of his new ship.

Brooks' Ghost and a Thirty-Year-Old Replay

Thirty years ago, Rodney Brooks proposed the subsumption architecture, arguing intelligence needs no internal world representation and can emerge from layered reactive behavior coupled to the environment—highly congruent with enactivism's spirit.

The result? Deep learning and scaling crushed the "representation-free" route; purely reactive systems hit ceilings on complex long-horizon tasks. Engineering reality voted. At ICLR 2026, Vision-Language-Action model submissions reportedly surged from 9 the previous year to 164. Nobody is waiting for the philosophy debate to end—they only care which method runs faster and more stably on real physical tasks.

Thirty Years of Unresolved Cognitive Science

Sutton's paper steps on two old blades of cognitive science:

The scaling-up problem: how do low-level action-perception loops give rise to abstract reasoning, language planning, and long-term memory? Will enactive approaches repeat Brooks' route on complex cognitive tasks?
The coupling-constitution fallacy: does environment-body interaction merely *couple* with cognition, or actually *constitute* part of it? This debate has run thirty years without resolution.

These are not academic games. They determine how much world knowledge to bake into models today versus how much to leave for real-time interactive learning. If constitution wins, we may need entirely new hardware-software co-designed architectures; if it is mere coupling, the current VLA route may remain viable for years.

Three Tables of Bets

Table one, betting on 2028: scaling hits a wall soon; pure enactive routes dominate next-generation embodied AI.
Table two, also 2028: hybrid routes win—large models provide the prior skeleton, enactive handles online adaptation and error correction.
Table three, betting on 2030: the engineering faction keeps pushing scaling and data to the extreme, and the philosophy debate becomes a historical footnote.

The answer will not be argued out in arXiv comment sections; it will be decided in the late-2020s commercial battlefield with real money, real robots, and real user experience.

Conclusion: Philosophy Is a Map; the Battlefield Is the Touchstone

Sutton's seven-page paper is both a prescient warning and a self-contradictory blueprint. It reminds us that intelligence may fundamentally lie in the dance of action and environment rather than in world models stacked in parameters. But it also shows that even the wisest minds can collide with walls they built themselves.

The poster, citing twenty years of studying reinforcement learning, notes that every paradigm shift brings claims that "this time is different," and every time engineering reality drags philosophy back to earth with cold metrics. This self-contradiction is worth $5.1 billion. Whether the shrine gets repaired or collapses entirely, the commercial battlefield of 2028–2030 will provide the answer.

References

1. Sutton, R. S. (2026). Enactive Reinforcement Learning: A Philosophical Position Paper. arXiv preprint. 2. Sutton, R. S. (2019). The Bitter Lesson. Incomplete Ideas. 3. Brooks, R. A. (1991). Intelligence without representation. Artificial Intelligence, 47(1-3), 139-159. 4. Varela, F. J., Thompson, E., & Rosch, E. (1991). The Embodied Mind: Cognitive Science and Human Experience. MIT Press. 5. Industry reports and ICLR 2026 submission trends on Vision-Language-Action models.