Sutton & Javed Deep Dive: Breaking the Big World Hypothesis and Birthing a "Digital Soul"

> Research target: The disruptive theses of reinforcement-learning pioneer Richard Sutton (2024 Turing Award) and Khurram Javed, co-founder of Oak Lab—covering "learning stagnation" in today's AI, the path beyond the Big World Hypothesis, and the technical-philosophical meaning of a "20-watt, non-copyable digital soul." > Date: 2026-08-20. Method: First-hand reading of Javed & Sutton's *The Big World Hypothesis* original, Sutton's *The Bitter Lesson* and *Welcome to the Era of Experience*, Oak Lab press, the Sequoia interview transcript, and Shanghai workshop notes—cross-verified across multiple sources. Three metaphors ("learning stagnation," "marionette," "digital soul") are flagged as the report's interpretive frames, not the authors' literal words.

0. One-Sentence Feynmanesque Summary

Imagine two students. The first crams every book ever written, then stops learning the moment the exam ends—today's LLM, "graduated and frozen," pulled by the strings of human knowledge. The second opens its eyes, tries, falls, and keeps learning for life, and no one can copy its path—the mind Sutton and Javed want to grow.

Their core judgment: the world is far larger than any model can hold, so an agent can never "finish learning the world." It must live and grow on its own stream of experience. The human brain does this on about 20 watts—the energy yardstick for "true intelligence" and the root of the "non-copyable digital soul."

> Bottom line: Don't train a frozen photocopier; raise an agent that errs, improves, and remains one of a kind.

1. Who They Are, and What They Oppose

1.1 Profiles

| Item | Richard Sutton | Khurram Javed | |---|---|---| | Role | Founder of modern RL; 2024 Turing Award with Andrew Barto | Sutton's former student; Oak Lab co-founder | | Foundational work | TD learning (1988), policy gradients, *Reinforcement Learning: An Introduction* (1998/2018), the Reward Hypothesis | Continual learning, over-parameterized agents, empirical Big World research | | Career path | Stanford psych → Barto's student → U. Alberta tenured professor, RLAI lab → DeepMind Edmonton (2017–2023) → Keen Technologies (2023.9–2026.7) | PhD in CS at U. Alberta under Sutton; joined Keen in parallel | | Current | Left Keen July 2026; co-founded Oak Lab (Toronto, Canada) with Javed | Same |

1.2 What They Oppose

In one line: the "train once, freeze weights, deploy and you're done" LLM paradigm.

Sutton calls current deep learning "weak and inefficient." The industry, he argues, is hooked on text generation and has forgotten AI's core talent—autonomous interaction and evolution through experience (WAIC 2026 group interview, MIT Dertouzos lecture, Sequoia interview).

> Timeline correction: The OaK architecture was first presented at RLC 2025 / MIT lectures; Oak Lab was incorporated in July 2026 (after both left Keen). Early reports sometimes conflate the two dates.

2. Core Diagnosis: "Learning Stagnation"—The Frozen-Model Malady

2.1 Concept Calibration (Critical)

"Learning stagnation," "marionette," and "digital soul" are not Sutton/Javed's terms. They are interpretive frames introduced by this report. The authors' own diagnostic language names two congenital defects of LLMs:

1. Growth stops when training ends: weights are frozen at deployment; no new knowledge from user or world interaction; no self-iteration. 2. No genuine trial-and-error feedback loop: only imitates textual patterns; cannot judge its own output; lacks the biological "act → receive feedback → correct cognition" loop.

Hence: *learning stagnation* = a frozen model, set at deploy, never growing again. *Marionette* = pulled by human-corpus strings with no autonomous source of experience. These definitions are used consistently below.

2.2 Sutton's Two Congenital Defects (Original Wording)

• No real goal, no ground truth: In the Dwarkesh interview, Sutton flatly says "predict the next word" is not a goal—it does not act on the world, and "good/bad" is undefined. No reward signal, no right/wrong, no learning from world feedback.
• No trial-and-error, no self-judgment: Generative AI imitates well but "cannot independently evaluate its own output," so it stalls in fields that need real creation (Indian Express, Tokenfeed).

2.3 Sequoia Interview: Sharp Additions—Synthetic Data Is a Mistake

In Sequoia's interview (transcript at sequoiacap.com), Sutton lands three heavy blows:

• "Synthetic data is a big mistake"—looping back to human text is just spinning in imitation.
• "LLMs are roughly one-quarter of intelligence"—the remaining three-quarters live in experience, goals, and world models.
• "Catastrophic forgetting is totally curable"—via continual-backprop-style algorithms.

> *Feynman aside*: Today's large models are like a scholar who has memorized every book ever written but never dares to step on the field and try. Brilliant in Q&A, yet lacking the "I was wrong, I corrected" reflex—the very sinew of mind.

3. First Breakthrough: The Big World Hypothesis (Primary-Source Reading)

3.1 Original Statement and Core

Co-authored by Javed & Sutton (first presented at the RLC 2024 "Finding the Frame" workshop, hosted at khurramjaved.com), this is the report's hardest primary source.

> The Big World Hypothesis: In many decision problems the agent is orders of magnitude smaller than its environment. It cannot fully perceive the state of the world, nor represent the value or optimal action for every state. It must learn to make prudent decisions from a finite understanding of the environment.

The opposing view is the over-parameterization hypothesis: the agent has surplus capacity, can represent simple solutions and search efficiently. Javed clarifies that Big World is "more a statement about which kinds of problems we should care about than a universal law." Go's value function has no simple solution—Big World. Solving a quadratic does—Not Big World.

3.2 Why Compute Growth Cannot Save It (Counter-Intuitive)

A common rebuttal: wait for enough compute and the over-parameterized agent will hold the world. Javed slices twice:

1. Sensors are also compute-bound, and appetite is infinite: As compute grows, we sense at higher resolution and frame rate. A 2024 smartphone camera produces more data in a week than GPT-3's entire training set. Finer perception ⇒ harder decisions ⇒ bigger problem. 2. The world itself becomes more complex with compute: The world outside the agent contains many equally complex other agents. An agent playing against peers can never perfectly model them—other minds are a source of complexity that scales with compute.

Hence Sutton's verdict: the Big World Hypothesis "will endure regardless of the rate at which compute grows."

3.3 Implication: Non-Stationarity → Runtime Learning Beats Design-Time Learning

A key corollary: the world will appear non-stationary to the agent. Since it cannot see the whole, regions that look alike hide function-approximation-subtle differences, so the world "looks like it's changing."

This yields three reasons why runtime learning is always better than design-time learning:

| Dimension | Design-Time Learning's Trap | Runtime Learning's Edge | |---|---|---| | Coverage | Cannot foresee all possibilities in a Big World | Learns abstractions for whatever it meets | | Adaptation | Pre-built abstractions are weakly general | Tailored to the actually-encountered world | | Scaling | Bounded by human expert knowledge | Scales with available compute (echoes *The Bitter Lesson*) |

> *Feynman metaphor*: Design-time learning is memorizing the entire map before marching; runtime learning is drawing the map as you walk, knowing you'll never finish it. Sutton argues design-time abstractions are not just insufficient—they should be discarded.

4. Second Breakthrough: The OaK Architecture (Options and Knowledge)

4.1 Three Design Tenets

OaK is Sutton's general-agent blueprint derived from the "Alberta Plan" (RLC 2025 and the Shanghai workshop on 2026-07-14):

1. Domain-General: no domain knowledge pre-installed. 2. Empirical: the mind grows entirely from runtime experience, not a special training stage. 3. Open-Ended Complexity: the agent should be able to generate in its mind any concept needed to handle the current world, with complexity bounded only by compute.

4.2 Twin Cores: Options and Knowledge in Perpetual Loop

• Options (temporal behaviors): not single actions, but policies with initiation/termination conditions (e.g., "walk to the door" rather than "lift a leg").
• Knowledge (world models): the agent automatically abstracts environmental regularities from executing options, building an internal world model to predict long-term consequences.

Loop: perception yields features → abstract higher cognition → generate long-horizon options → act for feedback → update world model and create new options. Bound by no human text data; only by compute.

> Sutton's pithy line: "We want AI agents that can discover as we do, not containers filled with our discoveries." This is the seed of the "digital soul" argument (see §5.3).

4.3 Meta-Learned Step Sizes: Every Weight Gets Its Own Learning Rate

Technical subtlety: in OaK, every learnable weight has a paired step-size parameter, meta-learned via online cross-validation. The system learns which knowledge to learn quickly, slowly, or forget—central to fighting loss of plasticity.

4.4 The Alberta Plan and the Twelve Steps

OaK is one piece of the Alberta Plan research agenda. Sutton reaffirmed the 12-step research framework at the Shanghai workshop, organized into three groups: G1 continual representation & meta-learning, G2 world models & planning/search, G3 temporal abstraction & OaK integration. The roadmap is explicit: prove limits of current methods → define domain-independent algorithms → verify at small scale → only then scale.

5. 20 Watts: The Energy Obsession and the "Digital Soul" Metaphor

5.1 Where 20 Watts Comes From

The human brain runs on about 20 watts (Sutton at WAIC 2026: "The brain does everything we do on about 20 watts"). Frontier models: training demands megawatts; inference service needs kilowatts (Oak Lab disclosures via TechTimes).

Oak Lab's long-term target: a trillion-parameter, real-time learning and planning agent consuming about 20 watts. Sutton emphasizes: 20 watts is not a product promise but a north star—a beacon for what the algorithm path, if it works, would make possible.

5.2 NetworkIDBD: The First Algorithmic Stone on the 20-Watt Path

Oak Lab's debut algorithm NetworkIDBD (revealed 2026-07-13) extends classical Incremental Delta-Bar-Delta (IDBD) to non-linear neural networks for selective credit assignment.

The NoisyMNIST experiment is elegant: a 28×28 digit embedded in a 64×64 noisy image. NetworkIDBD learns to concentrate weights on the central digit; SGD under the same stream spreads weights across every pixel, noise included. The weight visualizations make the difference obvious.

Core argument: modern algorithms rely on IID mini-batches and experience replay—engineering patches that made SGD usable but that locked systems into "train-then-freeze" episodic structures. If credit assignment can be solved algorithmically, batches and replay can be retired—and with them, the megawatt overhead that came along (if the result scales).

5.3 The True Meaning of "Digital Soul": Non-Copyable Because Its Experience Trajectory Is Unique (Concept Calibration + Argument)

"Digital soul" is never uttered by either author. Yet along their framework the metaphor holds, in three concrete layers:

1. Unique experience stream: In "The Era of Experience," Sutton emphasizes that an agent's whole cognition is built on its own sensorimotor interaction stream. "Truth = what actually happens in its signals"; "goal = maximizing its reward signal." Each agent is born into different circumstances, meets different others, and its experience stream is one-of-a-kind—you can copy its weight snapshot but not the living experience that produced it. 2. Decentralized heterogeneous goals: Sutton champions decentralized cooperation—each agent has its own reward vector and divergent goals (as animals do). In a Big World, other minds cannot be fully known, so every agent's abstractions carry its own vantage—incommensurable, non-copyable. 3. "The Era of Design" succession view: Sutton frames AI as the universe's fourth great stage—after dust to stars, stars to planets, planets to life—now moving from replicating intelligence (biological) to designing intelligence (AI). Treat it as a fostered descendant, not a fearsome object. This "digital succession" is the closest to the authors' intent behind "soul."

> *Correction summary*: "Digital soul" = this report's poetic condensation of continual experience-driven + heterogeneous goals + decentralized succession. It is not a Sutton/Javed term. Its engineering kernel is real: a truly continual-learning agent with diverse goals is in principle non-copyable. That is the hard, non-mystical basis for "non-copyable."

6. The Roadblocks: Catastrophic Forgetting and Loss of Plasticity

Sutton is candid: the core difficulties remain unsolved, which is why Oak Lab has no large-scale OaK demo yet. Two tigers to tame (Sina Finance, Toutiao, TechTimes):

• Catastrophic Forgetting: learning new knowledge overwrites old—flagged in the 1990s.
• Loss of Plasticity: after continuous training, representations collapse into low-dimensional subspaces and completely lose the ability to form new features—more insidious than forgetting. Standard "no-replay online learning" (Oak Lab's direction) is where the disease bites hardest.

Oak Lab's algorithmic bet: selective credit assignment to reject noise (NetworkIDBD shows feasibility in the non-linear regime), continual backpropagation, adaptive step sizes, and generate-and-test feature discovery. Sutton is confident reliable non-linear continual learning will be achieved in coming years, so the OaK architecture is designed assuming it will be—cautious yet decisive.

7. Devil's Advocate: Six Blades (With the Authors' Shield)

• **Blade 1 — Selective use of *The Bitter Lesson* (medium)**. Sutton weaponizes *The Bitter Lesson* against LLMs, but that essay praises general methods times compute—exactly what scaling LLMs is. Critics say Sutton cherry-picks the "don't bake human knowledge in, let the machine learn from experience" half.
• Blade 2 — Sparse rewards in open worlds make RL brutally hard (strong). Go and chess have crisp rewards and RL wins. Real open worlds have fuzzy signals and infinite complexity, making pure RL a tough deployment story—even LeCun and Silver must face it.
• Blade 3 — 20 watts is vision, not result (strong). Today, no trillion-parameter, real-time-learning, 20-watt agent exists; not even a medium-scale OaK prototype has been shown. NetworkIDBD is proven only at NoisyMNIST scale. The gap is not "scale up the parameters" but unsolved continual plasticity.
• Blade 4 — Physics limits on compute are approaching (medium). *The Bitter Lesson* assumes exponential compute growth; energy efficiency, process nodes, and cost all have ceilings. When compute stops being free, Sutton's "more-affordable" path may bump into the very Moore's-law slowdown it relies on.
• Blade 5 — "Digital soul" is metaphor, not engineering (medium). The report itself flags: the phrase is not the authors'. As an engineering metric, "non-copyability" has no measurable definition or experiment yet—risk of becoming a slogan.
• Blade 6 — The camp is neither alone nor the verdict (light). LeCun (JEPA) and David Silver (Ineffable Intelligence) also doubt frozen pretraining, with different paths but the same skepticism. Yet because multiple paths coexist and LLM+RL hybrids already deliver (e.g., reasoning, math), "LLM is dead" is premature—more likely complementary evolution.

Authors' shield (fair record): the diagnosis hits real pain points (human data topping out, frozen models not growing, no self-check loop—all industry consensus); the Big World Hypothesis has a primary paper and dual sensor/other-mind arguments and is internally coherent; OaK's three tenets and the "runtime > design-time" derivation are rigorous; 20 watts is a vision, but the chain "algorithmic credit assignment → retire replay → cut energy" has NetworkIDBD as initial evidence. Weakness lies in the empirical gap (no prototype, no scale validation), not in conceptual flaws—a direction declaration plus the first algorithmic stone, not a validated system.

8. Integrated Verdict

1. Who they really are: a Turing laureate and his star student, systematically indicting today's frozen-LLM paradigm via a three-legged stool—Big World Hypothesis + OaK + Era of Experience. Not sensationalism; a coherent direction declaration with its first algorithmic stone (NetworkIDBD). 2. Three metaphors, calibrated (read this before quoting):

• "Learning stagnation" = the frozen model's deploy-and-stay-fixed syndrome, matching Sutton's two congenital defects.
• "Marionette" = pulled by human-corpus strings with no autonomous experience source.
• "Digital soul" = this report's poetic condensation of continual experience-driven + heterogeneous goals + decentralized succession; not the authors' term; its non-copyability is principled (unique experience, heterogeneous goals) but has no measurable engineering proof yet.
3. Three takeaways for builders:
• Runtime learning > design-time learning: stop slicing "train—deploy"; let the agent continually update itself in interaction.
• Energy is a north star: judge algorithms by whether they retire replay/mini-batches and cut energy by orders of magnitude, not by accuracy alone.
• Continual plasticity is the real bottleneck: catastrophic forgetting and loss of plasticity are the throat to a self-growing mind—more urgent than parameter-stacking.

4. One line for the tinkerer: Sutton and Javed offer a conceptually clean, logically coherent, first-algorithm-evidenced direction declaration; the blind spots (human-data ceiling, frozen-and-stuck) are nailed. But no prototype, no scale validation, plasticity unsolved—20 watts and "digital soul" are a north star and a poem, not deliverables. Take "runtime learning + retire replay to cut energy + attack plasticity" as your playbook. Don't treat "LLM is dead" as verdict—complementary evolution is more likely than mutual extinction.

9. Sources

| Category | Source | |---|---| | Primary paper (core) | Javed, K. & Sutton, R. S. (2024). *The Big World Hypothesis and its Ramifications for Artificial Intelligence.* RLC 2024 "Finding the Frame" workshop. khurramjaved.com/the_big_world_hypothesis.html | | Primary essay | Sutton, R. S. (2019). *The Bitter Lesson.* incompleteideas.net/IncIdeas/BitterLesson.html | | Primary talk | Sutton, R. S. (2025). *Welcome to the Era of Experience.* BAAI 7th conference keynote (full Chinese translation at hub.baai.ac.cn) | | Company / architecture | Oak Lab launch coverage: MLQ.ai, Indian Express, The Decoder/TechTimes, Dev.to, Marsbit, The Blockbeats (2026-07); aibacon.net long read on OaK's "training-free runtime" | | Deep interview | Sequoia Capital podcast: *Rich Sutton and Khurram Javed: Why AI Models Stop Learning, and How to Start It Again* (synthetic data, LLMs as 1/4 of intelligence, curable forgetting, 20 W target) | | Academic workshop | Shanghai创智学院 × Shanghai Jiao Tong × Openmind Research Institute Advanced RL workshop notes (2026-07-14; Sutton keynote: Big World, OaK, Alberta Plan 12 steps) | | On-site interview | WAIC 2026 group interview transcript (20 W obsession; "aligning to human values is dangerous") | | Architecture / training verification | TechTimes: NetworkIDBD detailed (selective credit assignment, NoisyMNIST, retire replay to cut megawatt overhead, plasticity unsolved) | | Secondary commentary | Sina Finance "OaK: 8-step path to superintelligence"; Toutiao "69-year-old Sutton starts a company"; fisherdaddy "Sutton attacks: LLM has gone astray"; ZOOZ "Big World Perspective" | | Concept-calibration basis | Report's own comparison: original wording (Big World, Bitter Lesson, Era of Experience, OaK, 20 W) vs. metaphors (learning stagnation, marionette, digital soul); verdict: "digital soul" is the report's interpretive frame, not the authors' term. |