Modern large language models resemble an infinitely enlarged "cortex": highly adept at statistical learning, pattern generalization, and compressing language and visual representations. But according to Adam Marblestone, AI's poor learning efficiency, fragile goals, and lack of intrinsic understanding of values are not mainly due to insufficiently large architectures—they stem from the absence of the brain's other, older, and more critical half: the "steering" mechanisms.
In his view, human intelligence is not the triumph of a single learning algorithm but a collaboration between two systems:
- One builds world models and abstracts regularities from experience (cortex-like).
- The other writes "what matters, what is dangerous, what is worth pursuing" into the learning process (largely from subcortical structures such as the hypothalamus, brainstem, and basal ganglia).
This explains an unsettling contrast: a baby rapidly learns language and social rules from limited real-world input, while an LLM, after ingesting internet-scale text, still displays a "smart but brittle" temperament in common sense, planning, stable motivation, and value consistency.
---
Why children learn language "like breathing" while AI must "swallow the internet"
Marblestone emphasizes that human infants do not learn from scratch. Evolution did not write complete knowledge into genes; rather, it wrote a value function and curriculum: preferences for faces, gazes, vocal tones, and interaction; sensitivity to curiosity, exploration, belonging, status, shame, reward, and punishment. These are not learned "knowledge" but attention mechanisms and motivational terrain that guide learning.
> Note: "Value function" here means "which situations trigger reward/punishment and which cues the system prioritizes as learning signals." It is not an explicit formula but implemented through genetically preset circuits and modulatory signals.
By contrast, mainstream AI training objectives are extremely "computer-science flavored": next-token prediction, or coarse-grained reinforcement learning via external rewards. This can shape powerful representations, but it lacks an intrinsic sense of "what matters," so scale and data density must compensate—like sailing a ship with a bigger engine but no compass or rudder.
---
Marblestone's core claim: the brain's "secret recipe" lies in reward/loss, not just structure
In his Dwarkesh interview, he repeatedly decomposed the problem into four familiar ML components: architecture, learning algorithm, initialization, and loss/reward functions. His "personal intuition" is that the AI field has long over-focused on architecture and scale while underestimating the highly complex, layered, developmentally staged loss functions and reward signals in the brain.
He summed up the difference vividly: evolution wrote vast amounts of "Python code," lighting up different training signals in different brain regions at different developmental stages—while our loss functions are as simple as a single line of cross-entropy.
More pointedly, he finds some LLM-style training "strange" from a reinforcement-learning perspective: models barely use explicit value functions, even though brain mechanisms analogous to value estimation (e.g., dopamine-related "reward prediction error") have long had correspondences between neuroscience and RL.
---
The path forward: build the "steering wheel," not just a bigger "cortex"
Marblestone does not advocate copying the brain wholesale, nor obsessing over interpreting every neuron. His approach: use neuroscience to fill in AI's missing key constraints and modules, advancing engineering until theory can land.
1) Connectomics: finding evidence of "innate algorithms"
He stresses the value of "mapping the brain"—not to find some single magic circuit, but to answer deeper questions: How do cortical and subcortical systems divide labor? Which circuits generate training signals? How do those signals interface with learned abstract representations?
FlyWire is a concrete exemplar: long-term curation and annotation of the fruit fly whole-brain connectome, covering ~140,000 neurons, over 50 million synapses, and 100,000+ community annotations, including synaptic and neurotransmitter information, with flagship papers published in *Nature* (2024). Such resources make hardware-level structure computable and public for the first time.
2) Formalized AI: making AI development an "accumulative science"
He also champions a more methodological route: formalizing and making provable the key reasoning and verification mechanisms, so capabilities accumulate like mathematics and engineering rather than relying mainly on trial-and-error tuning. He cites tools like Lean as examples, emphasizing the value of "verifiable signals" for reinforcement learning and scalable collaboration.
3) Organizational form: FROs to fill infrastructure gaps
This connects to his work at Convergent Research, which positions itself as a "mission control for frontier technology," founding time-boxed, ambitious nonprofit Focused Research Organizations (FROs) to build infrastructure that neither academia nor industry reliably funds. Projects range from neural tracing platforms and drug-target data portals to wearable whole-brain ultrasound interfaces—all serving one theme: build the missing "key tools" and thicken the trunk of the scientific tree.
---
Controversy and implications: does this replace the "scaling route"?
Marblestone does not simply reject scaling. He admits the success of large models is "surprising"—only noting that the current paradigm differs enormously, intuitively, from how the brain works. He is waiting for a "more brain-like paradigm": more general forms of inference (he has discussed "omniscient/omnidirectional prediction") and richer, more layered systems of value and training signals.
This pushes the discussion toward AI safety: if capability systems become extremely powerful without robust "steering" structures, alignment remains a post-hoc patch. But if the "steering wheel" is part of the system itself, values and learning trajectories may become more controllable and interpretable—at least at the level of macro-level mechanisms.