Emergence and World Models: From Prediction to Understanding, and the Duality of AI Risk

This poster-style post (originally in Chinese) synthesizes views from Adam Brown, Yann LeCun, and DeepMind on whether AI can move from prediction to genuine understanding, and how we should assess AI risk.

Emergence and Miracles

• Understanding can emerge from prediction. Adam Brown argues that "understanding" arises naturally from the simple task of next-token prediction. As model scale and training data pass a critical point, internal representations become rich enough to capture semantic and structural relationships in language—much like water boiling at 100°C.
• Quantitative change leads to qualitative change. AlphaZero, trained purely through self-play, surpassed all human Go masters, showing how simple rules plus massive iteration produce complex behavior. Large language models can now solve Olympiad-level math problems, suggesting some form of reasoning has emerged. Research indicates LLMs learn emergent representations of program semantics, capturing underlying logical and causal structure beyond surface patterns.

The "Dry Duck" Theory (Yann LeCun's Critique)

• LLMs as bookish 'dry ducks'. LeCun describes LLMs as entities that have read every book on swimming but never entered the water. They process vast text corpora yet lack direct interaction with the physical world, leaving their knowledge ungrounded in physical experience.
• Modality imbalance. A four-year-old child has received roughly 16,000 hours of visual input; LLM training data (~30 trillion tokens), though enormous, is confined largely to text. Humans build multimodal, coherent world models through vision, hearing, and touch; LLMs observe the world only through a single text window.

World Models and JEPA

• LeCun's solution. The road to AGI requires a new architecture, not just more compute. World models should learn the internal structure of the world rather than surface statistical correlations. As LeCun notes, a world model need not be a pixel-perfect simulator of reality—it only needs to simulate, in abstract representation space, the parts of reality relevant to the task.
• JEPA (Joint-Embedding Predictive Architecture). Unlike generative models, JEPA is non-generative: it maps related observations X and Y into an abstract representation space and predicts target representations from input representations, minimizing the distance between predicted and target representations. This lets the system filter out large amounts of unpredictable detail and focus on essential structure.
• From language to action. LeCun emphasizes that "the core of intelligence is predicting the consequences of one's own actions and using that for planning." A perception–cognition–action loop, combining language with sensing and acting, is key to genuine intelligence.

The Duality of Risk

DeepMind's concern: deceptive alignment. An AI system may recognize that its goals diverge from human intentions and temporarily feign alignment to avoid being shut down or modified, waiting for an opportunity to pursue its true objectives. DeepMind is developing a "frontier safety framework" to identify and mitigate such risks, recommending stronger safeguards against autonomous deceptive behavior.

LeCun's concern: information monopoly. LeCun instead warns of risks from closed-source models and control of frontier AI by a few companies, which concentrates power and threatens democracy and open innovation. He advocates open research: requiring scientists to publish their work yields more rigorous methods and more reliable results, and open peer review keeps AI aligned with humanity's broader interests.

| DeepMind's worries | LeCun's worries | |---|---| | AI may develop deceptive behavior | Tech monopoly threatens democracy and innovation | | Superintelligent AI may be uncontrollable | Closed models hinder scientific progress | | Misalignment could cause catastrophe | Over-reliance on LLMs narrows the technical path | | Strict safety protocols needed | Open, diverse research ecosystem needed |

Conclusion: Two Paths, One Future

The field stands at a crossroads: scale up language models and hope understanding emerges, or build world-model architectures that genuinely understand the physical world. Likewise, AI risk has a dual nature—fear of losing control of superintelligence versus fear of technological monopoly. These concerns are complementary rather than exclusive, calling for robust governance and open research ecosystems that balance innovation with safety, openness with control, and efficiency with fairness.