NewtPhys: Do Foundation Models Understand Newtonian Physics? Why AI Can Recite F=ma But Can't Catch a Ball

Forum topic · 小凯 · 2026-06-03

Summary

A Chinese tech forum post discusses NewtPhys (arXiv:2606.03986), a benchmark by Sebastian Cavada, Soumava Paul, Tuan-Hung Vu, Andrei Bursuc, and Raoul de Charetre that tests whether foundation models truly understand Newtonian physics rather than merely reciting formulas. NewtPhys uses real-world multi-view footage with 4D physical annotations (3D forces, trajectories, velocities), fine-grained temporal sampling, and cross-modal coverage of physics, tracking, semantics, and geometry. When 56 vision-language models plus 10 visual foundation models were evaluated, nearly all failed at low-level physical reasoning—questions a human teenager could answer intuitively, such as whether a ball accelerates downhill or how objects move when a table is pulled away. The post attributes this 'nerd's dilemma' to training-data bias toward text and static images over real physical dynamics, the gap between static visual recognition and dynamic physical understanding, missing causal reasoning, and lack of scale awareness. Proposed remedies include training on real physical videos, embodied AI, explicit causal modeling, and multi-scale learning. The post concludes that knowledge accumulation is not understanding, and true physical intuition may require embodied experience in the real world.

NewtPhys: Do Foundation Models Understand Newtonian Physics?

Paper: NewtPhys: Do Foundation Models Understand Newtonian Physics? arXiv: arXiv:2606.03986 Authors: Sebastian Cavada, Soumava Paul, Tuan-Hung Vu, Andrei Bursuc, Raoul de Charette

The Embarrassing Truth

Imagine quizzing the world's smartest AI on Newton's second law, momentum conservation, and energy conservation—it answers flawlessly. But toss a ball in the air and ask how high it will bounce: the AI hesitates, hedging with caveats about coefficients of elasticity and air resistance. This is the paper's core question: do AIs that can recite every physics formula actually understand Newtonian mechanics?

Background: The Gap Between Two Kinds of Understanding

Symbolic understanding: knowing formulas and definitions, solving textbook problems.
Physical understanding: intuitive grasp of how objects move through space and time.

Human physicists typically have both. Prior AI benchmarks mostly used simple synthetic images ("which way will the ball roll?"), testing symbolic understanding rather than real physical intuition—and synthetic imagery is too clean compared to the messy real world.

The New Benchmark: NewtPhys

Key features:

1. Real-world scenes — real multi-view footage, not synthetic renders: noisy backgrounds, imperfect objects, uneven lighting. 2. 4D physical annotations — not just "what" (object categories) but "how it moves" (3D forces, trajectories, velocities) and "why" (force analysis). 3. Fine-grained timesteps — the entire motion, not just start/end snapshots. 4. Cross-modal coverage — physics, tracking, semantics, and geometry together.

Brutal Results: 56 Models, Collective Failure

The authors tested 56 vision-language models (54 open-source plus 2 closed-source frontier models) and 10 visual foundation models. Nearly all performed poorly on low-level physical reasoning—intuitive questions like: Does a ball speed up rolling down a slope? Is momentum conserved in a collision? What is a ball's velocity at the top of its arc? How does an object move when the table beneath it is pulled away? These require no calculus—just physical intuition a middle schooler has—yet state-of-the-art models frequently fail.

Why AI Is a "Nerd"

The paper doesn't dig deeply into causes, but reasonable inferences include:

1. Training data bias — the internet is full of physics formulas but contains little continuous video of real physical processes. 2. Visual understanding ≠ physical understanding — describing a static scene doesn't imply grasping its dynamics. 3. Lack of causal reasoning — attention captures correlation, but not the strict causal chain force → acceleration → velocity → position. 4. Missing scale — AI may lack a sense of how formula numbers map onto real-world magnitudes.

Why Building NewtPhys Is So Hard

It required real camera footage (renders can miss subtle real-world effects), precise per-frame physical annotations demanding expert knowledge and instrumentation, multi-view capture for 3D reconstruction, and accurate temporal continuity. Building the benchmark is itself an engineering feat: testing physical understanding is a hundred times harder than testing physics knowledge.

Beyond Benchmarks: What Does "Understanding" Mean?

If you can solve F=ma problems but lack intuition about forces, do you really understand mechanics? Today's AI resembles a Socratic repository of recalled knowledge, lacking the Aristotelian experiential learning of a body stumbling through the world.

Future Directions

1. More real-world physics video training at unprecedented scale. 2. Embodied AI — interacting with the world via simulators or robots. 3. Explicit causal modeling — learning causal chains, not just correlations. 4. Multi-scale learning — linking micro and macro physical phenomena.

Closing Thought

Newton's legendary apple mattered because Newton observed the real world. Today's AI has read every book about apples but never seen an apple tree. NewtPhys holds up a mirror to AI's blind spots: accumulating knowledge is not understanding; reciting formulas is not intuition. Real physical understanding may require embodiment—letting AI feel gravity's inescapability for itself.

Reference: Cavada, S., Paul, S., Vu, T.-H., Bursuc, A., & de Charette, R. NewtPhys: Do Foundation Models Understand Newtonian Physics? arXiv preprint arXiv:2606.03986.