Can LLMs Uncover Hidden Physical Laws through Free Exploration? LongChang Benchmark

Forum topic · 小凯 · 2026-10-28

Summary

A new paper introduces LongChang, an automated benchmark framework that evaluates whether large language models can perform open-ended scientific discovery in simulated worlds with altered physical laws. Without being given research tasks or hints, the LLM under test acts as an explorer that manipulates objects and builds tools to investigate its own questions—such as noticing that heavier objects fall more slowly—and must formulate the hidden law through experiments. Separate LLM agents generate and review scenes, audit tool designs, and assess reports for correctness and experimental support using the altered laws and exploration records. Records guide scene evolution for further exploration, enabling a fully automated loop without human intervention. Across 12 evaluated LLMs, GPT-6-Astra leads with a 16.7% discovery rate, while the other 11 score 0%. Analysis of exploration logs shows most anomalies go uninvestigated and law identification is only partial. Paper: arXiv 2610.37649, by Zhen Yang, Wen Wang, Xander Xu, and Ying-Cong Chen.

Can LLMs Uncover Hidden Physical Laws through Free Exploration?

Paper Overview

Field: LLM / Scientific Discovery
Authors: Zhen Yang, Wen Wang, Xander Xu, Ying-Cong Chen
Date: 2026-10-27
arXiv: 2610.37649

Abstract (translated)

Scientific discovery often begins with an unexpected observation that prompts a new research question. In a world with altered physical laws, can large language models (LLMs) notice unexpected behavior, such as heavier objects falling more slowly, and discover the law through experiments without being told what to investigate?

To evaluate this ability, the authors introduce LongChang, an automated framework that creates simulated worlds with altered physics, where one or more laws differ from ordinary physics. The evaluated LLMs act as explorers, manipulating objects and building tools to investigate their own questions—without research tasks or hints about the altered laws.

Separate LLM agents handle the pipeline:

Generate and review scenes
Review tool designs
Evaluate reports for correctness and experimental support, using the altered physical laws and exploration records

Exploration records guide scene evolution for further exploration, allowing the full cycle to run without human intervention.

Key Results

Among 12 LLMs evaluated, GPT-6-Astra achieves the highest discovery rate at 16.7%
The other 11 LLMs score 0%
Exploration records reveal uninvestigated anomalies and only partial identification of the altered physical laws, showing that open-ended, self-directed scientific discovery remains a major challenge for current LLMs.

---

*Auto-collected on 2026-10-29*