JEPA-Anything: Learning Predictive Models Across Different Worlds

Forum topic · 小凯 · 2026-09-19

Summary

JEPA-Anything is a domain-agnostic world modeling framework introduced in arXiv paper 2609.20800, built on orthogonal predictive factorization (OPF), an extension of joint-embedding predictive architectures (JEPA). OPF decomposes latent prediction targets into complementary factors, learns each through dedicated pathways, and recombines them within a shared predictive design. The framework is evaluated across seven domains—vision, biology, clinical trajectories, control, molecular dynamics, physical fields, and weather—covering representation learning, intervention prediction, out-of-distribution generalization, and long-horizon dynamics. Reported results include improved metrics on all 10 matched dynamics tasks versus JEPA baselines, a 34.8% reduction in single-intervention prediction error on Interventional Pong, the lowest single-step and 100-step molecular errors across four systems, and latent orbit patterns that recover the Kepler scaling exponent with fitted slope -1.4991. A biology intervention nominated by factors also received experimental support in cell co-culture, patient-derived organoids, tumor slices, and mice. Code is available at https://github.com/Gen-Verse/JEPA-Anything.

Overview

Research area: NLP

Authors: Taoyong Cui, Zhongyao Wang, Xinyue Xu, Weiyang Liu, Zhaochen Yu, Yuying Zhang, Qiang Gao, Mengyue Yang, Wanli Ouyang, Pheng Ann Heng, Yingcheng Wu, Zhenfei Yin, Ling Yang

Published: 2026-09-17

arXiv: 2609.20800

Abstract

World modeling enables intelligence to anticipate consequences, guide interventions, and learn from interaction. Yet predictive models remain domain-specific: can a common learning principle support world modeling across radically different systems? The authors introduce JEPA-Anything, a domain-agnostic framework based on orthogonal predictive factorization (OPF). Extending joint-embedding predictive architectures (JEPA), OPF decomposes latent targets into complementary factors, learns them through dedicated pathways, and recombines them within a shared predictive design.

Evaluation

JEPA-Anything is evaluated across seven domains: vision, biology, clinical trajectories, control, molecular dynamics, physical fields, and weather. Experiments span representation learning, intervention prediction, out-of-distribution generalization, and long-horizon dynamics, including:

10 matched dynamics tasks
Predictions over 1,000+ clinical events
100-step molecular rollouts across four systems

Key Results

Improves reported metrics on all 10 dynamics tasks compared with matched JEPA baselines
Reduces single-intervention prediction error by 34.8% on Interventional Pong
Achieves the lowest single-step and 100-step molecular errors among compared methods across all four systems
Latent orbit patterns recover the Kepler scaling index with a fitted slope of -1.4991
Beyond prediction, a factor-nominated biology intervention gained experimental support in cell co-culture, patient-derived organoids, tumor slices, and mice

These results support a common decomposed-prediction principle across heterogeneous worlds, linking world modeling to intervention and experimentally grounded scientific discovery.

Code

https://github.com/Gen-Verse/JEPA-Anything

--- *Auto-collected on 2026-09-19*