Paper: Eradicating Negative Transfer in Multi-Physics Foundation Models via Shodh-MoE (arXiv 2605.15179)

Forum topic · 小凯 · 2026-05-15

Summary

This forum post shares an arXiv paper (2605.15179) by Ellwil Sharma and Arastu Sharma on overcoming negative transfer in Scientific Machine Learning (SciML) foundation models. The authors identify that co-training disparate PDE regimes—such as broadband open-channel fluid dynamics and boundary-dominated porous media flows—causes gradient conflict, unstable optimization, and plasticity loss in dense neural operators. They propose Shodh-MoE, a sparse-activated latent transformer architecture for multi-physics transport. The model operates on compressed 16^3 physical latents generated by a physics-informed autoencoder featuring a Helmholtz-style velocity parameterization, which constrains decoded states to be divergence-free. By routing different physics regimes to specialized experts, the sparse mixture-of-experts design avoids the incompatible spectral and geometric demands of a single dense parameter path. The post was auto-collected from zhichai.net on 2026-05-15, with the full abstract truncated in the original listing.

论文概要

研究领域: ML 作者: Ellwil Sharma, Arastu Sharma 发布时间: 2026-05-14 arXiv: 2605.15179

English Overview

This post shares an arXiv paper addressing negative transfer in multi-physics foundation models for Scientific Machine Learning (SciML).

Problem

Scaling SciML toward universal foundation models is bottlenecked by negative transfer: co-training disparate partial differential equation (PDE) regimes simultaneously can induce:

Gradient conflict
Unstable optimization
Plasticity loss in dense neural operators

In particular, broadband open-channel fluid dynamics and boundary-dominated porous media flows impose incompatible spectral and geometric demands on a single dense parameter path.

Proposed Solution: Shodh-MoE

The authors introduce Shodh-MoE, a sparse-activated latent transformer architecture for multi-physics transport:

Operates on compressed 16³ physical latents produced by a physics-informed autoencoder
Uses an intra-tokenizer Helmholtz-style velocity parameterization, restricting decoded states to divergence-free velocity fields (abstract truncated in source)

Original Abstract (truncated)

> Scaling Scientific Machine Learning (SciML) toward universal foundation models is bottlenecked by negative transfer: the simultaneous co-training of disparate partial differential equation (PDE) regimes can induce gradient conflict, unstable optimization, and plasticity loss in dense neural operators. In particular, broadband open-channel fluid dynamics and boundary-dominated porous media flows impose incompatible spectral and geometric demands on a single dense parameter path. We introduce Shodh-MoE, a sparse-activated latent transformer architecture for multi-physics transport. Shodh-MoE operates on compressed 16^3 physical latents produced by a physics-informed autoencoder with an intra-tokenizer Helmholtz-style velocity parameterization, restricting decoded states to divergence-free ve...

---

*Auto-collected on 2026-05-15. Read the full paper at arXiv:2605.15179.*