UNICON: A Frozen Model Approaches Expert-Level Performance on Three Unseen Disciplines via In-Context Numerical Learning

Paper: A foundation model of numerical intelligence with cross-disciplinary generalization arXiv: https://arxiv.org/abs/2607.28432 (HTML full text) Authors: Chenghan Wu, Zongmin Yu, Liu Yang (Department of Mathematics, National University of Singapore)

Key points

• UNICON is a frozen foundation model that performs near-expert forecasting on disciplines entirely absent from training — air quality, web activity, and space weather — using only in-context examples at inference time.
• All nine training disciplines (hydrology, traffic, power systems, weather, land, ocean, soil, solar, human mobility) are encoded in one unified graph format: nodes carry numeric features, edges encode spatial or logical connections, and in-context examples are history→future key-value pairs.
• Training is cheap and one-shot: 2× NVIDIA H200 for 28 hours, after which weights are frozen; every evaluation uses the same weights.
• The model is never told which discipline it is looking at — it must learn a generic ability to extract predictive relations from numeric examples, not recognize domain-specific patterns.

Three levels of evaluation

The paper's evaluation design separates three levels of generalization:

1. Unseen periods of training sources — same datasets, later timestamps. 2. Unseen datasets within training disciplines — e.g., trained on CAMELS-US (US basins), tested on CAMELS-CL (Chilean basins). 3. Entirely unseen disciplines — air quality, web activity, space weather: zero exposure during training.

UNICON beats kNN baselines at all three levels and approaches specially trained expert models on many systems. Most strikingly, on web activity — a dataset and discipline never seen in training — its inference performance reaches expert level.

Do the context examples actually matter? Four controls

To prove gains come from in-context learning rather than something else, the authors ran four ablations:

• No examples (query only)
• Correctly paired but randomly sampled examples
• Mismatched examples (history and future incorrectly paired)
• Numerically corrupted examples (correct pairing, noised values)

Results ordered: correct random > mismatched > corrupted > none. Mismatched and corrupted examples performed worse than no examples at all — the model genuinely extracts relations from examples, and feeding it wrong relations actively misleads it.

Breadth beats depth in the training corpus

Under a fixed training budget, comparing two corpora:

• 10-D (deep): ten sources from only five training disciplines
• 10-B (broad): ten sources spanning more discipline types, one dataset each

The broad-corpus model generalizes better to completely unseen disciplines while staying roughly equal on training disciplines — echoing the LLM-era finding that diversity matters more than scale, now extended from language to numerics.

LLM agent + numerical model

UNICON can be paired with an LLM agent: the LLM interprets task descriptions, orchestrates multi-step reasoning, and integrates results; UNICON extracts predictive relations from numeric context. This combination surpassed the SOTA specialist model on web activity — a discipline UNICON never saw in training. The authors call this the collaboration of linguistic and numerical intelligence: the agent organizes, the numerical model predicts.

Limitations and open questions

• Only forecasting was evaluated; imputation, anomaly detection, simulation, and control are untested.
• The corpus-diversity experiment compares specific combinations only; no complete scaling law is established.
• There is no systematic method yet for selecting and organizing context examples for a new system.

Why it matters

Engineering: new systems no longer require new models. Represent the system as a graph, provide a few history-future pairs, and reason directly — a shift from "one model per project" to "one model for all projects."

Scientific: in-context learning is shown not to be a language-specific property but a deeper statistical regularity — any sufficiently structured data can support learning from examples. This broadens the very notion of "foundation model" beyond language to numerical, graph-structured, and time-series worlds.

The paper closes with a nod to Wittgenstein's *Tractatus* 6.522: "There are, indeed, things that cannot be put into words. They make themselves manifest." Traffic flow, river runoff, and geomagnetic disturbances "make themselves manifest" through numeric observations — and UNICON shows these relations can be learned from context even before anyone names them.

Key takeaway: one frozen model, three disciplines never seen in training, near or above expert-level performance from inference-time context alone.