Exactness at Inference: A Representational Criterion for Out-of-Distribution Generalization

Forum topic · 小凯 · 2026-09-23

Summary

This paper proposes a representational criterion for out-of-distribution generalization: a model generalizes beyond its training distribution only when it computes a representation structurally equivalent to the generating mechanism, rather than an approximation fitted to it. The authors argue that extrapolation is governed by exactness at inference, regardless of implementation. They illustrate the criterion with Tensor Logic, where zero-temperature contraction is equivalent to discrete logic, with Boolean tensors, orthonormal embeddings, and continuous arithmetic only; lacking infinite recursion it reaches Datalog rather than Prolog and requires external memory to bind novel entities. The criterion constrains inference rather than training, requiring neither discrete representations nor extracted expressions: exact [0,1] marginals pass, while neural networks thresholded to hard labels and Logic Tensor Networks fail, whereas differentiable ILP and zero-temperature Tensor Logic pass. A propagation rule for hybrid architectures follows: outputs inherit the bounds of every fitted estimator along their path. On law-derived partitions, only the exact hypothesis class certifies conclusions not determined by training data, answering 56.3% of remote queries where ensembles produce spurious confidence. Paper: arXiv 2609.24942.

Exactness at Inference: A Representational Criterion for Out-of-Distribution Generalization

Field: Machine Learning Authors: Filipe Marinho Rocha, Inês Dutra, Vítor Santos Costa, Luís Paulo Reis Published: 2026-09-21 arXiv: 2609.24942

Abstract (translated summary)

A model generalizes outside its training distribution only when it computes a representation structurally equivalent to the generating mechanism, not an approximation fitted to it. Such equivalence is necessary for exactness in and out of distribution, and extrapolation is governed by this exactness at inference, whatever its realization.

Tensor Logic demonstrates this: a zero-temperature contraction is equivalent to discrete logic, deducing in place with no artefact extracted, its tensors Boolean, its embeddings orthonormal, only its arithmetic continuous. Lacking infinite recursion it reaches Datalog, not Prolog; though exact over closed domains, it needs external memory to bind a novel entity.

The criterion needs neither a discrete representation nor an extracted expression, and it constrains inference rather than training: exact [0,1] marginals pass, while a neural network thresholded to hard labels does not. Logic Tensor Networks fail the criterion; differentiable ILP and zero-temperature Tensor Logic pass.

Piecewise-linear extrapolation divergence and inability to bind novel entities are two manifestations of insufficient exact representability. For hybrid architectures, a propagation rule follows: an output inherits the bounds of every fitted estimator along its path — explaining which axes fail in equivariant models and the induction/transduction split observed on ARC-AGI.

Only an exact hypothesis class can certify conclusions not determined by the training data: on partitions derived from laws, it answers 56.3% of remote queries, while ensemble methods respond with spurious confidence and distance metrics rank them inversely.

The authors conclude that common inductive biases (from symmetry to memory) achieve exactness only because humans inject them — arguing that exact representations should be induced, rather than fitting surrogate models whose residuals diverge even at the arithmetic floor of training and compound under composition.

Exactness at Inference: A Representational Criterion for Out-of-Distribution Generalization

Summary

Exactness at Inference: A Representational Criterion for Out-of-Distribution Generalization

Abstract (translated summary)

Links

Tags