Soft Ontology Priors over Latent Concept Posteriors: Fixing Reasoning Shortcuts in Neuro-Symbolic Learning

Forum topic · 小凯 · 2026-10-24

Summary

A new arXiv paper (2610.30391) by Tengwei Song and Robert Hoehndorf addresses reasoning shortcuts in neuro-symbolic (NeSy) predictors trained with only task-label supervision. Such models can learn distinct concept mappings that achieve identical maximum likelihood, making recovered concepts unidentifiable and unsafe for downstream reuse. The authors inject a soft ontology prior over latent concepts into the training objective and compile it, together with the task constraint, into a smooth, decomposable probabilistic circuit. Under independent Bernoulli perception, normalisers of the structured concept posterior reduce to weighted model counts linear in compiled circuit size, since neural leaf weights fold into the circuit before evaluation. The soft compilation places expert axioms in the circuit's weights rather than its support: task-structural axioms remain hard constraints, while noisy expert axioms only redistribute prior mass and cannot empty the support. The paper characterises how the prior reweights shortcut mass via ontology violation energy differences, proves bounded log-density perturbations under bounded axiom noise, and shows that a hard-support counterpart can become degenerate from a single corrupted axiom. Validation uses an audited six-concept view of the BDD-OIA driving benchmark plus controlled NeSy tasks.

Paper Overview

Field: Machine Learning Authors: Tengwei Song, Robert Hoehndorf Published: 2026-10-24 arXiv: 2610.30391

Abstract

Neuro-symbolic (NeSy) predictors that supervise only the task label suffer from reasoning shortcuts: distinct concept mappings achieve the same maximum likelihood, leaving the recovered concepts unidentifiable and unsafe for downstream reuse. This work attacks that failure by injecting a soft ontology prior over the latent concepts into the training objective and compiling it together with the task constraint into a smooth, decomposable probabilistic circuit.

Key Contributions

Efficient posterior normalisation: Under independent Bernoulli perception, the normalisers of the resulting structured concept posterior reduce to weighted model counts that are linear in the compiled circuit size, because the neural leaf weights fold into the circuit before evaluation.
Soft compilation strategy: Expert axioms are placed in the circuit's *weights* rather than its *support*. Task-structural axioms guaranteed by the data-generating process stay hard, so noisy expert axioms only redistribute prior mass and cannot empty the support.
Theoretical guarantees: The paper characterises how the prior reweights label-restricted shortcut mass through differences in ontology violation energy, shows that soft priors incur bounded log-density perturbations under bounded axiom noise, and proves that the hard-support counterpart can be made degenerate by a single corrupted axiom.

Evaluation

The predictions are validated on an audited six-concept view of the real-world BDD-OIA driving benchmark and on controlled NeSy tasks.

---

Paper link: https://arxiv.org/abs/2610.30391

*Auto-collected 2026-10-24*