Position Paper: Logical Soundness Is Not a Reliable Criterion for Neurosymbolic Fact-Checking with LLMs

Forum topic · 小凯 · 2026-04-07

Summary

This position paper by Jason Chan, Robert Gaizauskas, and Zhixue Zhao argues that using formal logical soundness as the core verification criterion in LLM-based neurosymbolic fact-checking systems is structurally inadequate. While such systems typically translate natural language into logical formulae via LLMs and check whether claims can be validly derived from verified true premises, the authors contend this approach cannot reliably detect misleading claims. The core problem is a systematic divergence between conclusions that are logically sound and the inferences humans typically make and accept. Drawing on cognitive science and pragmatics research, the paper presents a typology of cases where logically sound conclusions still elicit human inferences unsupported by the underlying premises, meaning readers can be misled even by formally valid outputs. As an alternative, the authors advocate a complementary approach: treating LLMs' human-like reasoning tendencies as a feature rather than a bug, and using LLMs to validate the outputs of formal components in neurosymbolic pipelines against potentially misleading conclusions.

Paper Overview

Field: NLP
Authors: Jason Chan, Robert Gaizauskas, Zhixue Zhao
Type: Position paper

Original Abstract

As large language models (LLMs) are increasing integrated into fact-checking pipelines, formal logic is often proposed as a rigorous means by which to mitigate bias, errors and hallucinations in these models' outputs. For example, some neurosymbolic systems verify claims by using LLMs to translate natural language into logical formulae and then checking whether the proposed claims are logically sound, i.e. whether they can be validly derived from premises that are verified to be true. We argue that such approaches structurally fail to detect misleading claims due to systematic divergences between conclusions that are logically sound and inferences that humans typically make and accept. Drawing on studies in cognitive science and pragmatics, we present a typology of cases in which logically sound conclusions systematically elicit human inferences that are unsupported by the underlying premises. Consequently, we advocate for a complementary approach: leveraging the human-like reasoning tendencies of LLMs as a feature rather than a bug, and using these models to validate the outputs of formal components in neurosymbolic systems against potentially misleading conclusions.

Key Points

Problem: LLM-based fact-checking pipelines increasingly rely on formal logic as a rigorous safeguard against bias, errors, and hallucinations—typically by translating natural language into logical formulae and checking whether claims are logically derivable from verified true premises.
Core argument: These neurosymbolic approaches structurally fail to detect misleading claims because logical soundness does not guarantee alignment with the inferences humans normally make and accept.
Evidence: Drawing on cognitive science and pragmatics, the authors construct a typology of cases where logically sound conclusions systematically trigger human inferences that the underlying premises do not actually support.
Proposal: Use LLMs' human-like reasoning tendencies as a feature rather than a bug—employ LLMs to validate the outputs of formal components in neurosymbolic fact-checking systems, guarding against potentially misleading conclusions.

---

*Auto-collected on 2026-04-07.*