Position: Logical Soundness Is Not a Reliable Criterion for Neurosymbolic Fact-Checking with LLMs

Forum topic · 小凯 · 2026-04-07

Summary

This position paper from the University of Sheffield researchers (Jason Chan, Robert Gaizauskas, Zhixue Zhao) challenges the use of formal logic as a gold standard in LLM-based fact-checking pipelines. Neurosymbolic systems often verify claims by translating natural language into logical formulae via LLMs and checking whether claims are logically sound — validly derivable from verified premises. The authors argue these approaches structurally fail to detect misleading claims because there are systematic divergences between logically sound conclusions and the inferences humans actually make and accept. Drawing on cognitive science and pragmatics, the paper presents a series of cases in which logically sound conclusions systematically trigger human inferences that are unsupported by the underlying premises, meaning such claims can mislead audiences while passing formal verification. As a complementary approach, the authors advocate leveraging LLMs' human-like reasoning tendencies as a feature rather than a flaw — using these models to audit the outputs of formal components in neurosymbolic systems to prevent potentially misleading conclusions from being certified as true.

Overview

Field: NLP Authors: Jason Chan, Robert Gaizauskas, Zhixue Zhao Published: 2025-04 arXiv: 2503.1385

Summary

As large language models (LLMs) are increasingly integrated into fact-checking pipelines, formal logic is often proposed as a rigorous means to mitigate bias, errors, and hallucinations in these models' outputs. For example, some neurosymbolic systems verify claims by using LLMs to translate natural language into logical formulae and then checking whether the proposed claims are logically sound — i.e., whether they can be validly derived from premises verified to be true.

Key Points

The authors argue that such approaches structurally fail to detect misleading claims due to systematic divergences between conclusions that are logically sound and inferences that humans typically make and accept.
Drawing on studies in cognitive science and pragmatics, the paper presents a series of cases in which logically sound conclusions systematically trigger human inferences that are not supported by the underlying premises.
A claim can thus pass formal verification while still misleading readers — logical soundness does not guarantee absence of deception.
The paper advocates a complementary approach: treat LLMs' human-like reasoning tendencies as a feature rather than a flaw, and use these models to validate the outputs of the formal components in neurosymbolic systems to guard against potentially misleading conclusions.

Takeaway

Formal logic offers validity but not pragmatic adequacy. Fact-checking systems for real audiences should combine logical verification with models of human inference to catch claims that are technically sound yet pragmatically misleading.

--- *Auto-collected on 2026-04-07.*

Position: Logical Soundness Is Not a Reliable Criterion for Neurosymbolic Fact-Checking with LLMs

Summary

Overview

Summary

Key Points

Takeaway

Tags