NeSy-RAG: Neuro-Symbolic RAG with Attributable Prolog for Explainable Question Answering

Key Points

• Core problem. Standard RAG pipelines conflate retrieval and reasoning inside the LLM, producing answers whose provenance cannot be audited (attribution) and that silently ignore missing user context (gap blindness).
• Architecture. NeSy-RAG is a three-stage neuro-symbolic pipeline.
1. *Attributable Prolog Synthesis* — for every retrieved passage, the LLM emits Prolog facts, each annotated with the source-chunk ID. 2. *Knowledge-Gap Detection* — the user query is compiled into a Prolog goal; unbound variables flag missing facts and trigger a clarification question. 3. *Execution-Trace Answering* — Prolog returns a stepwise proof in which every premise is labeled as a user-supplied fact or a cited document fragment.
• Headline result. On the ShARC rule-based QA benchmark, NeSy-RAG scores 61.1 % vs 42.8 % for a standard RAG baseline (+18.3 pp) without any task-specific training, demonstrating that the bottleneck of RAG is the traceability of reasoning, not retrieval recall.
• Comparison with Euclid-MCP. Both delegate logical inference to Prolog, but Euclid-MCP assumes hand-authored RBAC rules, while NeSy-RAG synthesizes rules automatically from retrieved documents and adds active knowledge-gap detection plus document-level attribution.
• Honest limits (acknowledged by the authors).
• *Rule fidelity* — the LLM may flatten probabilistic wording into deterministic Prolog clauses; future work should attach confidence to each rule.
• *Benchmark scope* — ShARC is synthetic with crisp rules; real medical/legal text is noisier.
• *Expressiveness* — Prolog covers boolean/rule reasoning, not numeric dose–response or probabilistic inference.
• *Cost* — synthesizing Prolog per chunk raises inference cost vs one-shot RAG; no detailed analysis is provided.
• Position in a broader pattern. NeSy-RAG is the latest data point in a "reasoning outsourcing" family: octopus DNA pre-training plus RNA inference-time computation, Euclid-MCP, and now NeSy-RAG all move hard inference out of the LLM and into a formal substrate, while keeping the LLM for NL ↔ formal translation.
• Implications for agent engineering. Knowledge-gap detection should be made explicit (Prolog variable binding as a computable missing-fact signal); execution traces are the substrate of auditability in high-stakes domains; the LLM-as-translator / Prolog-as-reasoner split is more robust than either LLM-only or Prolog-only designs.

Findings (bullet form)

• Attribution becomes a first-class artifact. Every Prolog clause carries a chunk ID, so a downstream auditor can replay the proof and verify that each premise came from a specific passage, not from LLM priors.
• Gap detection is computable, not heuristic. Unbound variables in the Prolog query are the formal signal that prompts a follow-up question, eliminating the "LLM guesses when uncertain" failure mode.
• No new training required. The +18.3 pp ShARC gain comes purely from architectural redesign, suggesting diminishing returns from larger models when the reasoning path is opaque.
• Workload split. LLM owns natural-language understanding and NL→Prolog compilation; Prolog owns factual lookup, rule firing, and trace emission. The split is the source of reliability.
• Open risks. Faithful translation of modal/probabilistic language, real-world document noise, lack of numeric/probabilistic reasoning, and higher per-chunk inference cost remain unsolved.
• Directional takeaway. As agents enter regulated domains (medicine, law, finance), auditable reasoning will shift from a feature to a requirement; LLM + Prolog + per-rule attribution is a credible blueprint.

Source Links

• Paper: https://arxiv.org/abs/2608.06292
• HTML: https://arxiv.org/html/2608.06292v1
• Related: https://zhichai.net/topic/178503701
• Code: not publicly released by the authors.