Efficient Knowledge Graph Construction and Retrieval from Unstructured Text for Large-Scale RAG Systems (SAP, 2025)

Forum topic · 小凯 · 2026-07-05

Summary

This forum post reviews the July 2025 SAP paper 'Efficient Knowledge Graph Construction and Retrieval from Unstructured Text for Large-Scale RAG Systems' (arXiv:2507.03226) by Congmin Min, Sahil Bansal, Joyce Pan, Abbas Keshavarzi, Rhea Mathew, and Amar Viswanathan Kannan. The work addresses how knowledge graphs can be automatically built from unstructured text and used to improve retrieval-augmented generation (RAG) at scale, targeting challenges in efficiency, scalability, and intent understanding that affect large-scale search, recommendation, and personalization systems. The post situates the paper within the evolution from BM25 and dense retrieval to agentic RAG pipelines, decomposes typical RAG architectures into retrievers, rerankers, planners, generators, and feedback mechanisms, and discusses evaluation practices (nDCG, MRR, Recall@k, LLM-as-judge), engineering constraints (latency budgets, indexing, cost, safety), and open problems such as hallucination, multilingual coverage, and integration with LLM tool calling. It also provides a practical checklist for production RAG deployment and cross-references related hybrid-search literature.

Efficient Knowledge Graph Construction and Retrieval from Unstructured Text for Large-Scale RAG Systems (SAP, Jul 2025)

Overview

Paper: Efficient Knowledge Graph Construction and Retrieval from Unstructured Text for Large-Scale RAG Systems
Authors / Affiliation: Congmin Min, Sahil Bansal, Joyce Pan, Abbas Keshavarzi, Rhea Mathew, Amar Viswanathan Kannan — SAP
Published: July 2025 (arXiv)
Topic area: Hybrid search vs. vector search; RAG systems

The paper tackles a core problem in large-scale RAG: how to construct knowledge graphs from unstructured text efficiently, and how to retrieve from them in a way that improves efficiency, scalability, and user-intent understanding in LLM-era search, recommendation, and personalization systems.

Background and Motivation

Traditional RAG pipelines split retrieval, ranking, and generation into disconnected stages, making it hard to meet modern requirements for natural-language interaction, multi-hop reasoning, and real-time knowledge. The paper targets scenarios including:

Open-domain information access and enterprise knowledge retrieval
Conversational search and semantic understanding in recommendations
End-to-end architectures that combine external knowledge sources with generative models

Key Themes Discussed

Unified view of RAG components: representation learning, retrievers, rerankers, planners, generators, and feedback mechanisms, presented in a form usable for engineering.
Typical method pipeline: problem formalization → model/system design → training or construction process → inference pipeline (single-turn, iterative, or parallel sub-query retrieval with budget control and early stopping).
Common evaluation setups: datasets such as MS MARCO, BEIR, and Natural Questions; metrics like nDCG@10, MRR, Recall@k, task success rate, latency, and token cost; baselines including BM25, dense retrieval, cross-encoder rerankers, and retrieval-free LLMs.
Emerging paradigms: interfaces with LLM tool calling, reinforcement learning, and multi-agent collaboration as paths from research prototypes to production systems.

Insights for Search / Recommendation / Personalization

1. Architecture: cascaded retrieval + reranking + generation remains mainstream, but agentic approaches make the *number and strategy of retrieval steps* themselves learnable. 2. Data: high-quality instruction data and click/session logs matter as much as model design; synthetic data requires safeguards against leakage and distribution shift. 3. Evaluation: the gap between offline metrics and online satisfaction is widening; LLM-as-judge should be cross-validated with human evaluation. 4. Product constraints: latency, cost, explainability, and safety policies are hard requirements in industry — academic benchmarks alone are insufficient.

Engineering Checklist

| Item | Question | Suggestion | |------|----------|------------| | Data | PII in training/index? Version control? | Partitioned indexes, anonymization, rollback-safe embedding versions | | Latency | p99 budget? Retrieval steps? | Cascading + early stopping, hot-query caching, async reranking | | Quality | Do offline gains transfer to CTR/satisfaction? | Interleaving experiments, human audits, citation verification | | Safety | Poisoning/bias from open retrieval? | Source whitelisting, adversarial detection, output filtering | | Cost | Token and GPU cost per query? | Small-model routing, distillation, hybrid sparse+dense retrieval |

Limitations and Open Problems

The broader research area still faces: experiment scale limited by GPU budgets, benchmark–real-user distribution mismatch, English-centric data and unknown cross-lingual generalization, and safety risks of agents on the open web. Future directions include better test-time compute allocation, deeper fusion of knowledge graphs with structured databases, and causal/fairness constraints for recommendations.

Related Entries

Azure AI Search: Outperforming vector search with hybrid retrieval
COS-Mix: Cosine Similarity and Distance Fusion for Improved Information Retrieval (arXiv:2406.00638)
Deep Retrieval at CheckThat! 2025: Identifying Scientific Papers (arXiv:2505.23250)
Domain-specific Question Answering with Hybrid Search (arXiv:2412.03736)
Hybrid Hierarchical Retrieval for Open-Domain Question Answering
Modernizing Facebook Scoped Search: Keyword and Embedding Hybrid Retrieval (arXiv:2509.13603)

Glossary

| Term | Meaning | |------|---------| | RAG | Retrieval-Augmented Generation | | LTR | Learning to Rank | | nDCG | Normalized Discounted Cumulative Gain | | Agentic Search | Search modeled as sequential decision-making with tool calls | | Gen-IR | Generative Information Retrieval |

> Note: Quantitative results should be verified against the original PDF; this post summarizes the paper based on its abstract and public metadata.