A Systematic Framework for Enterprise Knowledge Retrieval: Leveraging LLM-Generated Metadata to Enhance RAG Systems

Source: https://arxiv.org/abs/2512.05411 Authors: Pranav Pushkar Mishra, Kranti Prakash Yeole, Ramyashree Keshavamurthy, Mokshit Bharat Surana, Fatemeh Sarayloo Published: 2025-12-05 Category: Agentic Search

Overview

In enterprise settings, efficiently retrieving relevant information from large and complex knowledge bases is essential for operational productivity and informed decision-making. This research presents a systematic empirical framework for metadata enrichment using large language models (LLMs) to enhance document retrieval in Retrieval-Augmented Generation (RAG) systems.

Key Contributions

• A structured pipeline that dynamically generates meaningful metadata for document segments, substantially improving their semantic representations and retrieval accuracy.
• A controlled 3 x 3 experimental matrix comparing three chunking strategies — semantic, recursive, and naive — against three embedding techniques — content-only, TF-IDF weighted, and prefix-fusion.
• Ablation analysis isolating the contribution of each component.
• Cross-encoder reranking for silver-standard ground truth generation, with statistical significance confirmed via Bonferroni-corrected paired t-tests.

Key Results

• Metadata-enriched approaches consistently outperform content-only baselines.
• Recursive chunking paired with TF-IDF weighted embeddings yields 82.5% precision.
• Naive chunking with prefix-fusion achieves the strongest ranking quality (NDCG 0.813).
• Metadata enrichment improves vector space organization while maintaining sub-30 ms P95 latency.

Context in the Field

Neural information retrieval has evolved from BM25 through BERT cross-encoders, dense bi-encoders, late interaction (e.g., ColBERT), to generative retrieval and LLM-based agentic search. Each generation balances efficiency, effectiveness, and maintainability. RAG and agentic search extend external knowledge access from one-shot retrieval into an iterative, verifiable, plannable process — shifting evaluation from static nDCG toward task success rate, citation accuracy, and multi-hop reasoning chain completeness.

Implications for Practitioners

1. Architecture: Cascaded retrieval + reranking + generation remains mainstream, but agentic paradigms increasingly treat retrieval strategy itself as a learnable object. 2. Data: High-quality instruction data and session logs are critical; synthetic data requires safeguards against knowledge leakage and distribution shift. 3. Evaluation: The gap between offline metrics and online satisfaction is widening; LLM-as-judge should be cross-validated with human evaluation. 4. Deployment: Latency, cost, explainability, and safety are hard constraints in production — the paper's sub-30 ms P95 result shows metadata enrichment can be practical at enterprise scale.

Original Abstract (verbatim)

> In enterprise settings, efficiently retrieving relevant information from large and complex knowledge bases is essential for operational productivity and informed decision-making. This research presents a systematic empirical framework for metadata enrichment using large language models (LLMs) to enhance document retrieval in Retrieval-Augmented Generation (RAG) systems. Our approach employs a structured pipeline that dynamically generates meaningful metadata for document segments, substantially improving their semantic representations and retrieval accuracy. Through a controlled 3 X 3 experimental matrix, we compare three chunking strategies -- semantic, recursive, and naive -- and evaluate their interactions with three embedding techniques -- content-only, TF-IDF weighted, and prefix-fusion -- isolating the contribution of each component through ablation analysis. The results demonstrate that metadata-enriched approaches consistently outperform content-only baselines, with recursive chunking paired with TF-IDF weighted embeddings yielding 82.5% precision and naive chunking with prefix-fusion achieving the strongest ranking quality (NDCG 0.813). Our evaluation employs cross-encoder reranking for silver-standard ground truth generation, with statistical significance confirmed via Bonferroni-corrected paired t-tests. These findings confirm that metadata enrichment improves vector space organization and retrieval effectiveness while maintaining sub-30 ms P95 latency, providing a quantitative decision framework for deploying high-performance, scalable RAG systems in enterprise settings.

References

• Original paper: arXiv:2512.05411