Plan*RAG: Efficient Test-Time Planning for Retrieval Augmented Generation

Authors: Prakhar Verma, Sukruta Prakash Midigeshi, Gaurav Sinha, Arno Solin, Nagarajan Natarajan, Amit Sharma Published: October 28, 2024 Source: https://arxiv.org/abs/2410.20753

Overview

Plan*RAG is a novel framework that enables structured multi-hop reasoning in retrieval-augmented generation (RAG) through test-time reasoning plan generation.

Problem with Existing Approaches

While existing approaches such as ReAct maintain reasoning chains within the language model's context window, the authors observe that this often leads to plan fragmentation and execution failures.

Key Insight

By isolating the reasoning plan as a directed acyclic graph (DAG) outside the LM's working memory, Plan*RAG enables:

1. Systematic exploration of reasoning paths; 2. Atomic subqueries enabling precise retrievals and grounding; 3. Efficiency through parallel execution and bounded context window utilization.

Moreover, Plan*RAG's modular design allows it to be integrated with existing RAG methods, providing a practical solution to improve current RAG systems.

Results

On standard multi-hop reasoning benchmarks, Plan*RAG consistently achieves improvements over recently proposed methods such as RQ-RAG and Self-RAG, while maintaining comparable computational costs.

Original Abstract

> We introduce Plan*RAG, a novel framework that enables structured multi-hop reasoning in retrieval-augmented generation (RAG) through test-time reasoning plan generation. While existing approaches such as ReAct maintain reasoning chains within the language model's context window, we observe that this often leads to plan fragmentation and execution failures. Our key insight is that by isolating the reasoning plan as a directed acyclic graph (DAG) outside the LM's working memory, we can enable (1) systematic exploration of reasoning paths, (2) atomic subqueries enabling precise retrievals and grounding, and (3) efficiency through parallel execution and bounded context window utilization. Moreover, Plan*RAG's modular design allows it to be integrated with existing RAG methods, thus providing a practical solution to improve current RAG systems. On standard multi-hop reasoning benchmarks, Plan*RAG consistently achieves improvements over recently proposed methods such as RQ-RAG and Self-RAG, while maintaining comparable computational costs.

Related Work

• A Systematic Framework for Enterprise Knowledge Retrieval
• Retrieval Augmented Generation and Understanding in Vision: A Survey
• Synergizing RAG and Reasoning: A Systematic Review
• AceSearcher: Bootstrapping Reasoning and Search for LLMs via Reinforcement Learning
• Agentic Information Retrieval

Takeaways for Practitioners

• Researchers: Reproduce the core comparisons and check whether statistical significance and computational cost are reported.
• Engineers: Consider Plan*RAG's pluggable planner module and evaluate integration cost against existing RAG pipelines.
• Product teams: Focus on user-perceivable benefits such as answer groundedness, latency, and multi-turn consistency rather than offline metrics alone.

> Note: The detailed experimental design notes and checklists in the original forum post are editorial commentary; quantitative results should be verified against the original PDF.