H-RAG: Hierarchical Parent-Child Retrieval for Multi-Turn RAG Conversations

> Paper: H-RAG at SemEval-2026 Task 8: Hierarchical Parent-Child Retrieval for Multi-Turn RAG Conversations > Authors: Passant Elchafei, Hossam Emam, Mohamed Alansary, Monorama Swain, Markus Schedl > arXiv: 2605.00631 | 2026-04-30

1. The RAG System That Forgets What It Just Said

Imagine chatting with an AI assistant about a complex topic:

• User: "Introduce quantum computing"
• AI: "Quantum computing uses qubits..."
• User: "What advantages does it have over classical computing?"
• AI: "Quantum computing, on specific problems..." (retrieves new documents, forgets prior context)
• User: "What are its specific applications?"
• AI: "Quantum computing can be applied to cryptography, drug discovery..." (retrieves yet again, disconnected from the previous turns)

The problem: independent retrieval at every turn leads to incoherent answers and lost context.

2. Conversation Amnesia in Multi-Turn RAG

Problems of traditional RAG in multi-turn conversations:

1. Independent retrieval

• Each turn retrieves based only on the current question
• Conversation history is ignored
• Answers may contradict earlier content

2. Granularity mismatch

• Retrieved documents vary in granularity
• Some too broad, some too fine
• Cannot adapt to the conversational need

3. Unfaithful evidence

• Generated content claims to be based on retrieval
• But actually introduces external knowledge
• Dangerous in scenarios requiring strict evidence grounding

In MTRAGEval (multi-turn RAG evaluation), Task C requires:

• Accurate answers
• Faithful grounding in retrieved evidence
• Multi-turn consistency

3. H-RAG: Hierarchical Parent-Child Retrieval

The paper proposes H-RAG (Hierarchical Parent-Child Retrieval):

Core idea: > Separate fine-grained child-level retrieval from coarse-grained parent-level context reconstruction.

Technical approach:

1. Hierarchical document structure

• Parent: coarse-grained document/section
• Provides overall context
• Ensures multi-turn consistency
• Child: fine-grained paragraph/sentence
• Provides precise evidence
• Used for specific answers

2. Hierarchical retrieval

• Child retrieval: find the most relevant fine-grained content for the current question
• Parent context reconstruction: map children to their parent documents for broader context
• Generation uses both: children supply precise evidence, parents supply coherent context

3. Multi-turn consistency

• Parent documents stay stable across the conversation
• Different turns cite the same knowledge source
• Avoids "self-contradiction"

Like academic writing: you cite specific sentences (children) to support claims, but those sentences all come from the same paper (parent) — keeping your argument coherent.

4. Why Hierarchy Beats Flat Retrieval

Problems with flat retrieval:

• Fragmented context: each turn retrieves different documents with no shared knowledge base; answers read like collages, not coherent narratives
• Single granularity: either too broad (whole document) or too fine (single sentence), with no way to adapt

H-RAG's advantages:

• Dual-granularity coverage: children answer specific questions precisely; parents keep the overall picture consistent
• Knowledge anchoring: parent documents act as anchors; all turns revolve around a common knowledge base, avoiding drift
• Traceable evidence: children provide exact evidence; parents provide its context, satisfying faithful-grounding requirements

5. Understanding Requires Levels

When Feynman explained physics, he always worked at multiple levels:

> "If you can't explain something from simple to complex, you don't understand it. Good explanations need levels — big picture first, then details."

In multi-turn conversation:

> "Good RAG also needs levels — keep the conversation's overall consistency (parent) first, then supply specific evidence (child). Retrieval without hierarchy is like a textbook without chapters: the information is all there, but you can't navigate it."

H-RAG's philosophy: conversation is hierarchical, so retrieval should be too.

6. Takeaways

If you're building a conversational RAG system, ask yourself:

1. "Does my multi-turn dialogue stay consistent?" 2. "Can retrieval granularity adapt as needed?" 3. "Is there a 'parent' knowledge base anchoring the conversation?" 4. "Is generated content faithful to retrieved evidence?"

H-RAG reminds us: multi-turn RAG is not "independent retrieval per turn" but "incremental exploration over a shared knowledge base."

When an AI can manage knowledge at multiple levels — seeing both the trees (children) and the forest (parent) — it can truly conduct coherent, consistent, trustworthy multi-turn conversations.

In the world of RAG, hierarchy is not added complexity; it is deepened understanding.

*Originally posted on zhichai.net (智柴 AI Lab).*