CHIQ: Contextual History Enhancement for Improving Query Rewriting in Conversational Search

Overview

This post summarizes the paper CHIQ: Contextual History Enhancement for Improving Query Rewriting in Conversational Search (arXiv, 2024-06-07) by Fengran Mo, Abbas Ghaddar, Kelong Mao, Mehdi Rezagholizadeh, Boxing Chen, Qun Liu, and colleagues.

• Paper: https://arxiv.org/abs/2406.05013
• Code & data: https://github.com/fengranMark/CHIQ
• Category: Multi-Turn Conversational Search

Core Idea

From the original abstract:

> In this paper, we study how open-source large language models (LLMs) can be effectively deployed for improving query rewriting in conversational search, especially for ambiguous queries. We introduce CHIQ, a two-step method that leverages the capabilities of LLMs to resolve ambiguities in the conversation history before query rewriting. This approach contrasts with prior studies that predominantly use closed-source LLMs to directly generate search queries from conversation history. We demonstrate on five well-established benchmarks that CHIQ leads to state-of-the-art results across most settings, showing highly competitive performances with systems leveraging closed-source LLMs. Our study provides a first step towards leveraging open-source LLMs in conversational search, as a competitive alternative to the prevailing reliance on commercial LLMs. Data, models, and source code will be publicly available upon acceptance at https://github.com/fengranMark/CHIQ.

Key Points

• Problem: Conversational queries are often ambiguous; prior query rewriting approaches rely heavily on closed-source commercial LLMs to generate search queries directly from conversation history.
• Method: CHIQ is a two-step approach — first use an LLM to disambiguate the conversation history, then rewrite the query based on the clarified context.
• Results: State-of-the-art performance on most settings across five established benchmarks, competitive with closed-source LLM-based systems.
• Openness: Data, models, and source code are publicly released.

Experiments (as described)

• Evaluated on five well-established conversational search benchmarks.
• Baselines include systems leveraging closed-source LLMs for direct query generation.
• Note: precise numeric results should be verified against the original PDF tables; this post summarizes experimental design based on the abstract and public metadata.

Insights for Search / RAG Practitioners

1. Architecture: Cascade retrieval + reranking + generation remains mainstream, but agentic paradigms increasingly treat retrieval frequency and strategy as learnable decisions. 2. Data: High-quality instruction data and click/session logs matter; synthetic data requires guarding against knowledge leakage and distribution shift. 3. Evaluation: The gap between offline metrics and online user satisfaction is widening; LLM-as-judge should be cross-validated with human evaluation. 4. Deployment: Latency, cost, interpretability, and safety are hard constraints in production — not just academic benchmarks.

Limitations & Future Work

Potential limitations include experiment scale constrained by GPU budgets, benchmark-vs-real-user distribution mismatch, English-centric data limiting cross-lingual generalization, and safety risks of agentic systems on the open web. Future directions include more efficient test-time compute allocation, deeper integration with knowledge graphs / structured databases, and causal/fairness constraints for recommendation systems.

Related Work

• A Survey on Multi-Turn Interaction Capabilities of Large Language Models
• Evaluating LLM-based Agents for Multi-Turn Conversations: A Survey
• Aligning Query Representation with Rewritten Query and Relevance Judgments
• An Empirical Analysis on Multi-turn Conversational Recommender Systems
• Beyond Whole Dialogue Modeling: Contextual Disentanglement for Conversational Understanding
• Few-Shot Conversational Dense Retrieval (SIGIR 2021)

Glossary

| Term | Meaning | |------|---------| | IR | Information Retrieval | | RAG | Retrieval-Augmented Generation | | nDCG | Normalized Discounted Cumulative Gain, a ranking quality metric | | Agentic Search | Modeling search as sequential decision-making and tool use | | Gen-IR | Generative Information Retrieval |

References

• CHIQ: Contextual History Enhancement for Improving Query Rewriting in Conversational Search. arXiv:2406.05013