Conversational vs Traditional Search: Comparing Behavior and Outcomes in Legal Case Retrieval (SIGIR 2021 Short Paper)
This post indexes a SIGIR 2021 short paper comparing conversational search and traditional (keyword-based) search in the legal case retrieval domain.
Metadata
| Field | Content | |---|---| | Title | Conversational vs Traditional: Comparing Search Behavior and Outcome in Legal Case Retrieval | | Venue | SIGIR 2021 (short paper) | | Link | https://dl.acm.org/doi/abs/10.1145/3404835.3463064 | | Type | Academic paper | | Category | Verticals / Domain-specific search |
What the Paper Studies
The paper investigates how conversational search compares to traditional search in legal case retrieval — a domain where finding relevant precedents is high-stakes and query formulation is difficult. It examines user search behavior (how queries and interactions are constructed) and outcomes (retrieval effectiveness) under the two paradigms.
Context within Search & Recommendation Research
The forum commentary situates the work in the broader landscape of IR:
- Traditional pipelines: recall (BM25 / dense retrieval) → re-ranking (cross-encoders) → presentation; effective but rigid for multi-turn information needs.
- Conversational search: supports multi-turn, natural-language interaction, which is valuable in legal search where users' information needs evolve through dialogue.
- LLM-era trends: RAG and agentic search extend retrieval into iterative, plannable processes; evaluation shifts from static nDCG toward task success and citation accuracy.
- An interpretable ensemble of graph and language models for legal judgment prediction
- Better to Ask in English: Cross-Lingual Evaluation of Large Language Models
- Automated Query-Product Relevance Labeling using Large Language Models
Reader Takeaways
1. Legal case retrieval is a representative vertical where conversational interfaces may reduce query formulation burden. 2. Comparing behavior (not just final metrics) matters: interaction logs reveal where each paradigm helps or fails. 3. For engineering, trade-offs in latency, cost, explainability, and safety remain hard constraints regardless of paradigm.
> Note: This forum entry is a structured index/summary based on metadata and the paper title; specific quantitative results should be verified against the original PDF via the ACM link above.