SIGIR 2024 The Second Workshop on Generative Information Retrieval (Gen-IR 2024)

Forum topic · 小凯 · 2026-07-05

Summary

Gen-IR 2024 is the second SIGIR workshop dedicated to generative information retrieval, held alongside SIGIR 2024. The forum post compiles an overview of the workshop's scope, covering open-domain search, conversational and agentic search, retrieval-augmented generation (RAG), and the redivision of responsibilities among retrievers, rerankers, planners, and generators in the LLM era. It summarizes core technical themes: unified frameworks for generative IR, component decomposition (representation learning, retrieval, reranking, generation, feedback), reproducible benchmarks, and open problems such as evaluation credibility, latency and cost, hallucination and safety, and cross-lingual/multimodal extension. The post also includes a practical engineering checklist spanning data privacy, p99 latency budgets, quality auditing, security against corpus poisoning, and per-query cost control, plus a glossary of Gen-IR terms. Readers following the retrieval-to-ranking-to-generation-to-evaluation pipeline can use this entry to locate complementary surveys, open-source frameworks, and industry case studies.

SIGIR 2024 The Second Workshop on Generative Information Retrieval (Gen-IR 24)

Overview

Title: SIGIR 2024 The Second Workshop on Generative Information Retrieval
Type: Conference / Workshop
Link: Gen-IR 24 workshop page
Section: Conferences, Workshops

Background and Scope

Large-scale search, recommendation, and personalization systems have long faced challenges in efficiency, scalability, and user-intent understanding. Traditional pipeline approaches treat retrieval, ranking, and generation separately, which makes it hard to meet LLM-era demands for natural-language interaction, multi-hop reasoning, and real-time knowledge. The Second Workshop on Generative Information Retrieval at SIGIR 2024 was organized against this backdrop to consolidate and advance the theory and practice of the intersection between generative models and information retrieval.

Core application scenarios include open-domain information access, enterprise knowledge retrieval, conversational search, semantic understanding in recommendation, and end-to-end architectures that coordinate external knowledge sources with generative models.

Key Themes and Contributions

A unified perspective that brings scattered generative-IR work into a comparable framework.
Clear decomposition of method components: representation learning, retrievers, rerankers, planners, generators, and feedback mechanisms.
Reproducible benchmarks, datasets, and taxonomies that lower the entry barrier for follow-up research.
Interfaces with emerging paradigms such as LLM tool calling, reinforcement learning, and multi-agent collaboration, including migration paths from research prototypes to industrial systems.
Explicit open problems: evaluation credibility, latency and cost, hallucination and safety, and cross-lingual / multimodal extension.

Typical Technical Pipeline

Methods in this space generally follow four stages:

1. Input & representation — encode queries, documents, and user context as dense/sparse representations or structured prompts. 2. Core modules — retrievers, rerankers, planners, memory modules, and tool interfaces, chained or parallelized per task. 3. Learning strategies — supervised fine-tuning, contrastive learning, distillation, RL (including process rewards), and bootstrap data synthesis. 4. Inference strategies — single-turn retrieval, iterative retrieval, parallel sub-queries, early stopping, and budget control.

Insights for Search / Rec / Personalization

1. Architecture: cascaded retrieve–rerank–generate remains mainstream, but the agentic paradigm makes the number and policy of retrieval steps itself learnable. 2. Data: high-quality instruction data and click/session logs are equally critical; synthetic data must guard against knowledge leakage and distribution shift. 3. Evaluation: the gap between offline metrics and online satisfaction is widening; LLM-as-judge needs cross-validation with human assessment. 4. Productization: latency, cost, explainability, and safety policies are hard constraints in deployment — not just academic benchmark optimization.

Engineering Checklist

| Item | Question | Suggestion | |------|----------|------------| | Data | Does training/indexing contain PII? How are versions managed? | Partitioned indexes, sanitization, rollback-capable embedding versions | | Latency | What is the p99 budget? How many retrieval steps? | Cascading + early stopping, caching hot queries, async reranking | | Quality | Do offline gains convert to online CTR/satisfaction? | Interleaving experiments, manual audits, citation verification | | Safety | Does open retrieval introduce poisoning/bias? | Source whitelists, adversarial detection, output filtering | | Cost | Per-query token and GPU usage? | Route to smaller models, distillation, sparse+dense hybrid |

Limitations

Possible limitations include experiment scale constrained by GPU budgets, mismatch between benchmarks and real user distributions, unknown cross-lingual generalization due to English-centric data, and security risks of agentic systems on the open web. Future directions include more efficient test-time compute allocation, deeper integration with knowledge graphs / structured databases, and causal and fairness constraints for recommendation.

Related Entries

2025 SIGIR Workshop on eCommerce
CIKM 2024 1st Workshop on Multimodal Search and Recommendations
EACL 2024 Workshop on Personalization of Generative AI Systems
ICDM MMSR 2025

Glossary

| Term | Meaning | |------|---------| | IR | Information Retrieval | | RAG | Retrieval-Augmented Generation | | LTR | Learning to Rank | | nDCG | Normalized Discounted Cumulative Gain | | Agentic Search | Modeling search as sequential decision-making and tool calls | | Gen-IR | Generative Information Retrieval |

References

Workshop homepage: Gen-IR 24