Survey: Large Language Models for Generative Information Extraction (Frontiers of Computer Science, 2024)

Overview

This entry catalogs a 2024 survey, "Large language models for generative information extraction: a survey", published in *Frontiers of Computer Science* (DOI: 10.1007/s11704-024-40555-y). It belongs to the "Search Surveys" section of the curated list.

• Type: Survey
• Link: https://link.springer.com/content/pdf/10.1007/s11704-024-40555-y.pdf

Key points

• The survey examines the shift of information extraction (IE) from discriminative, task-specific pipelines toward generative, unified LLM-based frameworks.
• It covers core IE subtasks: named entity recognition, relation extraction, event extraction, and related structured-prediction tasks.
• Methods are organized along dimensions such as:
• Modeling paradigm: encoder-based discriminative extraction vs. generative decoding of structured outputs.
• LLM integration: RAG, instruction tuning, tool use, and agentic workflows.
• Supervision settings: fully supervised, few-shot, zero-shot, and distantly supervised extraction.
• The survey discusses interfaces with emerging paradigms including LLM tool calling, reinforcement learning, and multi-agent collaboration, plus transfer paths from research prototypes to industrial systems.
• Open problems identified include: benchmark reliability, latency and cost, hallucination and safety, and cross-lingual / multimodal extension.

Context within neural retrieval and extraction

The evolution traced by related work follows this timeline:

• 2019–2021: BERT rerankers and dense retrieval (DPR) establish neural foundations.
• 2022–2023: RAG-style retrieval–generation fusion matures.
• 2024 onward: conversational/agentic search and generative recommendation expand rapidly; LLM-based generative IE becomes a major research direction.

Four main technical lines are typically contrasted:

• Dense retrieval: high recall, low latency; suited for first-stage retrieval.
• Late interaction (e.g., ColBERT): higher accuracy but larger index footprint.
• Generative IR: directly "generating" documents or structured outputs, simplifying cascades.
• Agentic approaches: modeling search/extraction as sequential decisions with multi-hop reasoning and self-reflection.

Evaluation practices

Common datasets and metrics discussed in this research area:

• Datasets: MS MARCO, BEIR, Natural Questions, domain-specific corpora.
• Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency and token cost.
• Baselines: BM25, dense retrieval, cross-encoder rerankers, retrieval-free LLMs.

Note: specific quantitative results should be verified against the original PDF tables.

Takeaways for practitioners and researchers

1. Architecture: cascaded retrieval + rerank + generation remains mainstream, but agentic paradigms treat the retrieval strategy itself as learnable. 2. Data: high-quality instruction data and click/session logs are critical; synthetic data requires care to avoid leakage and distribution shift. 3. Evaluation: the gap between offline metrics and online satisfaction is widening; LLM-as-judge should be cross-validated with human assessment. 4. Deployment: latency, cost, interpretability, and safety are hard constraints for industrial adoption—do not optimize academic benchmarks alone.

Related entries

• A Comprehensive Survey on Reinforcement Learning-based Agentic Search
• A Survey of Conversational Search, Oct 2024, arXiv
• A Survey of LLM Empowered Agents for Recommendation
• A Survey of Model Architectures in Information Retrieval, Jan 2025
• A Survey on AI Search with Large Language Models, July 2025
• A Survey on Knowledge-Oriented RAG, Mar 2025

Glossary

| Term | Meaning | |------|---------| | IE | Information Extraction | | RAG | Retrieval-Augmented Generation | | LTR | Learning to Rank | | nDCG | Normalized Discounted Cumulative Gain | | Agentic Search | Search modeled as sequential decision-making and tool use | | Gen-IR | Generative Information Retrieval |

References

• Original paper: *Large language models for generative information extraction: a survey*, Frontiers of Computer Science, 2024. PDF