Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL (arXiv, June 2024)
Metadata
| Field | Content | |------|---------| | Title | Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL | | Authors / Affiliations | Zijin Hong, Zheng Yuan, Qinggang Zhang, Hao Chen, Junnan Dong, Feiran Huang, et al. (7 authors total) | | Published | June 2024 | | Source | https://arxiv.org/abs/2406.08426 | | Type | Survey | | Section | Querying Structured Information |
One-line Summary
This survey systematically organizes research on using large language models (LLMs) for Text-to-SQL — converting natural language questions into SQL queries — as the basis of next-generation database interfaces.
Background and Motivation
Large-scale search, recommendation, and personalization systems have long faced challenges around efficiency, scalability, and user-intent understanding. Traditional pipeline approaches often separate retrieval, ranking, and generation, which struggles to meet the combined demands of the LLM era: natural-language interaction, multi-hop reasoning, and real-time knowledge. This survey addresses exactly this intersection, aiming to systematically map the theoretical and practical boundaries of the field.
The core scenario includes open-domain information access, enterprise knowledge retrieval, conversational search, semantic understanding in recommendation, and end-to-end architectures that coordinate external knowledge sources with generative models.
Core Contributions
- Proposes a unified perspective that brings scattered related work into a comparable framework.
- Decomposes method components (representation learning, retrievers, rerankers, planners, generators, feedback mechanisms) for engineering adoption.
- Provides reproducible benchmarks, datasets, and classification tables, lowering the entry cost for researchers.
- Discusses interfaces with emerging paradigms such as LLM tool calling, reinforcement learning, and multi-agent collaboration, indicating migration paths from research prototypes to industrial systems.
- Lists open problems: evaluation trustworthiness, latency and cost, hallucination and safety, cross-lingual and multimodal extension.
- 2019–2021: BERT reranking and DPR establish the foundations of neural retrieval.
- 2022–2023: RAG and FreshLLM push retrieval–generation fusion.
- 2024 onward: Conversational/agentic search and Gen-RecSys explode.
- 2025–2026: RL-trained search agents, Deep Research, and GraphRAG become new growth frontiers.
- Datasets: MS MARCO, BEIR, Natural Questions, domain corpora, public recommendation datasets (for Text-to-SQL: Spider, WikiSQL, BIRD, etc.).
- Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency, and token cost.
- Baselines: BM25, dense retrieval, cross-encoder reranking, retrieval-free LLMs, commercial search APIs.
- Ablations: contribution of each module (retrieval steps, reranking depth, training data size).
- A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where ar…
- A Survey on Employing Large Language Models for Text-to-SQL Tasks, May…
- Large language model for table processing: a survey, Jan 2025, Frontie…
- Querying Databases with Function Calling, Jan 2025, arxiv
- Original paper: Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL, June 2024, arXiv:2406.08426.
Taxonomy
| Dimension | Subclass | Representative ideas | Strengths | Limitations | |------|------|------|------|------| | Modeling paradigm | Discriminative / generative retrieval | Dual-tower, cross-encoder, DSI, GPT indexing | Mature, scalable | Semantic drift, update cost | | LLM integration | RAG / Agent / Tool-use | Retrieval augmentation, search agents, API calls | Flexible, interpretable | Latency, error propagation | | Optimization goals | Relevance / diversity / freshness | Multi-objective LTR, RLHF, online learning | Business-aligned | Scarce annotations | | Evaluation | Offline / Online / Human | nDCG, MRR, LLM-as-judge, A/B | Comparable | Deviates from real satisfaction |
Evolution Over Time
Evaluation Paradigms
Typical benchmarks and trends covered by surveys in this area:
Quantitative results should be checked against the original PDF; this report is based on the abstract and public metadata.
Key Insights
Implications for Search / Rec / Personalization:
1. Architecture: Cascaded retrieval + reranking + generation remains mainstream, but the agentic paradigm is making "number of retrievals and strategy" itself learnable. 2. Data: High-quality instruction data and click/session logs matter equally; synthetic data must guard against knowledge leakage and distribution shift. 3. Evaluation: The gap between offline metrics and online satisfaction is widening; LLM-as-judge needs cross-validation with human evaluation. 4. Product: Latency, cost, interpretability, and safety policies are hard constraints for industrial deployment — do not optimize only academic benchmarks.
Open Problems and Future Directions
Authors commonly point to: insufficient unified benchmarks, non-reproducible private data, LLM evaluation bias, and the safety/cost constraints of agentic systems. Future work includes finer-grained process supervision, joint retrieval–reasoning training, enterprise-grade metadata governance, and multimodal/cross-lingual consistency.
Limitations
Possible limitations include: experiment scale constrained by GPU budgets, mismatch between benchmarks and real user distributions, English-centric data leaving cross-lingual generalization unknown, and safety risks of agent systems on the open web. Future exploration may include more efficient test-time compute allocation, deeper integration with knowledge graphs and structured databases, and causal/fairness constraints for recommendation.