Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL (Jun 2024, arXiv)
Metadata
| Field | Content | |---|---| | Title | Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL | | Authors / Affiliations | Zijin Hong, Zheng Yuan, Qinggang Zhang, Hao Chen, Junnan Dong, Feiran Huang, et al. (7 authors) | | Published | June 2024 | | Source | https://arxiv.org/abs/2406.08426 | | Type | Survey | | Section | Querying Structured Information |
One-line Summary
This survey systematically organizes LLM-based Text-to-SQL research, mapping how large language models are reshaping next-generation database interfaces.
Background and Motivation
At-scale search, recommendation, and personalization systems have long struggled with efficiency, scalability, and user-intent understanding. Traditional pipelines separate retrieval, ranking, and generation, which makes it hard to meet LLM-era demands for natural-language interaction, multi-hop reasoning, and up-to-date knowledge. This survey addresses that gap for the Text-to-SQL domain: translating natural-language questions into SQL over structured databases. Core scenarios include open-domain information access, enterprise knowledge retrieval, conversational search, and end-to-end architectures that couple external knowledge sources with generative models.
Core Contributions
- Provides a unified perspective that organizes scattered Text-to-SQL work into a comparable framework.
- Decomposes method components (schema representation, retrieval, re-ranking, planning, generation, feedback) for engineering practice.
- Offers reproducible benchmarks, datasets, and taxonomy tables, lowering the entry cost for new researchers.
- Discusses interfaces with LLM tool calling, reinforcement learning, and multi-agent collaboration, and paths from research prototypes to industrial systems.
- Lists open problems: evaluation trustworthiness, latency and cost, hallucination and safety, cross-lingual and multimodal extension.
- 2019–2021: BERT re-ranking and DPR establish neural retrieval foundations.
- 2022–2023: RAG and FreshLLMs drive retrieval-generation fusion.
- 2024: Conversational/agentic search and Gen-RecSys take off.
- 2025–2026: RL-trained search agents, Deep Research, and GraphRAG become new growth fronts.
- Datasets: MS MARCO, BEIR, Natural Questions, domain corpora, public recommendation sets; for Text-to-SQL, Spider and BIRD-style benchmarks.
- Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency, token cost.
- Baselines: BM25, dense retrieval, cross-encoder re-ranking, retrieval-free LLMs, commercial search APIs.
- Ablations: contributions of retrieval steps, re-ranking depth, and training data scale.
- A Survey of Text-to-SQL in the Era of LLMs
- A Survey on Employing Large Language Models for Text-to-SQL Tasks
- Large language model for table processing: a survey, Jan 2025
- Querying Databases with Function Calling, Jan 2025
- Original: *Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL*, June 2024, arXiv: https://arxiv.org/abs/2406.08426
Taxonomy of Approaches
| Dimension | Sub-categories | Representative ideas | Strengths | Limitations | |---|---|---|---|---| | Modeling paradigm | Discriminative / generative retrieval | Dual-tower, cross-encoder, DSI, GPT-indexed | Mature, scalable | Semantic drift, update cost | | LLM integration | RAG / Agent / Tool-use | Retrieval-augmented, search agents, API calls | Flexible, interpretable | Latency, error propagation | | Optimization goals | Relevance / diversity / freshness | Multi-objective LTR, RLHF, online learning | Business-aligned | Scarce annotations | | Evaluation | Offline / online / human | nDCG, MRR, LLM-as-judge, A/B tests | Comparable | Diverges from real satisfaction |
The field generally juxtaposes four main lines: dense retrieval (high recall, low latency, first-stage), late interaction such as ColBERT (higher accuracy, larger indexes), generative IR (directly generating docids, simplifying cascades), and agentic search (sequential decision-making with multi-hop and self-reflection).
Timeline of Research Evolution
Evaluation Paradigms
Typical benchmarks and trends covered:
Quantitative results should be checked against the original PDF tables.
Key Takeaways
1. Architecture: cascaded retrieve → re-rank → generate remains mainstream, but agentic paradigms make retrieval policy itself learnable. 2. Data: high-quality instruction data and click/session logs matter equally; synthetic data risks leakage and distribution shift. 3. Evaluation: the gap between offline metrics and online satisfaction is widening; LLM-as-judge needs human cross-validation. 4. Product: latency, cost, explainability, and safety are hard industrial constraints—not just academic benchmarks.
Open Problems and Future Directions
The authors highlight insufficient unified benchmarks, non-reproducible private data, LLM evaluation bias, and the safety/cost constraints of agentic systems. Future work includes finer-grained process supervision, joint retrieval-reasoning training, enterprise metadata governance, and multilingual/multimodal consistency.
Engineering Checklist
| Item | Question | Recommendation | |---|---|---| | Data | PII in training/index? Version control? | Partitioned indexes, anonymization, rollback-capable embeddings | | Latency | p99 budget? Retrieval steps? | Cascade + early stop, hot-query caching, async re-ranking | | Quality | Do offline gains translate online? | Interleaving experiments, human audits, citation checks | | Safety | Poisoning/bias from open retrieval? | Source whitelists, adversarial detection, output filtering | | Cost | Per-query token/GPU usage? | Small-model routing, distillation, hybrid sparse-dense |