A Survey of Text-to-SQL in the Era of LLMs: Where Are We, and Where Are We Going?

Forum topic · 小凯 · 2026-07-05

Summary

This arXiv survey (arXiv:2408.05109, August 2024) reviews Text-to-SQL research in the era of large language models (LLMs), examining how the shift from pre-trained encoder models to instruction-tuned and reasoning-capable LLMs has transformed the task of translating natural language questions into SQL queries. It consolidates the fragmented landscape into a unified, comparable framework, decomposing the Text-to-SQL pipeline into components such as schema linking, prompt design, in-context learning, fine-tuning, decomposition and planning, tool use, and self-correction/refinement strategies. The survey covers major benchmarks (e.g., Spider, BIRD, WikiSQL), evaluation protocols and their limitations, and analyzes how LLM-based approaches perform on cross-domain, large-schema, multi-table, and dialect-generalization settings. It also discusses integration with retrieval-augmented generation, agentic workflows, and tool calling, and identifies open problems including evaluation trustworthiness, hallucinated SQL, latency and cost, security concerns around executing generated queries, and handling private enterprise schemas. The authors position the work alongside related surveys on LLMs for Text-to-SQL and LLM-based database interfaces, providing taxonomy tables and cross-references to help researchers and engineers navigate the field.

A Survey of Text-to-SQL in the Era of LLMs: Where Are We, and Where Are We Going?

A structured English digest of the arXiv survey (August 2024) originally shared on zhichai.net.

Source: https://arxiv.org/abs/2408.05109
Authors: Xinyu Liu, Shuyu Shen, Boyan Li, Peixian Ma, Runzhi Jiang, Yuxin Zhang, et al. (10 authors)
Type: Survey
Category: Querying Structured Information

Key points

Problem: Text-to-SQL — translating natural language questions into executable SQL — is a core bottleneck for natural-language access to relational databases, and LLMs have reshaped both the methods and evaluation of this task.
Unified view: The survey consolidates a decade of work (from pre-LLM seq2seq models to modern instruction-tuned and reasoning LLMs) into a comparable framework rather than treating papers as isolated contributions.
Pipeline decomposition: It breaks down the modern Text-to-SQL stack into schema linking, prompt construction, in-context examples, fine-tuning, query decomposition/planning, tool use, and self-correction / execution-feedback refinement.
Benchmarks: Covers widely used datasets such as Spider, BIRD, and WikiSQL, and discusses evaluation metrics and their limitations (execution accuracy vs. exact match, dataset leakage, and generalization to unseen schemas and SQL dialects).
LLM-era techniques: Reviews prompting strategies, retrieval-augmented schema/example selection, decomposition-based reasoning for complex queries, and agentic designs that interleave generation with execution feedback.
Open problems: Evaluation trustworthiness, hallucinated or unsafe SQL, latency/cost constraints, cross-domain and cross-dialect generalization, and applicability to private enterprise schemas with complex metadata.

Positioning and evolution

The survey traces the field's trajectory: early neural seq2seq approaches, pre-trained encoder-based models, then the explosion of LLM-based methods from 2023 onward. It situates Text-to-SQL at the intersection of natural language understanding, structured data access, and LLM tool-calling/agentic paradigms.

Related works referenced by the list

A Survey on Employing Large Language Models for Text-to-SQL Tasks (May 2024)
Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL (arXiv:2406.08426)
Large language model for table processing: a survey (Jan 2025, Frontiers)
Querying Databases with Function Calling (arXiv:2502.00032, Jan 2025)

Takeaways

1. For researchers: Attention should move beyond Spider-style leaderboards toward realistic settings — large schemas, ambiguous questions, dialect diversity, and measurable cost/latency trade-offs. 2. For engineers: Execution-feedback loops and schema-aware retrieval are the most practically impactful components; benchmark numbers should be validated against private-schema behavior. 3. For evaluation: Offline execution accuracy alone is insufficient; citation-style verification of generated SQL and human audits are increasingly important as LLM-as-judge methods spread.

*Note: This digest is based on the original post and public metadata; consult the PDF for precise quantitative claims.*

A Survey of Text-to-SQL in the Era of LLMs: Where Are We, and Where Are We Going?

Summary

A Survey of Text-to-SQL in the Era of LLMs: Where Are We, and Where Are We Going?

Key points

Positioning and evolution

Related works referenced by the list

Takeaways

Tags