Querying Databases with Function Calling (arXiv 2502.00032, Jan 2025)
Metadata
| Field | Content | |-------|---------| | Title | Querying Databases with Function Calling | | Authors / Affiliations | Connor Shorten, Charles Pierse, Thomas Benjamin Smith, Karel D'Oosterlinck, Tuana Celik, Erika Cardenas, et al. (12 authors total) | | Published | January 2025 (listed on arXiv) | | Source | https://arxiv.org/abs/2502.00032 | | Type | Academic paper | | Category | Querying Structured Information |
Background and Motivation
At scale, search, recommendation, and personalization systems have long faced challenges in efficiency, scalability, and intent understanding. Traditional pipeline approaches separate retrieval, ranking, and generation, which struggles to meet modern expectations for natural-language interaction, multi-hop reasoning, and up-to-date knowledge in the LLM era. This paper addresses the question of how LLMs should query structured databases — using function calling (tool invocation) rather than only generating SQL — and where the responsibility boundaries between retrieval, ranking, generation, and tool calls should be redrawn.
Core Contributions
- Frames database querying as a function-calling / agentic problem, complementing the Text-to-SQL literature.
- Provides a unified view connecting retrievers, rerankers, planners, generators, and feedback mechanisms in a comparable framework.
- Discusses interfaces with emerging paradigms: LLM tool calling, reinforcement learning, and multi-agent collaboration.
- Identifies open problems: evaluation trustworthiness, latency and cost, hallucination and safety, cross-lingual and multimodal extension.
- Datasets: MS MARCO, BEIR, Natural Questions, domain corpora, public recommendation sets.
- Metrics: nDCG@10, MRR, Recall@k, Hit@k, human preference, task success rate, latency and token cost.
- Baselines: BM25, dense retrieval, cross-encoder reranking, retrieval-free LLMs, commercial search APIs.
- Ablations: contributions of retrieval steps, rerank depth, and training data scale.
- A Survey of Text-to-SQL in the Era of LLMs
- A Survey on Employing Large Language Models for Text-to-SQL Tasks (ACM)
- Large Language Model for Table Processing: A Survey (Frontiers, Jan 2025)
- Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL (arXiv:2406.08426)
- Original paper: Querying Databases with Function Calling. https://arxiv.org/abs/2502.00032
Typical System Architecture
The generic pipeline described in this line of work follows four stages:
1. Input & representation — encode queries, documents, and user context as dense/sparse representations or structured prompts. 2. Core modules — retrievers, rerankers, planners, memory modules, and tool interfaces, composed serially or in parallel. 3. Learning strategies — supervised fine-tuning, contrastive learning, distillation, RL (including process rewards), and synthetic data bootstrapping. 4. Inference strategies — single-shot retrieval, iterative retrieval, parallel sub-queries, early stopping, and budget control.
Evaluation (typical protocol for this area)
Note: exact numerical results should be verified against the original PDF; this post is based on the abstract and public metadata.
Key Insights for Search / Rec / Personalization
1. Architecture: cascaded retrieval + rerank + generation remains mainstream, but agentic paradigms make *when and how often to retrieve* a learnable decision. 2. Data: high-quality instruction data and click/session logs are both critical; synthetic data risks leakage and distribution shift. 3. Evaluation: the gap between offline metrics and online satisfaction is widening; LLM-as-judge needs cross-validation with human evaluation. 4. Product: latency, cost, explainability, and safety are hard constraints for industrial deployment — academic benchmarks alone are insufficient.
Limitations and Future Work
Likely limitations include experiment scale bounded by GPU budgets, benchmark mismatch with real user distributions, English-centric data limiting cross-lingual generalization, and safety risks of agents operating on the open web. Future directions: more efficient test-time compute allocation, deeper integration with knowledge graphs and structured databases, and causal/fairness constraints for recommender systems.