Overview
Field: ML Authors: Zhiyi Chen, Jie Song, Peng Li Published: 2026-06-10 arXiv: 2606.12387
Summary
Large Language Models (LLMs) have democratized database access through Text-to-SQL, but moving from prototypes to production remains difficult. Real deployments must handle strict SQL dialects, massive schemas, and evolving user preferences, while supervised fine-tuning is costly and rigid and agentic test-time scaling is expensive.
TAHOE treats prompt optimization as a dynamic data management problem. It uses an error-driven hint learning pipeline across Development and Deployment to consolidate debugging traces into a structured Hint Bank:
- Syntax Hints: distilled from compiler feedback as reusable rules for dialect-specific SQL syntax.
- Semantic Hints: derived from execution and user feedback for schema- and user-specific logic.
- Policy layer: models conflicting user intents as competing policies under shared natural-language triggers, with recency signals and post-learning attribution statistics summarizing experiential success, harm, inaction, and support.
- Pass rate: 61.95% → 79.42%
- Pass-at-4: 72.57% → 87.61%
- 100% Snowflake syntax pass rate
- Average compiler-feedback critique rounds per sampled candidate: 2.79 → 0.12
- The same Hint Bank transfers to weaker backbones, including a 19.7 percentage-point pass-rate improvement on Doubao-2.0-lite.
- Paper: arXiv 2606.12387
At inference time, TAHOE retrieves relevant hints and guides the LLM through logical planning followed by SQL synthesis. The current implementation and evaluation cover the development-phase workflow; deployment-time human-feedback updates are left to future work.
Results
On Spider 2.0-Snow (113 supervised Spider 2.0-Snow-0212 examples, GPT-5.5), without updating model parameters: