Paper Overview
Field: NLP Authors: Arjun Rao, Nishant Gopinath Published: 2026-11-10 arXiv: 2611.14477
Abstract
Text-to-SQL systems over large pooled catalogs must first retrieve the tables a question needs. FERN (Feature-Engineered Ranking with No generative LLM) compares against the published table recall of RASL, which uses Claude 3.5 models to extract keywords and select tables, on pooled BIRD, Spider, and Fiben catalogs.
Method
FERN instead:
- Fuses three frozen retrievers into 100 candidates
- Describes each candidate with up to 120 features
- Reorders them with LightGBM rankers fitted once on BIRD's training questions
- Spider (not fitted): The candidate pool alone reaches RASL's published recall at fifteen tables (98.0); the ranker raises complete@5 (share of questions with every gold table in the top five) from 83.9% to 95.6%
- At five tables: FERN's recall is within one point of RASL's on Spider (96.7 vs 97.0), 96.6 vs 97.5 on BIRD with evidence hints, and 8.2 points lower on BIRD without hints (89.3)
- BEAVER (121 real questions, not fitted): Recall at fifteen tables (82.1) falls between the two BEAVER-trained retrievers (80.8, 84.5), with complete@15 below both (49.6 vs 54.5 and 62.0)
- Fiben: A clear gap at fifteen tables (54.2 vs 69.2, on different question sets); recall at fifty tables is close to RASL's question-only retrieval (80.7 vs 79.4)
- Feature ablation: On BIRD dev with evidence hints, non-retrieval features improve complete@5 by 7.7 points over retrieval-only signals
Key Results
Efficiency
On BIRD dev, FERN runs at a median latency of 208 ms per question on a single A100 GPU, with no calls to hosted models.
--- *Auto-collected on 2026-11-12*