RAG Regression: From Vector Databases Back to BM25 and SQLite
> Research date: 2026-05-11 > Core thesis: While most people chase vector databases, a group of RAG practitioners are collectively "regressing" — using BM25 and SQLite to handle most scenarios. Code retrieval, in particular, should return to traditional methods. This is not technical regression but a return to fundamentals.
1. The Phenomenon: Disenchantment with Vector Search
Vector databases (Pinecone, Weaviate, Qdrant, Milvus) saw explosive growth in 2023-2024 — DB-Engines reports search interest in Vector DBMS grew 340% since January 2024. But production experience increasingly shows pure vector search underperforms in high-value scenarios.
A widely circulated claim that "126 developers found vector search unnecessary in 85% of scenarios" could not be traced to an original source. However, supporting evidence exists:
- BEIR benchmark: BM25 remains a strong zero-shot baseline; dense models only reliably beat it when training and test domains overlap.
- Production query logs: roughly 30-50% of real queries are lookup-style (users already know the exact string), not conversational.
- Industry forecasts: 85% of enterprise search expected to adopt RAG by 2027 — but most RAG systems are moving to hybrid architectures, not pure vector search.
- https://www.sqlite.org/fts5.html
- Built-in full-text search with BM25 ranking (
rank=bm25()), prefix/phrase/NEAR queries, transactional integration. Ideal for local-first apps, offline RAG, mobile, prototyping. No vector support (pair with sqlite-vec). - https://github.com/asg017/sqlite-vec
- C-level SQLite vector extension: float32 and binary vectors, Hamming distance + cosine similarity. ~135 MB for 1M docs with binary vectors. SitePoint benchmarks (2026-02-19): p50 latency 2.3ms @ 10k docs, 22ms @ 100k, 215ms @ 1M.
- https://github.com/pgvector/pgvector
- HNSW (up to 16,000 dims) and IVF-Flat indexes, integrates with SQL transactions, filters, JOINs. Suits teams already on PostgreSQL with < 5M vectors. Hybrid recipe:
tsvector+ pgvector + RRF. - https://github.com/jankovicsandras/plpgsql_bm25
- Pure PL/pgSQL BM25 — zero external dependencies, works in any managed Postgres where Rust extensions are unavailable. 8 BM25 variants, 16-language stopword filtering, built-in hybrid (BM25 + pgvector + RRF) implementation. Optimized
BM25optis 30-40x faster than Python'srank_bm25. Public domain (Unlicense). - https://lancedb.com/
- Embedded, zero-copy columnar vector DB with disk-based IVF-PQ indexes and native hybrid search (kNN + BM25 via Tantivy), built-in rerankers (RRF, linear combination, Cohere, ColBERT).
- https://www.meilisearch.com/
- Developer-friendly open-source engine with built-in hybrid search (HNSW vectors + improved BM25) in a single query, typically < 50ms latency.
- https://huggingface.co/MongoDB/mdbr-leaf-ir
- MongoDB Research's 23M-parameter retrieval model, distilled from snowflake-arctic-embed-m-v1.5. Ranks #1 on BEIR among ≤100M-parameter models, supports MRL and quantization, runs on CPU (~120 queries/sec on 2 vCPUs, 87MB memory). Proves small distilled models can replace expensive embedding APIs.
- RRF (Reciprocal Rank Fusion):
score = Σ 1/(k + rank_i), k=60. No score normalization needed, robust to outliers. The default choice. - Weighted score fusion: normalize both scores to 0-1 and weight — tunable but needs labeled data.
- BEIR: BM25 is a strong zero-shot baseline across datasets; dense models win mainly with domain overlap; cross-encoder rerankers generalize well.
- AutoRAG (2024): BM25 beat vector retrieval on multiple precision metrics.
- RAG Benchmark for Company Internal Knowledge (arXiv:2605.05253): BM25 (OpenSearch) vs vector search (Qdrant + text-embedding-3-large) traded wins on document recall; a grep/find/head-based Bash Agent sometimes matched or beat both.
- Vector search = semantic similarity (find things *like* this)
- BM25 = keyword relevance (find things *containing* this)
- Hybrid = complementary (semantically relevant AND exactly matching)
2. Five Systematic Failure Modes of Vector Search
Based on Gabriel Anhaia's analysis (2026-04-18):
1. Rare identifiers — searching SKU-47291 or CVE-2024-3094 returns neighboring IDs because embeddings average subword fragments.
2. Proper nouns — searching Dvorak keyboard returns QWERTY/ergonomic keyboards: semantically close but wrong.
3. Abbreviations — RCE in JPA returns generic Java ORM vulnerability content, missing the specific CVE.
4. Negation and exact phrases — "How do I cancel without a fee?" surfaces fee-containing policies, not no cancellation fee pages.
5. Version strings — Python 3.11.7 returns Python 3.11 results; embeddings lack sub-version resolution.
> These are not edge cases — they often exceed 30% of real query logs in technical products.
3. Code Retrieval Must Return to Traditional Methods
| Feature | Vector search problem | BM25/traditional advantage |
|---|---|---|
| Exact function/class names | get_user_by_id gets over-generalized | Exact string match |
| Versions / commit hashes | v2.1.3 ≈ v2.1.4 in embedding space | Exact match, sortable |
| Error codes | E_PERMISSION_DENIED_42 gets blurred | Exact match |
| API parameter names | userName vs user_name seen as equivalent | Regex/pattern matching |
| Type signatures | List<String> vs List<Integer> indistinguishable | AST analysis + exact match |
Supporting evidence: on MTEB (Code), BM25-to-dense gaps are smaller than in general text, with hybrid (BM25 + vector + cross-encoder reranker) performing best. CodeSearchNet experiments show pure vector retrieval achieves only 40-60% of BM25's top-1 recall on exact function-name queries. Industry practice: Vercel's Bash Agent (2025) uses iterative grep/find/head-based search instead of vectors, and OpenClaw's memory-lancedb Issue #7629 requests BM25 hybrid search because pure vectors "miss exact keyword matches (IDs, env vars, code symbols)."
4. Seven Tools in Depth
SQLite FTS5
sqlite-vec
pgvector
plpgsql_bm25
LanceDB
Meilisearch
mdbr-leaf-ir
5. Hybrid Search Implementation
Hybrid = BM25 (sparse) + dense vector retrieval + reranker — run in parallel, ranks fused.
Fusion algorithms:
Platform support:
| Platform | Full-text | Vector | Native fusion |
|---|---|---|---|
| SQLite + sqlite-vec | FTS5 | sqlite-vec | manual (SQL CTE + RRF) |
| PostgreSQL | plpgsql_bm25 / tsvector | pgvector | manual / RRF SQL |
| LanceDB | Tantivy | IVF-PQ/HNSW | query_type="hybrid" |
| Meilisearch | improved BM25 | HNSW | single query |
| Weaviate | BM25 | HNSW | hybrid param |
| Qdrant | sparse encoding | HNSW | Prefetch + Fusion API |
| Pinecone | learned sparse | dense | alpha param |
| MongoDB Atlas | Lucene/BM25 | kNN | \(search +\)vectorSearch + RRF |
| Elasticsearch/OpenSearch | BM25 | k-NN | rrf retriever |
Cross-encoder rerankers matter: reranking the fused top 30-50 candidates (e.g., bge-reranker-v2-m3) often yields quality gains exceeding the fusion step itself, at ~30-80ms cost.
6. Academic and Industry Validation
7. Practical Checklist You Can Do Monday Morning
1. Add a sparse retriever: enable tsvector or plpgsql_bm25 on Postgres; turn on hybrid mode in ES/OpenSearch/Pinecone/Weaviate/Qdrant; FTS5 + sqlite-vec dual index on SQLite.
2. Fuse with RRF (k=60); move to weighted fusion only with labeled data.
3. Add a cross-encoder reranker: rerank top 30-50 with bge-reranker-v2-m3 or cross-encoder/ms-marco-MiniLM-L-2-v2.
4. Monitor query classification: label ~200 recent queries as lookup vs conversational; a 20-40 point recall improvement on lookup queries is the business case for migration.
5. Code retrieval strategy: exact match + BM25 for names, AST analysis for API signatures, exact/range queries for versions, inverted index for error codes; reserve semantic search for concept discovery.
8. Conclusion: Not a Regression, a Return to Fundamentals
1. Vector search is not omnipotent — it predictably fails on identifiers, negation, versions, and exact matches. 2. BM25 is not obsolete — a 40-year-old algorithm remains the strong zero-shot baseline. 3. Code retrieval needs traditional methods — structured code favors BM25 + AST + exact matching. 4. Hybrid search is mainstream — every serious vector DB added hybrid support in 2024-2025; it's mandatory, not optional. 5. Small models are changing cost structures — mdbr-leaf-ir achieves SOTA on CPU, enabling local RAG.
Recommended stacks by scenario:
| Scenario | Stack |
|---|---|
| Existing Postgres | tsvector/plpgsql_bm25 + pgvector + RRF |
| Local-first/edge | SQLite FTS5 + sqlite-vec + optional cross-encoder |
| Large disk-indexed datasets | LanceDB (hybrid mode) |
| Instant search UX | Meilisearch (built-in hybrid) |
| MongoDB ecosystem | Atlas Search + Atlas Vector Search + RRF |
| Code retrieval | BM25 + exact match + AST, vectors only as auxiliary |
> "Your vector database is a retriever. It's a retriever that's good at one class of queries. But it's not a search engine, and adding embedding dimensions won't turn it into one. The solution is 40 years old and costs one fusion function." — Gabriel Anhaia, 2026
---
References
1. Gabriel Anhaia — "Your Vector Database Is Not a Search Engine" (2026-04-18) https://dev.to/gabrielanhaia/your-vector-database-is-not-a-search-engine-heres-why-thats-killing-your-rag-2db5 2. SitePoint — "Local-First RAG: Vector Search in SQLite with Hamming Distance" (2026-02-19) https://www.sitepoint.com/local-first-rag-vector-search-in-sqlite-with-hamming-distance/ 3. MongoDB Research — "LEAF: Distillation of State-of-the-Art Text Embedding Models" (2025-11-25) https://mongodb.com/company/blog/engineering/leaf-distillation-state-of-the-art-text-embedding-models 4. arXiv:2509.12539 — "LEAF: Knowledge Distillation of Text Embedding Models with Teacher-Aligned Representations" https://www.arxiv.org/pdf/2509.12539 5. BEIR Benchmark — Thakur et al. (2021) https://arxiv.org/pdf/2104.08663 6. AutoRAG Framework — arXiv:2410.20878 (2024) https://arxiv.org/pdf/2410.20878 7. A RAG Benchmark for Company Internal Knowledge — arXiv:2605.05253 (2026-05-05) https://arxiv.org/html/2605.05253v1 8. Fin-RATE Financial Benchmark — arXiv:2602.07294 (2026) https://arxiv.org/html/2602.07294v3 9. LanceDB Hybrid Search Docs (2026-05-04) https://docs.lancedb.com/search/hybrid-search 10. OpenClaw memory-lancedb Issue #7629 (2026-02-02) https://github.com/openclaw/openclaw/issues/7629 11. plpgsql_bm25 — https://github.com/jankovicsandras/plpgsql_bm25 12. SQLite FTS5 — https://www.sqlite.org/fts5.html 13. sqlite-vec — https://github.com/asg017/sqlite-vec 14. pgvector — https://github.com/pgvector/pgvector 15. LanceDB — https://lancedb.com/ 16. Meilisearch — https://www.meilisearch.com/ 17. mdbr-leaf-ir — https://huggingface.co/MongoDB/mdbr-leaf-ir