Vector Databases: Giving AI a Sixth Sense for Meaning, Not Just Keywords

Forum topic · 小凯 · 2026-06-08

Summary

This zhichai.net forum post from the easy-learn-ai series explains vector databases through a practical HR scenario: searching 'can unused annual leave be cashed out' fails with keyword matching if the handbook says 'converted at average wage,' but succeeds with vector search. The article covers how embeddings map text into high-dimensional space where semantic similarity becomes numeric proximity; the four-step ingestion pipeline (chunking, embedding, metadata attachment, index building); approximate nearest-neighbor query with Top-K; index structures like HNSW and IVF; metadata filtering and the filter-before-vs-after-search tradeoff; vector search's weakness with exact identifiers, dates, and amounts; and hybrid search combining vector scores with BM25 keyword scores. It also offers a selection framework by data scale, ops preference, and hybrid needs (pgvector, Qdrant, Pinecone, Milvus, Elasticsearch), and positions vector databases as the memory palace powering RAG.

Introduction: A Practical HR Question

"After an employee resigns, can unused annual leave be cashed out?"

In a company with thousands of employees and hundreds of pages of policy documents, finding the accurate answer is hard. Traditional keyword search looks for terms like "resignation," "annual leave," and "cash out." But if the handbook says "unused leave is converted at average wage" instead of "cashed out," the search may miss it—even though the meaning is the same.

This is the problem vector databases solve: they don't ask whether text *looks* similar, but whether it *means* something similar.

Turning Meaning into Numbers: What Is a Vector?

Imagine a coordinate system with hundreds or thousands of dimensions. Two passages with similar meaning—"resignation settlement rules" and "unused leave conversion"—occupy nearby points in this space, while an unrelated passage like "office supplies procurement" sits far away.

A vector compresses a piece of text (or image, audio) into a sequence of numbers, carefully designed so that semantically similar content is numerically close:

"How is annual leave settled at resignation" → [0.42, -0.11, 0.78, 0.23, -0.36, 0.64, 0.08, -0.51]
"Methods for converting unused leave" → [0.45, -0.09, 0.76, 0.25, -0.33, 0.61, 0.10, -0.48]
"Office supplies procurement process" → [-0.23, 0.67, -0.15, 0.82, 0.44, -0.21, 0.91, 0.05]

Vector search turns semantic similarity into mathematical proximity.

Four Steps of Ingestion

1. Chunking: Long documents are split into short, semantically complete paragraphs (e.g., individual policy clauses). 2. Embedding: Each chunk passes through an embedding model, producing a vector of hundreds of dimensions. 3. Metadata: The vector is stored alongside the source text, document name, version, department, and permissions, so retrieval can later check both similarity *and* access rights. 4. Indexing: Without an index, every query would scan the entire database. Indexes pre-build "roads" connecting nearby points so queries navigate quickly to the target region.

Querying: Finding Neighbors

The user's question is also converted into a vector. The database finds the nearest data points in high-dimensional space—like dropping a pin on a map and finding the closest restaurants.

A key parameter is Top-K: how many nearest neighbors to retrieve.

K too small → relevant material gets missed;
K too large → irrelevant content distracts the AI.

Typically K is 3–10, depending on the scenario.

Why It's Fast: Indexes Are Pre-Built Roads

With hundreds of millions of vectors, exhaustive scanning is impossible. The trick is the index: a "neighborhood network" where each vector connects to a few near neighbors. A query starts from an entry point and hops toward progressively closer neighbors—like navigating a pre-built road network instead of measuring straight-line distances across a wilderness.

Common algorithms include HNSW (Hierarchical Navigable Small World graphs) and IVF (Inverted File). All share one idea: trade space for time by pre-recording proximity relationships.

Metadata Filtering: Sifting Before Similarity

Vector search answers "is the meaning similar?"—but enterprises need filters by region, version, and permission:

"China-region resignation rules" vs. "overseas leave policies"
"2026 new edition" vs. "2024 old edition"
"Visible to all" vs. "managers only"

Semantic similarity can't distinguish the 2025 and 2026 versions of nearly identical text. So vector databases need metadata filtering—apply hard conditions first, then search within the filtered set.

This raises a classic question: filter first, then vector search? Or vector search first, then filter? Different databases handle this differently—some merge metadata with vector indexes, others process them separately—affecting both speed and accuracy.

The Weakness: Vectors Don't Understand Numbers and IDs

Vector search is poor at exact information:

"What is POL-HR-2047?"—a one-character difference means a completely different policy;
"Which version takes effect on 2026-03-01?"—dates must match exactly;
"Is the reimbursement cap 800 or 1000?"—a small amount error changes the answer.

"POL-HR-2047" and "POL-HR-2048" may be extremely close in vector space yet describe entirely different policies.

Hybrid Search: 1+1>2

Hybrid search is standard in production systems:

Vector search produces a semantic score (how similar the meaning is);
Keyword search (BM25, inverted index) produces a literal score (how exact the match is);
The two scores are merged with a weight and re-ranked.

Tuning the vector weight: a high weight (e.g., 80%) makes the system better at understanding intent despite different wording; a low weight (e.g., 20%) makes it stricter for exact IDs, dates, and amounts. The result is an assistant that both understands nuance and never confuses 800 with 1000.

Choosing a Database: No Silver Bullet, Only Scenarios

The easy-learn-ai project suggests asking three questions:

1. How much data?

Small (tens of thousands): pgvector, a Postgres plugin, is enough;
Medium (millions): Qdrant, Pinecone;
Large (hundreds of millions): Milvus, specialized infrastructure.

2. Do you want to manage operations?

Small team, fast launch → managed services like Pinecone;
Self-hosting for cost control → Qdrant, Milvus, pgvector.

3. Do you need hybrid keyword search?

Yes → Elasticsearch/OpenSearch (native keyword + vector), Qdrant;
Pure vector → more options: Pinecone, Milvus, Chroma, etc.

There's no "best" vector database—only the one best suited to your current stage.

Conclusion: The AI Era's Memory Palace

If a large language model is a learned scholar, the vector database is its memory palace—an external archive that can be consulted quickly, rather than memory stored in model parameters. When the AI answers, it first retrieves relevant material from this palace and answers based on it. That is the core mechanism of RAG (Retrieval-Augmented Generation).

From this view, a vector database isn't just a technical component—it is a bridge between AI and the real world, letting AI "remember" the latest company policies, "know" your private knowledge base, and "cite" breaking news.

Twenty years ago, search relied on keyword matching. Today, vector databases give search a sixth sense: ask "how is annual leave calculated" and it finds "unused leave conversion"; ask about "resignation procedures" and it recalls "settlement rules" and "approval workflows."

This isn't magic—it's math: distances between points in high-dimensional space, carefully laid paths through index networks, and the teamwork of vectors and keywords. Together with Embedding and RAG, this module completes a full AI application pipeline in the easy-learn-ai knowledge map.