Corpus2Skill: Replacing Retrieval with LLM Navigation over a Skill Tree

Forum topic · 小凯 · 2026-04-22

Summary

Corpus2Skill is a system from the Wix team that reframes enterprise knowledge-base question answering by letting an LLM agent browse a pre-compiled Markdown skill tree instead of performing vector retrieval. In an offline compile step, documents are embedded, hierarchically clustered, and summarized by an LLM into a folder-like structure of SKILL.md and INDEX.md files. At query time, only the top-level skill directory (~200 tokens) is preloaded, and the agent progressively drills down, reads full documents on demand, backtracks, or synthesizes across branches. On the WixQA benchmark (200 real customer-support queries) it outperforms dense retrieval, RAPTOR, and multi-turn agentic RAG. Failures (31%) mainly stem from top-level skill-selection errors and over-retrieval at leaf nodes, highlighting trade-offs between tree depth and decision complexity.

One-line summary

The Wix team turns an enterprise knowledge base into a "skill tree" so that an LLM agent *browses* it like a human navigating folders, rather than relying on keyword-based vector retrieval. No vector database, no embedding lookup at query time—just LLM reasoning over a compiled Markdown structure.

The Problem with Traditional RAG

Consider a customer-support query like: *"How do I convert my sole-proprietorship Wix account to an LLC?"* This touches account types, payment configuration, and legal-entity management. A typical RAG pipeline embeds the query, retrieves the top-K most similar chunks, and hands them to the LLM. Critical information—for example, *"account type cannot be changed directly; contact support"*—may be missed because its surface vocabulary doesn't match the query. Worse, the LLM has no global view of the knowledge base: it cannot decide to try a different direction or stitch together information from multiple departments. The paper calls this *"seeing trees but not the forest"*—RAG turns the LLM into a passive consumer of search snippets rather than an active explorer.

Core Idea: From Retrieval to Navigation

The insight is simple: instead of searching the corpus, have the LLM browse it.

Offline Compile Phase

1. Embed + cluster: All documents are embedded and hierarchically clustered into a topic tree. 2. LLM summarization: At each level, the LLM writes a summary—top level becomes SKILL.md (skill catalog), middle levels become INDEX.md (subtopic indexes), and leaves are individual documents. 3. Materialize as files: The output is a set of human-readable Markdown files organized like folders, not a database index.

Online Serve Phase

For each query, the LLM agent:

1. Sees the bird's-eye view — all top-level skill names and one-line descriptions (~200 tokens) are preloaded. 2. Selects a skill — reads the corresponding SKILL.md to learn the subtopic structure. 3. Drills down — reads INDEX.md to see specific documents. 4. Fetches on demand — uses a get_document tool to read full content. 5. Backtracks or crosses branches — switches paths when needed, or combines information across branches.

The flow mirrors how humans use a company wiki: check the table of contents, click into a relevant page, return when off-track, and ultimately lock onto the target.

Key Design: Progressive Disclosure

Preloading every navigation file would drown the context in summaries; preloading nothing leaves the LLM blind. Progressive disclosure takes a middle path: only the directory (skill names + one-sentence descriptions) is loaded for the first decision. Detailed content is loaded dynamically as the agent commits to a branch.

Two benefits:

Token efficiency: long documents are read only when necessary.
Interpretability: every step is grounded in an explicit summary, making the navigation path transparent and auditable.

Experimental Results

On the WixQA benchmark (200 real enterprise customer-support queries), Corpus2Skill beats dense retrieval, RAPTOR, and multi-turn agentic RAG. Traditional retrieval is a one-shot operation—query to vector match to top-K—and if the first pass misses a critical document, the LLM never sees it. Corpus2Skill's navigation is iterative: the LLM can backtrack, switch directions, and synthesize across branches.

Two typical trajectories are shown:

Direct navigation (4 steps): query → pick skill → pick subgroup → read document → answer.
Cross-navigation (7 steps): query → pick skill → explore subgroup A → realize it's incomplete → switch to subgroup B → synthesize information from both → answer.

This trial-and-error plus synthesis capability is unavailable to black-box retrieval.

Honest Limitations

The paper devotes a full section to failure analysis, which is itself scientifically valuable.

Out of 200 queries, 62 (31%) failed, for three main reasons:

1. Navigation errors (38, 61% of failures): the LLM picks the wrong top-level skill (e.g., a CMS-collection-sorting question routed to "site editing" instead of "CMS"). Top-level category granularity is the bottleneck—themes are too coarse and boundaries between similar skills are blurry. 2. Over-retrieval (19, 31%): the LLM finds the right topic but pulls in too many adjacent, irrelevant documents, diluting context. Leaf-level grouping precision needs improvement. 3. Synthesis errors (3, 5%): the LLM misreads conditional instructions as general rules, or over-generalizes. This is an inherent LLM reasoning issue, not an architectural flaw.

An ablation compresses the top-level clustering from 15 categories down to 3 ("compact mode"), reducing navigation errors—fewer options means lower decision complexity. This reveals a trade-off: flatter trees make navigation easier but each node's summary grows longer; deeper trees keep summaries short but add decision steps.

A Feynman-Style Question: Is This Really "Don't Retrieve"?

The slogan "Don't Retrieve, Navigate" is catchy but slightly misleading. The compile phase still does embedding and clustering—essentially retrieval preprocessing. Corpus2Skill simply moves retrieval from query time to compile time. The benefit is no vector database at runtime; the cost is that any knowledge-base update requires recompiling the entire skill tree.

A more accurate framing: *"shift retrieval cost from query time to compile time, and use LLM reasoning instead of vector similarity for final document selection."* This isn't a rejection of retrieval—it redistributes compute across time.

Adoption Guidance

Good fit when:

The enterprise knowledge base is relatively stable.
Queries are complex and require cross-topic synthesis.
Infrastructure such as Anthropic's Skills API or similar is available.

Not a good fit when:

Millisecond-level response is required (navigation needs multiple LLM API calls).
The knowledge base changes frequently (each update triggers recompilation).
Queries are simple factual lookups (traditional vector RAG is faster and cheaper).

Takeaway

Corpus2Skill's most valuable contribution is a paradigm shift: from *"the LLM consumes search results"* to *"the LLM actively explores a knowledge structure."* It reminds us that RAG's problem may not be retrieval accuracy but the whole interaction model—the LLM should not be a search-engine user, but an explorer that finds its own way through a knowledge forest.

That said, the 31% failure rate is real. Skill-tree quality, top-level classification precision, and LLM decision stability remain engineering challenges. The slogan is sexy; the deployment is bone-hard.

> *"Compile first, navigate later."* Future knowledge-base systems may no longer be "database + retriever" but "pre-compiled navigable structure + reasoning agent."

---

📄 Paper: arXiv:2604.14572 🏢 Authors: Yiqun Sun, Pengfei Wei, Lawrence B. Hsieh (Wix) 🔗 Code: https://github.com/dukesun99/Corpus2Skill