English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

PageIndex: Vectorless RAG with Tree Search Achieves 98.7% on FinanceBench

Forum topic · 小凯 · 2026-05-16

Summary

PageIndex (VectifyAI, MIT-licensed, ~30k GitHub stars) replaces traditional vector-database RAG with a tree-based, reasoning-driven retrieval system. Instead of chunking documents into embeddings, it parses PDFs, detects the table of contents, builds a hierarchical tree index, verifies and repairs it with a self-healing pipeline, and lets an LLM reason over the tree to navigate directly to the relevant pages. The flagship Mafin 2.5 financial RAG system scores 98.7% on FinanceBench, reportedly covering 100% of the benchmark, versus 98% (Fintool), 94% (Quantly), 45% (Perplexity), and 31% (ChatGPT 4o + Search). Accuracy held at 98.7% when swapping the underlying LLM from GPT-4o to DeepSeek v3, suggesting the retrieval framework matters more than model size. PageIndex offers page-level citations, transparent retrieval paths, and audit-friendly explainability, at the cost of higher latency and token usage. It is best suited for long, structured documents (financial filings, legal contracts, academic papers) rather than chat logs or unstructured short text, pointing toward hybrid architectures that route queries between tree-based and vector retrieval.

PageIndex: From Similarity Retrieval to Reasoning-Based Retrieval

> Project: PageIndex (GitHub: VectifyAI/PageIndex, ~30k stars) > Team: Mingtian Zhang, Yu Tang, VectifyAI Team > License: MIT > Core claim: Similarity ≠ Relevance — retrieval needs reasoning, not vector similarity > Benchmark: Mafin 2.5 financial RAG → 98.7% on FinanceBench

Key Points

  • The similarity trap: Traditional vector RAG assumes semantic similarity equals relevance. For financial documents this fails — a chunk resembling "Net Income" may come from the wrong company's statement, chunking shreds table headers, and vector space ignores document structure.
  • Core idea: PageIndex automates how human experts read long documents — check the table of contents, reason about which chapter likely contains the answer, then narrow to the section and page.
  • Pipeline: Parse → Detect TOC → Build Tree → Verify → Enrich → Search, using PyMuPDF for parsing, pattern matching + LLM assistance for TOC detection, and LLM summaries for node enrichment.
  • Three tree-building modes: TOC with page numbers, TOC without page numbers (LLM infers page mapping), and no TOC (chunk, generate initial tree, expand incrementally).
  • Self-healing verification: sampled section titles are checked via LLM against their assigned pages; accuracy <60% triggers degradation to a simpler mode, 60–100% triggers up to 3 repair retries.
  • How Retrieval Works (Agentic RAG)

    1. get_document_structure(doc_id) returns the tree structure (headings + summaries, no full text — saves tokens). 2. The LLM reasons over the tree: which branch most likely contains the answer? 3. get_page_content(doc_id, "21-28") extracts only the target pages. 4. The LLM answers from those exact pages, enabling page-level citations and auditable retrieval paths.

    FinanceBench Results

    | System | Accuracy | Full benchmark | Public results | |---|---|---|---| | Mafin 2.5 (PageIndex) | 98.7% | 100% | Yes | | Quantly | 94% | 100% | No | | Fintool | 98% | 66.7% | No | | ChatGPT 4o + Search | 31% | 66.7% | No | | Perplexity | 45% | 66.7% | No |

  • The 31% vs 98.7% gap reflects a retrieval architecture gap, not an LLM capability gap — web/vector search can't precisely locate tables, footnotes, and notes in filings.
  • Mafin 2.5 scored 98.7% with both GPT-4o and DeepSeek v3, implying the retrieval framework contributes more than the underlying model.
  • Caveat: FinanceBench question design and data distribution aren't fully public, so independent verification of overfitting is impossible.
  • Vector RAG vs PageIndex

    | Dimension | Vector RAG | PageIndex | |---|---|---| | Retrieval | Vector similarity | LLM reasoning over tree | | Storage | Vector DB (Pinecone/Weaviate/Chroma) | No vector DB; plain JSON index | | Explainability | Black-box similarity | Transparent ("Risk Factors → p21-28") | | Latency | Milliseconds | Seconds (multiple LLM calls) | | Token cost | Near zero at retrieval | LLM calls for indexing + retrieval | | Citations | Usually none | Exact page and section | | Best for | Short text, general knowledge | Long structured docs (finance/legal/academic) |

    The AlphaGo analogy: PageIndex mimics policy-guided tree search — the LLM acts as the policy network deciding which branch to explore, but via explicit, interpretable reasoning rather than a trained network. "No Vector DB. No Chunking" reduces system complexity and vendor lock-in, but trades latency and LLM dependency.

    Limitations and Outlook

  • Latency/cost: multiple LLM calls per query and for indexing; possible mitigations include tree caching and small-model pruning with large-model final judgment.
  • TOC-less documents: auto-generated trees are of uncertain quality.
  • PDF parsing: PyMuPDF-based; complex layouts still error-prone — vision models could help.
  • Not applicable to code repos, chat logs, or unstructured short text, where vector RAG remains efficient.
  • Single-document trees: cross-document queries need inter-document graphs and global indexes.
  • Conclusion

    PageIndex's value is not just the 98.7% figure but a paradigm proposal: documents are trees, not chunk clouds; retrieval is reasoned navigation, not nearest-neighbor matching. It is not a death sentence for vector databases — rather, structured knowledge needs structured retrieval. The likely endgame is a hybrid architecture: tree-based retrieval for structured long documents, vector retrieval for short/unstructured text, with query routing between the two.

    References

  • PageIndex GitHub: https://github.com/VectifyAI/PageIndex
  • Mafin 2.5 FinanceBench results: https://github.com/VectifyAI/Mafin2.5-FinanceBench
  • TypeScript port: https://github.com/FutureSpeakAI/agent-fridays-pageindex-rag
  • VectifyAI: https://vectify.ai
  • FinanceBench paper: https://arxiv.org/abs/2311.11944 (Patel et al., 2023)

Tags

#pageindex#rag#vectorless-rag#tree-search#financebench#llm#document-understanding#vectifyai

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/177620127