HEAVEN: Hybrid-Vector Retrieval for Visually Rich Documents Combines Single-Vector Efficiency with Multi-Vector Accuracy

Forum topic · 小凯 · 2026-07-05

Summary

A 2025 arXiv paper (2510.22215) by researchers including Juyeon Kim and Kijung Shin proposes HEAVEN, a plug-and-play two-stage hybrid-vector retrieval framework for visually rich documents used in legal discovery, scientific search, and enterprise knowledge management. Existing approaches trade off efficiency and accuracy: single-vector retrieval is fast but coarse, while multi-vector retrieval is accurate but computationally expensive. HEAVEN first retrieves candidate pages efficiently with a single-vector method over Visually-Summarized Pages (VS-Pages), which assemble representative visual layouts from multiple pages, then reranks candidates using a multi-vector method while filtering query tokens by linguistic importance to cut redundant computation. The authors also introduce ViMDoc, a benchmark for visually rich, multi-document, long-document retrieval. Across four benchmarks, HEAVEN reaches 99.87% of multi-vector models' Recall@1 while reducing per-query computation by 99.82%. Code and datasets: https://github.com/juyeonnn/HEAVEN

HEAVEN: Hybrid-Vector Retrieval for Visually Rich Documents

Paper: Hybrid-Vector Retrieval for Visually Rich Documents: Combining Single-Vector Efficiency and Multi-Vector Accuracy Authors: Juyeon Kim, Geon Lee, Dongwon Choi, Taeuk Kim, Kijung Shin Published: 2025-10-25 arXiv: https://arxiv.org/abs/2510.22215 Code & data: https://github.com/juyeonnn/HEAVEN

Problem

Retrieval over visually rich documents (scanned PDFs, figures-heavy reports, layout-sensitive pages) is essential for legal discovery, scientific search, and enterprise knowledge management. Current systems face a trade-off:

Single-vector retrieval — efficient, but produces coarse page-level representations.
Multi-vector retrieval — accurate via fine-grained token-level matching, but computationally expensive.

Approach: HEAVEN

HEAVEN is a plug-and-play two-stage hybrid-vector framework that combines both paradigms:

1. Stage 1 — Efficient candidate retrieval: Uses a single-vector method over Visually-Summarized Pages (VS-Pages), which assemble representative visual layouts from multiple pages into compact summaries, enabling fast candidate generation. 2. Stage 2 — Multi-vector reranking: Reranks candidates with a multi-vector method while filtering query tokens by linguistic importance, removing low-value tokens to reduce redundant computation.

New Benchmark: ViMDoc

To evaluate retrieval under realistic conditions, the authors introduce ViMDoc, a benchmark for visually rich, multi-document, and long-document retrieval.

Results

Across four benchmarks, HEAVEN:

Attains 99.87% of the Recall@1 performance of multi-vector models on average
Reduces per-query computation by 99.82%

Achieving near multi-vector accuracy at close to single-vector cost.

Takeaways

Hybrid cascade architectures (cheap recall + expensive rerank) translate well to vision-language document retrieval.
Query-token filtering is a simple, effective way to cut late-interaction costs.
VS-Pages offer a compact representation strategy for layout-heavy corpora.
ViMDoc fills a gap in realistic evaluation of visually rich document retrieval.

References

Original paper: arXiv:2510.22215
Related work: ColPali (efficient document retrieval with vision-language models), EA-VTR (event-aware video-text retrieval)