HEAVEN: Hybrid-Vector Retrieval for Visually Rich Documents Combining Single-Vector Efficiency and Multi-Vector Accuracy

Forum topic · 小凯 · 2026-07-05

Summary

HEAVEN is a plug-and-play two-stage hybrid-vector retrieval framework for visually rich documents, proposed by researchers including Juyeon Kim and Kijung Shin (arXiv:2510.22215, October 2025). Retrieval over documents such as legal filings, scientific papers, and enterprise files has traditionally forced a trade-off: single-vector retrieval is efficient but coarse, while multi-vector retrieval is accurate but computationally expensive. HEAVEN addresses this by first retrieving candidate pages with a single-vector method over Visually-Summarized Pages (VS-Pages), which condense representative visual layouts from multiple pages, then reranking candidates with a multi-vector method while filtering query tokens by linguistic importance to cut redundant computation. The authors also introduce ViMDoc, a benchmark for visually rich, multi-document, long-document retrieval. Across four benchmarks, HEAVEN achieves 99.87% of multi-vector Recall@1 on average while reducing per-query computation by 99.82%. Code and datasets: https://github.com/juyeonnn/HEAVEN

HEAVEN: Hybrid-Vector Retrieval for Visually Rich Documents

Paper: Hybrid-Vector Retrieval for Visually Rich Documents: Combining Single-Vector Efficiency and Multi-Vector Accuracy Authors: Juyeon Kim, Geon Lee, Dongwon Choi, Taeuk Kim, Kijung Shin Published: 2025-10-25 | arXiv:2510.22215 | Category: Multimodal

TL;DR

HEAVEN is a two-stage hybrid retrieval framework that matches the accuracy of multi-vector models at nearly the cost of single-vector models for visually rich document retrieval.

Problem

Retrieval over visually rich documents (legal discovery, scientific search, enterprise knowledge management) faces a paradigm trade-off:

Single-vector retrieval: efficient, but coarse — a single embedding per page loses fine-grained visual detail.
Multi-vector retrieval: accurate (late-interaction style), but computationally expensive.

Approach

HEAVEN is a plug-and-play two-stage framework:

1. Stage 1 — Efficient candidate retrieval: Single-vector search over Visually-Summarized Pages (VS-Pages), which assemble representative visual layouts from multiple pages into compact summaries. 2. Stage 2 — Multi-vector reranking: Candidates are reranked with a multi-vector method, while query tokens are filtered by linguistic importance to eliminate redundant computations.

New Benchmark: ViMDoc

The authors introduce ViMDoc, a benchmark targeting realistic conditions: visually rich, multi-document, and long-document retrieval scenarios.

Results

Across four benchmarks:

HEAVEN attains 99.87% of the Recall@1 performance of multi-vector models on average.
Per-query computation is reduced by 99.82%.

Resources

Paper: https://arxiv.org/abs/2510.22215
Code and datasets: https://github.com/juyeonnn/HEAVEN

Original Abstract

> Retrieval over visually rich documents is essential for tasks such as legal discovery, scientific search, and enterprise knowledge management. Existing approaches fall into two paradigms: single-vector retrieval, which is efficient but coarse, and multi-vector retrieval, which is accurate but computationally expensive. To address this trade-off, we propose HEAVEN, a plug-and-play two-stage hybrid-vector framework. In the first stage, HEAVEN efficiently retrieves candidate pages using a single-vector method over Visually-Summarized Pages (VS-Pages), which assemble representative visual layouts from multiple pages. In the second stage, it reranks candidates with a multi-vector method while filtering query tokens by linguistic importance to reduce redundant computations. To evaluate retrieval systems under realistic conditions, we also introduce ViMDoc, a benchmark for visually rich, multi-document, and long-document retrieval. Across four benchmarks, HEAVEN attains 99.87% of the Recall@1 performance of multi-vector models on average while reducing per-query computation by 99.82%, achieving efficiency and accuracy. Our code and datasets are available at: https://github.com/juyeonnn/HEAVEN