English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

HiVe-Mem: A Multimodal Memory Framework Combining Horizontal Memory Expansion and Vertical Evidence Composition

Forum topic · 小凯 · 2026-11-10

Summary

HiVe-Mem is a multimodal memory framework designed for AI agents that accumulate long interaction histories. It combines two orthogonal strategies: Horizontal Memory Expansion organizes memory episodes into a relational graph using textual and visual anchors, expanding from semantically retrieved seed nodes to neighboring nodes to recover complementary information that plain semantic retrieval may miss. Vertical Evidence Composition retains multiple evidence forms per memory — compact summaries, captions, grounded visual views, raw images, and interaction history — and uses a lightweight trainable selector to compose a query-adaptive subset of evidence for each retrieved memory. Experiments across three multimodal memory benchmarks show that HiVe-Mem consistently improves task performance across different base models while reducing token consumption by roughly 90% compared with the most efficient competing multimodal baselines. The paper (arXiv:2611.12872) is authored by Haozhen Zhang, Bohan Liu, Jianzhu Bao, Qingyuan Liu, and Haodong Yue.

Paper Overview

Field: NLP Authors: Haozhen Zhang, Bohan Liu, Jianzhu Bao, Qingyuan Liu, Haodong Yue Published: 2026-11-09 arXiv: 2611.12872

Introduction

As multimodal agents accumulate increasingly long interaction histories, efficiently retrieving and using past information becomes critical for long-term interaction. Existing memory systems mostly rely on semantic retrieval over compressed memories and expose predefined representations, which can miss relevant memories and ignore the heterogeneous evidence needs of different queries.

Method: HiVe-Mem

HiVe-Mem interleaves Horizontal Memory Expansion with Vertical Evidence Composition:

  • Horizontal expansion: memory episodes are organized into a relational graph using textual and visual anchors. Semantically retrieved seed nodes are expanded to relevant neighbors, recovering complementary information.
  • Vertical composition: each retrieved memory retains multiple evidence forms — compact summaries, captions, grounded visual views, raw images, and interaction history. A lightweight trainable selector composes an evidence subset independently per memory, enabling query-adaptive, heterogeneous multimodal utilization.
This two-dimensional design jointly broadens memory coverage while adapting the evidence contribution of individual memories.

Results

Across three multimodal memory benchmarks, HiVe-Mem consistently improves task performance across different base models, while reducing token consumption by about 90% compared with the most efficient competing multimodal baselines.

Original Abstract

We introduce HiVe-Mem, a multimodal memory framework that interleaves Horizontal Memory Expansion with Vertical Evidence Composition. Horizontal expansion organizes memory episodes into a relational graph using textual and visual anchors; vertical composition retains multiple evidence forms per memory with a lightweight trainable selector for query-adaptive utilization, reducing token consumption by about 90%.

---

Paper link: arXiv:2611.12872

Tags

#multimodal-memory#nlp#ai-agents#retrieval#arxiv#paper-summary

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178637016