Don't Trust the Retrieval Score Just Yet

By day three of chatting with a bot, you've changed your mind about something. Both the original statement and the correction sit in its memory store. You ask a new question, the system retrieves two contradictory records, and it starts confidently making things up.

This is one of the most painful problems in agent memory: it's not enough to remember — the system must know which piece of information has already been overridden by which.

A company called Actioneer in Mountain View, California, published a paper in late October reviving an old technique with a new use: after retrieval, don't take the similarity scores as final — let the knowledge graph recompute them. On the MemoryAgentBench benchmark, their system ActionRAG averages 55.1, 5.4 points above HippoRAG-v2, the strongest prior retrieval-based memory baseline, winning six of eight test columns. Synthetic cost per query stays under $0.001, and the retrieval stage calls no LLM at all.

The indexing-side pitfalls

Good retrieval presupposes a well-built store. The paper highlights two classic failure modes of conventional indexing: one person mentioned ten times under ten name variants becomes ten separate nodes (entity splitting); two coincidentally same-named things get collapsed into one node (entity merging). No amount of retrieval sophistication can recover from that.

ActionRAG handles it with tiered rules, from strict to loose. Unique nodes with identical name and type are reused directly. When there's no exact match, fuzzy matching only reuses a candidate when it's *clearly* the best — near-misses don't count. The rest are grouped by canonical name, type, and description similarity into single nodes, and all nodes are created before any relations are written, preventing parallel processes from creating conflicting ones. Every node and relation keeps a link back to the original conversation snippet, so evidence is always traceable.

The retrieval side: scores can be appealed

Traditional retrieval — including graph-augmented systems like HippoRAG-v2 — scores a batch of candidates, sorts, and stops. The first score is the final verdict.

ActionRAG's observation: relevant evidence rarely stands alone — it has paths around it. A triple with a mediocre score whose two neighbor triples both strongly match the question should be boosted. Conversely, a high-scoring triple whose neighbors are all irrelevant may be inflated.

The method: after retrieval, carve out a small neighborhood subgraph and let scores flow across it. Question-relevant evidence shares its score with neighbors; nodes receiving a lot of score rise, nodes unable to spread score fall, normalized by receptivity. Three safeguards: seed evidence that directly hits the query is locked and can't be washed out; high-degree hub nodes are limited in how much they can spread, so star nodes can't dominate; high-value paths discovered during diffusion are materialized back into the context as triples, recovering connections missed in the first round.

Crucially, graph connectivity is the second review, not the first. Semantic retrieval runs as usual; diffusion only corrects. And since it's purely numeric propagation, retrieval never calls an LLM — a direct contrast with HippoRAG-v2, which uses an LLM online to parse queries, saving tokens and latency on every query.

Breaking down the numbers

Six of eight columns beat HippoRAG-v2; two lose. Among the wins: multi-choice consistency (MCC) 82.0 vs 61.4, a 20.6-point margin; conflict resolution FC-SH 61.3 vs 54.0 and FC-MH 15.5 vs 5.0 — the latter a 3x gap, exactly the "new vs old information fighting" scenario from the opening. Event understanding 77.1 vs 67.6.

The two losses are worth reading carefully. Multi-hop QA (MH): 63.0 vs 66.0, a small 3-point deficit. Detective-style long-range reasoning (DetQA): 46.7 vs 57.7, an 11-point loss. Local graph diffusion only reaches so far — when the evidence chain gets too long, a small neighborhood can't span it. The paper itself notes that graph diffusion algorithms are sensitive to hyperparameters that may need re-tuning on new datasets.

Honesty about cost: the <$0.001 figure is the average synthetic cost across four cost columns, excluding graph construction, storage, and local embedding — one-time investments amortized into queries. Compared to stuffing long context into GPT-4.1-mini, ActionRAG is 39x cheaper on single-document QA; compared to memory agents like MIRIX, 17x cheaper on conflict QA. But against a naive BM25-plus-small-model combo, everyone is in the $0.001 range — and on that column ActionRAG scores 46.7 vs 52.1, a loss.

A hidden piece of good news in the latency numbers: the reported 440.7 ms is the sum of p99 latencies of retrieval components (measured on consumer Apple Silicon); typical p50 latency is only 95.5 ms. Taking 440 as representative means quoting the slowest percentile.

The bigger picture

A separate "Memory Tax" paper reported system-level bad news: of eleven memory systems, none comprehensively beat naive RAG. ActionRAG reads as a targeted rebuttal — every point of its advantage traces to an explicit mechanism: the conflict-column leads correspond to evidence back-links and conflict-aware summarization at indexing; the 20-point MCC margin corresponds to diffusion's second review. The system-level tax can be paid more cheaply, but only if every processing step can explain what it buys.

The divergence from the HippoRAG lineage is also clear: HippoRAG uses personalized PageRank to seed the whole graph from query entities, while ActionRAG diffuses only in the small neighborhood the query activates — a global-propagation vs local-correction debate, where MAB scores currently favor the local side.

Back to that three-day chatbot: your changed mind now sits as two records connected by a "latter supersedes former" edge, and during retrieval its neighbors' scores push it up. Retrieval scores can be appealed — provided the appeal is traceable, with every node linked back to the original words.

---

*Paper: ActionRAG: Evidence-Refining Graph Diffusion for Long-Term Memory in LLM Agents (arXiv:2610.32752, submitted 2026-10-23, ACML 2026 workshop, CC BY-NC-ND). Authors: Navyansh Mahla, Sashank Vandrangi (Actioneer). All benchmark figures come from Table 1/Table 4 and the abstract; cost methodology is detailed in Section 4.3.*