In 2026, Building AI Search Means Building Agent Memory

*Based on a talk by Xiao Han (Elastic VP, former founder/CEO of Jina AI) at the Elastic China AI Search Technology Conference, April 18, Beijing.*

Key points

• Search has evolved through six paradigm shifts: keyword retrieval (BM25, TF-IDF, inverted indexes) → vector search (Faiss, Milvus) → hybrid search → Transformer/BERT embeddings → RAG after ChatGPT → Deep Research with reasoning models (DeepSeek-R1). In 2026, agent platforms like OpenClaw let agents work autonomously for hours, shifting search's role from "giving users ten results" to "providing the agent one correct answer."
• The 2026 technical battleground is Long Horizon Tasks: when agents run 4–5 hours unattended, persistent memory becomes a requirement, not an option. Per Andrej Karpathy: today's LLM personalization is stuck on the same problem—they cannot selectively forget.

A real-world memory failure: the "911" case

Xiao Han asked an agent to find a "911 chart" he had previously made (911 = a Porsche model). The agent translated it to "September 11th" and searched in vain. Debugging showed:

• The model was Claude Opus 4.6 (top-tier); the failure was in query construction, not model capability.
• The retrieval pipeline itself was sound: hybrid Grep + vector semantic search across Memory Markdown, session logs, and the file system.
• Both queries—"911 chart graph visualization" and "September 11th chart visualization Twin Towers attack"—missed. When he queried with the model code "992.2," the agent immediately found the March 29 chart.

Conclusion: retrieval success often hinges entirely on how the query is constructed; one wrong query defeats an otherwise complete retrieval chain.

Two core problems: representation and forgetting

Memory representation

Should memory be chat logs, fact triples, a knowledge graph, raw vectors, timestamp-event pairs, or model weights? Each answer yields a different system. After choosing a representation, three questions remain: how to do CRUD on it, how to extract memories without breaking conversation flow, and whether memory can transfer across models. Han argues memory should be decoupled from the upper model to enable migration.

Selective forgetting

Mapping the brain to AI: hippocampus ≈ RAG real-time retrieval, neocortex ≈ fine-tuning/pretraining, prefrontal cortex ≈ context window. But current agent systems are append-only with no forgetting mechanism. Without forgetting, memory gets buried in noise and the agent is dragged down by its own history. Three defects plague current systems: omission (the 911 case), distortion/hallucination, and hallucination inheritance. This creates a cold-start paradox: new agents delight users with low expectations, then degrade over time as unmanaged memory accumulates, eroding trust and retention.

Three schools of agent memory products

Classified by where the "source of truth" lives:

• Database school: vectors, SQL, key-value stores, knowledge graphs. Structured and efficient, but schema-constrained.
• File school: Markdown/plain-text files that the agent reads and writes back. Transparent, editable, versionable; but files bloat and need smart forgetting. Representatives: OpenClaw, MemSearch.
• Model school: truth stored in model weights/context; the model decides what to remember and forget. Zero-config and adaptive but a black box. Representatives: Letta, ChatGPT.

Consensus workflow across frameworks

1. Extraction: LLM extracts facts/memory structures from chat history. 2. Storage: converted to vector, graph, or temporal databases (temporal ordering supports time-based forgetting). 3. Hybrid retrieval: RRF fusion, MMR deduplication, time-decay ranking. 4. Conflict resolution and decay: how to handle contradictory memories and downweight old ones.

Case study: OpenClaw's file-based memory

• Truth sources: daily Markdown diary folders, plus memory.md (long-term user profile) and soul.md (agent's self-metadata).
• Derived indexes: SQLite FTS5 (BM25) and LanceDB (vectors), with RRF fusion, MMR dedup, and time-decay ranking.
• Exposes memory_search and memory_get interfaces.
• Simulates human sleep with deep/light/REM stages to control which memories get promoted to long-term storage—modeling forgetting.

The model school: giving truth to the model itself

Han personally favors the model school. Context windows keep growing: Qwen 3.5 MoE scales to 1M tokens running smoothly on a 24GB GPU; EverMind AI has pushed context to 100M tokens with Memory Sparse Attention (MSA). If all agent memory can be compressed into the context, complex layered designs may be unnecessary—possibly a "Bitter Lesson" moment where scale alone solves memory.

Benchmarks

• LongMemEval: 500 questions covering extraction, multi-session reasoning, knowledge updating, temporal reasoning, safety refusal.
• MABench (ICLR 2026): precise retrieval, test-time learning, long-horizon understanding, conflict resolution.
• MemoryArena: memory in multi-turn conversations.
• EverMemBench: first multi-party collaboration benchmark, 1M+ tokens.
• Tsinghua MemoryBench: first benchmark based on user-feedback continual learning.

Findings: longer contexts degrade memory quality; multi-session reasoning and conflict resolution remain very hard. EverMind AI raises a further question: what happens when an agent joins a group chat—whose information should it remember?

Future trends

1. No unified paradigm yet—like deep learning before 2017, with CNN/RNN/LSTM coexisting. 2. Pure-text approaches are "lying flat"—Grep-only solutions have an obvious ceiling. 3. Unlearning is overdue tech debt—systems are append-only; machine unlearning has lacked a good solution since Han studied it in 2009. A polluted recommender profile (e.g., a ruined Douyin feed) can never be reset, for the same reason. 4. Recommender systems will rise again—agent memory is fundamentally personalization, exactly what recommendation companies (Xiaohongshu, Douyin, Taobao) have done for two decades.

Bottom line: in 2026, building AI search is building the agent's memory system. Whoever cracks unified paradigms, forgetting mechanisms, and cross-model memory transfer will lead the AI 2.0 era.