You ask it: "What am I allergic to?" It answers: "Tree nuts." You then ask: "Give me a macaron recipe." It enthusiastically writes: almond flour, icing sugar, egg whites...
It didn't forget. Moments earlier, it answered correctly. But at the exact moment the memory mattered, it left that memory in the drawer.
This is not a bug in some early prototype. It comes from a real conversation documented in a July 2026 arXiv paper called Keep It InMind, where the authors tested GPT-5-mini against a set of mainstream memory systems. The paper names this failure mode the "implicit-association blind spot" and releases a 125-question expert-reviewed benchmark, InMind, designed to hold every long-term memory system up to the mirror.
1. Memory systems hide an unspoken assumption
Today's mainstream "long-term memory" solutions all look roughly the same: the user says something, the system distills it into a record and pushes it into a vector store; next time the user asks a question, the system retrieves the most similar records by similarity search and stuffs them into the prompt.
Behind this architecture lies an assumption so natural nobody says it out loud:
> "Memories that need to be used will resemble the question that retrieves them."
Sounds reasonable — until you hit world knowledge.
"I'm allergic to tree nuts" and "give me a macaron recipe" have almost zero lexical overlap in vector space, and their semantic similarity is pitifully low. Yet they are connected by a piece of world knowledge: the standard macaron recipe uses almond flour, and almonds are a major tree-nut allergen. That bridging knowledge doesn't live in the user's memory — it lives in the model's pre-trained weights.
Here's the problem: the retriever can't see this bridge. It only sees two texts — one about allergies, one about a recipe — with low similarity, so it doesn't recall. When generating the recipe, the model never sees the "allergy" memory, so almond flour goes right in.
2. The InMind benchmark: 125 tasks, 10 life domains
The team (USTC + Metastone) built 125 such tasks covering ten life domains: health, career, relationships, finance, personal development, beliefs, legal, consumption, parenting, and others. 113 of the tasks are backed by citable public sources. Each task is deliberately structured so that "a bridge is required between the memory and the query."
The key design is paired controls. InMind doesn't just ask "can the system get it right" — it separates three causes of failure:
1. The fact was never stored — lost at write time 2. The model lacks bridging knowledge — stored and retrieved, but the model doesn't know the two things are related 3. The fact was stored but not surfaced at the critical moment — stored, recallable, just absent when it should have appeared
Existing evaluations lump these three into a single accuracy number. InMind uses paired controls to pull them apart.
3. The verdict: clean and brutal
Experiment 1: put the memory directly in context
When the decisive memory is placed directly in the prompt (bypassing the retriever), GPT-5-mini answers 84.0% of the indirect questions correctly. This shows the model itself has the bridging knowledge — it knows almonds are tree nuts, and macarons use almond flour.
Experiment 2: let the memory system retrieve
When the same memory must pass through a retrieval system to reach the model, the best accuracy across six mainstream memory systems (vector, graph, and agentic types) is only 14.4%.
The gap: 84.0% → 14.4%. A cliff of nearly 70 percentage points.
Experiment 3: on-demand recall
Even more painful: if you directly ask "what is the user allergic to," these systems retrieve that memory 100% of the time. They stored it, they can recall it — it just doesn't show up when it should.
Experiment 4: swap in a stronger embedding
Switching to an embedding with 8x the dimensions improves "on-demand recall" on every system — but final answer accuracy barely moves. Better vectors cannot fill this pit.
4. Why "search harder" can't save it
The authors devote a section to arguing this is not a matter of search effort. You can push top-k from 5 to 50, swap the retriever for graph retrieval or agentic multi-turn retrieval — but as long as the interface remains "query comes in → retrieve → stuff into prompt," the implicit-association blind spot persists.
Because the problem isn't "can it be found" — it's "does it know it should look." While generating the macaron recipe, the model has no idea it needs to retrieve the "allergy" memory. From its perspective, the user just wants a recipe, which has nothing to do with allergies.
Minimal diagnostic probe: keep the memory always visible
The authors ran a simple controlled experiment: keep the decisive memory persistently visible before the query arrives (no retrieval trigger required). Most of the accuracy gap closes.
This localizes the problem precisely — the failure is not in storage, not in embeddings, not in model capability, but in the query-conditioned retrieval interface itself.
5. The real problem is routing
If retrieval isn't the answer, what is?
The authors point the open question toward routing: deciding, before the query arrives, which facts must remain visible. Not "retrieve whatever the user asks about," but "which facts belong to the category that is fatal once relevant and whose relevance is unpredictable — those should live permanently in context."
This resembles how a good human assistant works: a good assistant doesn't wait for you to mention "nut allergy" to remember it — they proactively bring that information to the forefront in any food-related scenario. Retrieval is passive; routing is proactive.
The InMind benchmark is designed to measure routing capability — it doesn't reward or punish "can it be recalled," but "did it appear when it should have."
6. Why this matters
The paper's power lies not in the numbers themselves but in dismantling something the entire agent community treats as a default assumption. Over the past two years, Mem0, A-Mem, HippoRAG 2, MemoryOS and a pile of memory systems have emerged, each competing on recall, embedding dimensions, graph structures. InMind's verdict: you are all optimizing a metric that isn't the bottleneck.
Raising recall from 80% to 100% is great — but if "appearing when it should" accuracy sits at 14%, no amount of recall saves the user.
Harsher still: the paper implies a structural ceiling. As long as the memory system's interface is "query-triggered retrieval," the implicit-association blind spot exists, independent of model capability or embedding dimensions. This isn't a tuning problem — it's an architecture problem.
Another entry for the evaluation blind-spot law: for years, memory system evaluations have tested "was it stored" and "can it be recalled" — nobody tested "did it appear when it should." InMind dug out this blind spot, and the numbers are 84.0% vs 14.4%.
7. An honest assessment
The paper's limitations are clear:
- 125 tasks is not many for ten domains — 12.5 per domain on average, so statistical power is weak in some domains
- Bridging knowledge depends on pre-training — if the model doesn't know "almonds are tree nuts," it fails even when the memory is placed directly in context. Part of the 84.0% (not 100%) comes from this
- Routing remains an "open problem" — the paper points the direction but offers no solution. That's honest, but it means InMind is more a diagnostic tool than a treatment
> Paper link: arXiv:2607.24368 > Code: the paper mentions a GitHub repository; no public link on the arXiv page yet > One-line summary: your memory system doesn't fail to remember — it fails to remember at the right time. The 84.0% vs 14.4% gap is the structural ceiling of the entire retrieval-based memory architecture.