LLM-based Medical Assistant Personalization with Short- and Long-Term Memory Coordination (NAACL 2024)
Overview
This forum entry indexes a NAACL 2024 paper on LLM-based medical assistant personalization with coordination between short-term and long-term memory.
- Venue: NAACL 2024
- Link: https://aclanthology.org/2024.naacl-long.132/
- Category: Personalization
- Short-term memory — the current conversational context and immediate user intent;
- Long-term memory — persistent user history, preferences, and domain knowledge.
- A Survey of Personalization: From RAG to Agent (arXiv 2504.10147)
- Bridging Personalization and Control in Scientific Personalized Search (arXiv 2411.02790)
- IntentRec: Predicting User Session Intent (arXiv 2408.05353)
- Unified Embedding-Based Personalized Retrieval in Etsy Search (arXiv 2306.04833)
- User-LLM: Efficient LLM Contextualization with User Embeddings (WWW)
Research Context
The paper addresses a core question for LLM-era search and personalization systems: how to reallocate responsibilities among retrieval, ranking, generation, and tool invocation. Personalized medical assistance requires both:
Traditional pipelines that treat retrieval, ranking, and generation as separate stages struggle to meet user expectations for natural-language interaction and multi-hop reasoning, motivating a memory-coordinated personalization approach.
Method Structure
Following the typical four-stage pattern of such systems:
1. Input & representation — encoding queries, documents, and user context into dense/sparse representations or structured prompts; 2. Core modules — retriever, reranker, planner, memory modules, and tool interfaces, arranged serially or in parallel; 3. Learning strategy — supervised fine-tuning, contrastive learning, distillation, or reinforcement learning; 4. Inference strategy — single-turn vs. iterative retrieval, early stopping, and compute budget control.
Evaluation & Engineering Considerations
Typical evaluation covers nDCG@10, MRR, Recall@k, human preference, and task success rates. For deployment in medical settings, key engineering checklist items include:
| Concern | Recommendation | |---------|----------------| | Data | PII partitioning, sanitization, versioned embeddings | | Latency | Cascaded retrieval with early stopping, caching | | Safety | Source whitelisting, hallucination filtering | | Cost | Model routing, distillation, hybrid sparse+dense retrieval |
Insights for Personalization
1. Architecture: cascaded retrieval + reranking + generation remains mainstream, but agentic paradigms make retrieval policy itself learnable; 2. Data: high-quality instruction data and session logs are as critical as model architecture; 3. Evaluation: the gap between offline metrics and online satisfaction is widening; LLM-as-judge needs human cross-validation; 4. Product: latency, cost, explainability, and safety are hard constraints in medical deployment.
Related Entries
Glossary
| Term | Meaning | |------|---------| | IR | Information Retrieval | | RAG | Retrieval-Augmented Generation | | nDCG | Normalized Discounted Cumulative Gain | | Agentic Search | Modeling search as sequential decision-making and tool invocation | | Gen-IR | Generative Information Retrieval |