When AI Memory Runs Out: Predictive Sufficiency and the Science of LLM Session Handover

This post is a Chinese-language deep-dive commentary on the paper Handover of In-Context Learning State Across Session Boundaries (Masahiro Kato & Taka Kato, arXiv:2608.14528, AI/ML theory).

Key points

• The problem: LLMs have hard context-window limits. When tasks outlive one session, a "session handover" must occur — but unlike prompt compression, the handover record must be written *before* future queries are known. It must support the full set of possible later inputs, not a single anticipated query.
• Formalization: The paper models handover as transferring the task-relevant in-context learning (ICL) state. A record is *predictively sufficient* when, given the record, the new session's predictive distribution matches that of a session with the complete history.

Main theoretical results

• Under an exogeneity condition, predictive equivalence classes characterize the coarsest deterministic sufficient handover, yielding a direct fixed-length bit requirement — the needed memory depends on task complexity, not history length.
• The paper decomposes handover information loss into three parts:
1. Capacity loss from the memory budget (physical constraint); 2. Loss attributable to the chosen writer (strategy choice); 3. Gap between the continuation procedure and the ideal decoder (execution).
• Under log loss, discarded task information equals exactly a conditional mutual information between history and target given the handover record.

Practical method: a three-part record

1. Exact storage — all irreversible decisions and constraints (e.g., excluded options, budgets, deadlines). 2. Task-justified statistics — sufficient statistics for repeated observations/tools results, kept only when error guarantees hold for the downstream task. 3. Residual observations — raw retention for special observations that summaries cannot adequately replace.

Worked examples and bounds

• Gaussian linear regression: an exact finite-dimensional sufficient record exists, expressible as a sufficient statistic pair \((G_n, b_n)\) or equivalent synthetic sufficient demonstrations, with finite-bit perturbation bounds on posterior predictive error under quantization.
• Nonparametric settings: achievable upper bounds and necessary lower bounds connect memory size to squared prediction error.

Discussion highlights from the review

• Handover record vs. KV cache: a KV cache is a computational trace, not a task record — it contains no goals, constraints, or decisions.
• Handover record vs. RAG: retrieval mechanisms can *carry* handover information but do not determine whether retained content is sufficient for continuation.
• Philosophical angle: task identity across sessions is functional (predictive equivalence), not historical — a pragmatic answer to a Ship-of-Theseus question for AI tasks.

Open questions

• Quantifying and reducing the writer's foresight cost of writing records before seeing later queries; can adaptive handover mechanisms help?
• Multi-agent handover: whether information loss accumulates across chained handovers or can be reset.
• Connections to continual learning: does effective session handover imply or require continual-learning capability?

Reference

Kato, M., & Kato, T. *Handover of In-Context Learning State Across Session Boundaries*. arXiv:2608.14528.