Self-GC Deep Dive: Applying Java Garbage Collection Ideas to LLM Agent Context Management

This post analyzes Self-GC, a talk by Hao Xubin (AI Engineering Architect, Xiaohongshu) at AiCon 2026 (June 26-27, Shanghai), titled *Self-GC: A Multi-Turn Agent Context Governance Approach Combined with Prefix Cache Constraints*. Its one-line thesis: borrow Java GC ideas to treat multi-turn Agent session context as managed objects, combined with prefix-cache constraints, to achieve a 15%-20% net input TPM reduction.

Key points

1. The problem: long-horizon agents suffer 'memory leaks'

• Reported workload: average input ~70k tokens, average TPM ~100M tokens.
• Bottlenecks are shifting from single-step model capability to long-term stable operation under limited context, cache windows, and continuous tool interaction.
• With input:output ratios reaching 100:1, every generated token requires processing ~100 tokens of context.
• Existing work focuses on final compaction near the context limit; pre-compaction maintenance and coordination with prompt caches remain underexplored.

2. Core design: from garbage collection to context collection

Java GC maps onto agent context as follows:

| Java GC concept | Self-GC mapping | |---|---| | Objects | Context fragments (turns, tool calls, observations) | | Reference counting | Explicit addressing (which fragments are referenced later) | | Eden fast collection | Low-loss prune/mask/fold (lightweight compression) | | Survivor promotion | Important context moves to a retention layer | | Old Generation | Long-term memory / external storage beyond the window | | Full GC | Final compaction near the context limit | | Concurrent marking | plan/commit decoupling (mark first, execute later) |

Four mechanisms: 1. Explicit addressing — each fragment has a unique address; later turns reference instead of copying history (pointer instead of value semantics). 2. Low-loss prune/mask/fold — prune redundant fragments, mask them as read-only summaries, or fold related fragments into structured summaries; selective rather than blunt truncation. 3. Plan/commit decoupling — evaluate compression candidates and expected information loss first; commit only when quality checks pass, allowing deferral of frequently-referenced fragments. 4. Cache-aware delayed commit — compression is deferred to prefix-cache invalidation windows (TTL expiry, new sessions); no prefix-changing operations inside cache-hit hot zones.

3. Why not 'compress when full'?

Traditional threshold-based compaction causes latency spikes, concentrated information loss, prefix-cache invalidation, and indiscriminate compression. Self-GC instead provides continuous, fragment-granular, cache-friendly, incrementally-executed governance.

4. Industry positioning

• Claude Code: 92% threshold + 8-section structured summary; no cache awareness.
• Gemini CLI: 70% threshold + 5-section summary + filesystem persistence; still reactive.
• Manus: context state machine + masking instead of deletion; tool-management focused.
• OpenClaw /compact: user/system-triggered; no runtime governance.
• Mem0 / MemOS: cross-session memory, not within-session growth.
• LLMLingua: token-level prompt compression, no agent runtime awareness.

Self-GC's differentiators: runtime continuous governance, cache co-design, model-agnostic harness capability, and reversible/evaluable compression.

5. Quantified benefits (post's estimate)

• Baseline: 70k input/round, 100:1 ratio; 10 rounds → 700k cumulative input tokens; at $0.01/1k input tokens → $7.0 per session.
• With 15-20% reduction: ~560k-595k tokens → $5.6, saving ~$1.4 per 10-round session.
• At 100M TPM (~1,428 rounds/min at 70k each): saves ~1.5M-2.0M tokens/min ≈ $15-$20/min ≈ $788k-$1.05M annualized (24×7), excluding additional prefix-cache savings (cache reads typically ~10% of normal cost).

6. Toward a 'Context OS'

The author frames context management evolution in four stages: (1) unmanaged stacking, (2) passive threshold compaction (current mainstream), (3) runtime governance (Self-GC), and (4) a future Context Operating System featuring virtual memory (spill to vector DBs/file systems), page-based management, reference counting + GC, cache hierarchies (L1 window / L2 recent cache / L3 external), inter-agent isolation, and snapshot/rollback.

7. One-line takeaway

> Self-GC's essence: turn 'dump everything when full' into 'tidy as you go' — and do heavy reorganization only during cache-invalidation gaps, when the prefix cache must be rebuilt anyway.

Reference information

• Talk: AiCon 2026, Hao Xubin, *Self-GC: A Multi-Turn Agent Context Governance Approach Combined with Prefix Cache Constraints*.
• Technical background: Java GC (generational, incremental, concurrent), vLLM Prefix Caching (Radix Tree), Claude Code 8-section summarization, Gemini CLI 70/30 strategy.
• Industry practice: Manus KV-cache hit-rate optimization, OpenClaw context compaction, Mem0 cross-session memory.