MemCollab: Teaching AI Agents to Share Memory Across Thinking Boundaries

This post introduces MemCollab: Cross-Agent Memory Collaboration via Contrastive Trajectory Distillation (Chang, Wu, Wu, & Lin, 2026, arXiv preprint arXiv:2603.23234), explaining why directly sharing memory between AI agents fails and how contrastive distillation fixes it.

The problem: memory is locked in silos

AI agents accumulate memories from solving problems—techniques, mistakes, tool usage tips—but these memories stay isolated per model. This causes:

• Redundant work: ten agents solving the same problem type each explore from scratch
• Capability boundaries: memory from a 7B model may not help a 32B model, and vice versa
• Resource waste: mixed-size deployments (small models for cheap tasks, large for hard ones) cannot pool experience

Why naive memory sharing backfires

Directly copying memories between agents *hurts* performance:

| Task | Baseline (7B) | With copied memory | |------|---------------|--------------------| | MATH500 | 52.2% | 50.6% | | HumanEval | 42.7% | 34.1% |

The root cause is knowledge entanglement: raw memories mix transferable principles (correct reasoning paths, constraint handling, error recognition) with model-specific biases (individual reasoning styles, tool habits, heuristic shortcuts). Transferring the whole package drowns the signal in noise—like a Cantonese chef blindly following a Sichuan master's stir-fry notes.

The core idea: contrastive trajectory distillation

MemCollab pairs a weaker agent (Aw, e.g., 7B) and a stronger agent (As, e.g., 32B) on each task. A verifier labels trajectories as preferred (τ+) or dispreferred (τ−)—either model can succeed. By contrasting trajectories on the *same* problem, differences reveal what is model-specific bias versus task-essential structure.

Formally, a trajectory decomposes as τ = g(s, b), where s is task structure and b is agent bias. Since s is shared across agents but b differs, contrast isolates s, yielding memories m = φ(s).

Distilled memories take the form of rules, not examples:

• Violation patterns: systematic errors observed in failures (e.g., "doing numerical computation before establishing the full equation", "assuming event independence without verification", "missing boundary cases during enumeration")
• Reasoning invariants: principles kept in successes and broken in failures (e.g., "map variable dependencies first", "model joint probability via conditional probability", "systematically enumerate all relevant cases")

Task-aware retrieval

A two-stage retrieval mechanism:

1. Task classification: categorize the new problem (algebra vs. geometry, probability vs. number theory) 2. Relevance ranking: retrieve only the top-3 memories within that category

Retrieval shows an inverted-U curve: performance rises with 1–3 memories, then degrades beyond 3, since excessive constraints add noise rather than prune the search space.

Experimental results

MATH500:

| Model | No memory | MemCollab | Gain | |-------|-----------|-----------|------| | Qwen2.5-7B | 52.2% | 67.0% | +14.8% | | Qwen2.5-32B | 68.0% | 73.8% | +5.8% |

MBPP (code generation):

| Model | No memory | MemCollab | Gain | |-------|-----------|-----------|------| | Qwen2.5-7B | 47.9% | 57.6% | +9.7% | | Qwen2.5-32B | 59.4% | 64.3% | +4.9% |

Cross-family transfer (Qwen ↔ LLaMA): LLaMA3-8B improves from 46.6% to 74.4% (+27.8%), showing the distilled knowledge generalizes across architectures.

Additional findings:

• Cross-model contrast beats self-contrast: comparing one model's own successes and failures helps less, because a single model cannot escape its own biases—divergent perspectives are what filter out model-specific habits
• Efficiency gains: fewer reasoning turns are needed, as models avoid known dead-end paths
• A 7B model with MemCollab approaches or exceeds a memory-less 32B model

Why it works

• Contrastive learning: comparing positive and negative samples on the same task isolates discriminative, transferable features
• Information theory: memory distillation is information purification—maximizing I(task structure) while minimizing I(model bias) and noise, like distilling spirits
• Cognitive science: mirrors how human experts use structured mental models—classifying problems, recalling core concepts and common pitfalls—rather than memorizing specific solutions

Broader implications

• From isolated to collaborative AI: a shared, evolving knowledge base across agents, analogous to how science builds consensus through peer review
• Small models get a leg up: cheaper deployment, edge-device AI, lower adoption barriers
• Multi-agent systems: teams of specialized models sharing collective memory
• Continual learning: new techniques and error patterns accumulate in shared memory over time
• Future directions: scaling to many-agent communities, meta-learning (learning how to learn), self-evolving memory, and cross-modal extension to vision, robotics, and scientific discovery

Reference

Chang, Y., Wu, Y., Wu, Q., & Lin, L. (2026). MemCollab: Cross-Agent Memory Collaboration via Contrastive Trajectory Distillation. *arXiv preprint arXiv:2603.23234*.

*The original post was written in a Feynman-style expository voice, using analogies (a city building a public library, two chefs' differing stir-fry styles, a teacher comparing two students' homework) to make the research accessible to non-technical readers.*