Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols, and Harness Engineering

Forum topic · ✨步子哥 · 2026-07-25

Summary

A 54-page survey (arXiv:2604.08224) by researchers from Shanghai Jiao Tong University, Sun Yat-sen University, CMU, and OPPO proposes 'externalization' as a unifying principle for LLM agent design: shifting cognitive load from model weights to persistent, inspectable, reusable external structures. Drawing on Donald Norman's cognitive artifacts theory, the paper argues agents progress through three stages—Weights, Context, and Harness—and identifies three externalization dimensions: memory (recall becomes recognition), skills (generation becomes composition, e.g., Voyager, Reflexion, Agent Skills), and protocols (ad-hoc becomes structured, e.g., MCP, A2A, Function Calling). Harness engineering—orchestration, sandboxing, human approval, observability—is positioned as the runtime layer hosting these forms. The review covers trade-offs between parametric and externalized capability, embodied 'cerebrum-cerebellum split' architectures like RoboOS, and open challenges in evaluation, governance, and co-evolution of models with harness infrastructure.

Overview

This post is an in-depth Chinese-language analysis of the survey paper Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering (arXiv:2604.08224), authored by Chenyu Zhou and 20 co-authors from Shanghai Jiao Tong University, Sun Yat-sen University, Shanghai Innovation Institute, CMU, and OPPO. Submitted 2026-04-09, 54 pages. DOI: https://doi.org/10.48550/arXiv.2604.08224

Core Thesis

Connecting a stronger base model does not guarantee qualitative agent improvement, whereas adding persistent memory, reusable skills, and standardized tool interfaces often does. The paper attributes this to externalization: moving cognitive burden from a model's internal computation to persistent, inspectable, reusable external structures. Borrowing Donald Norman's notion of *cognitive artifacts*, it argues external tools do not enlarge the brain—they reshape the task itself (a shopping list turns recall into recognition; a map turns implicit spatial relations into visible structure).

> Reliable agent capability comes not merely from bigger models, but from systematic restructuring of task requirements so that internal capability and external infrastructure jointly cover the needed competencies.

Three Stages of Capability Migration

| Stage | Period | Carrier | Core Mechanism | Fundamental Limitation | |---|---|---|---|---| | Weights | 2022–23 | Model parameters | Pretraining compresses knowledge; scaling laws dominate | Costly updates, hard to audit/personalize/compose | | Context | 2023–24 | Prompts / context window | Prompting, CoT, RAG release knowledge at runtime | Limited window, lost-in-the-middle, per-session amnesia, fragile assembly | | Harness | 2024–present | Persistent external infrastructure | Memory, tool registries, protocols, sandboxes, orchestration | Cost, security risk, governance load |

The shift: from *what the model knows* → *what it can invoke* → *how it reliably completes tasks*.

Three Externalization Dimensions

1. Memory: recall → recognition

Four layers of externalized state: working context, episodic experience, semantic knowledge, and personalized memory. An architectural spectrum:

Monolithic context — everything in the prompt; easy prototyping, state dies with the session
Retrieval-backed context — long trajectories stored externally, retrieved on demand (GraphRAG, ENGRAM, SYNAPSE)
Hierarchical memory — OS-style decoupling (MemGPT/Letta, MemoryOS; Mem0, MemoryBank, MIRIX, MemOS, xMemory with explicit extract–consolidate–forget lifecycles)
Adaptive memory — dynamic modules with feedback-driven policy optimization (MemEvolve, MemVerse, MemRL, GAM), described as a shift *from storage to control*

Also highlighted: Generative Agents (memory stream + reflection), ReadAgent, A-MEM, Zep/Graphiti.

2. Skills: generation → composition

Voyager

Reflexion

Agent Workflow Memory (AWM)

EXTRACT

Anthropic Agent Skills / SKILL.md

3. Protocols: ad-hoc → structured

MCP

A2A

Function Calling

AG-UI

Harness: The Runtime Layer

The harness is not a fourth externalization but the engineering layer hosting the other three: agent loops, sandboxing, human approval gates, observability, configuration/policy, context budgeting. It plays two complementary roles:

Accelerator (cognitive flywheel): memory → skills → protocols → better execution → richer memory
Brake (safety): Claude Code's OS-level seatbelt/bubblewrap isolation and classifier-based approvals; OpenAI Operator/CUA's mandatory confirmation for sensitive actions. Principle: cap the blast radius before resorting to per-action approvals.

Representative systems: Claude Code / Agent SDK, Cursor, OpenAI Agents SDK, LangGraph, AutoGen/AG2, CrewAI.

Trade-offs and the Over-Externalization Trap

Paper §7.3 lists trade-off axes: update frequency/decay, reusability/portability, auditability/governability, latency/context burden. Community-raised risks:

Models degenerating into empty shells that only call tools, with degraded zero-shot generalization
A reverse internalization route (e.g., Skill0.5's 'internalize cognition, externalize skills'); Lilian Weng's concern that part of the harness will be 'eaten' by stronger models
Security risks: memory poisoning, malicious skill injection, protocol spoofing; positive-feedback error amplification
Evaluation vacuum: no quantitative metric for 'degree of externalization' or causal attribution per component

Embodied Externalization: Cerebrum–Cerebellum Split

Extending to physical agents: offload real-time motor control from the model to external structures. Examples: RoboOS (arXiv:2505.03673) — cloud Embodied Brain (RoboBrain MLLM) for perception/planning plus an edge Cerebellum Skill Library and Real-Time Shared Memory; Beihang's AeroAgent (agent as cerebrum, controller as cerebellum); CAICT's brain/cerebellum/body framework; SaiVLA-0 (frozen VLM + parallel decoders).

Outlook and Takeaways

Future directions: self-evolving harnesses and shared agent infrastructure (cross-vendor protocol ecosystems, organizational 'Agent OS' with governed shared memory, public-sector transparent harnesses). Open challenges include standardized system/task-level benchmarks and ablation protocols, harness-layer security certification, memory privacy (differential privacy, right to be forgotten), protocol identity/authn/audit, and co-evolution of models with externalization-aware pretraining.

The author's closing verdict: better agents are not better reasoners but better-organized cognitive systems. Weights → Context → Harness is layered, not mutually exclusive; the harness is currently the highest-leverage engineering layer to bet on; and embodied agents are externalization's ultimate proving ground.