MetaClaw: When AI Agents Learn to Get Smarter With Use
*Subtitle: A framework that lets your AI assistant automatically evolve during conversations*
Introduction: A Strange Phenomenon
Imagine hiring an assistant. On day one, they help you process files, reply to emails, and manage your schedule. You tell them: "From now on, always back up a file before modifying it." They nod and remember.
But three months later, you find them making the same mistake — never backing up before edits. You ask: "Didn't I tell you about this?" They look blank: "Did you? I don't remember."
That sounds absurd — but it's exactly how most AI agents work today. They are trained once, then frozen in time. No matter how long you chat with them or how many lessons you teach, their "brain" (model weights) never changes.
MetaClaw aims to change exactly this.
The Core Problem: Static Models in Dynamic Deployment
Imagine running a platform like OpenClaw — a personal AI assistant connected to 20+ messaging channels. Users ask questions on Feishu, send files via Telegram, and discuss projects on Discord. Task distributions drift daily, but the model stays frozen.
This creates a fundamental tension:
> The AI must serve users continuously (no downtime), yet its capabilities keep becoming outdated (needs updating).
Existing approaches all have drawbacks:
- Memory methods: Store conversation logs for later retrieval. But raw logs are verbose and redundant; the AI can't distill transferable behavioral patterns from them.
- Skill libraries: Compress experience into reusable instructions, but these libraries are static and never coordinated with model weight optimization.
- Reinforcement learning: Can update weights, but requires training downtime, and once skills evolve, old data "pollutes" the new model (punishing the model with failures from outdated skills).
- "Create a .bak backup before modifying any file"
- "All timestamps must use ISO 8601 format with timezone offsets"
- "Name files with date prefixes, e.g., 20260408_*.json" 4. Immediate effect: New skills are written to the skill library and take effect instantly — no retraining, no downtime. The very next conversation benefits.
- As a meta-parameter: an accumulated behavioral knowledge base that grows with each skill evolution.
- As an adaptation basis: at inference time, the most relevant skills are retrieved and injected into the system prompt for instant specialization.
- Day 1: The AI doesn't know to back up files, deletes something important. The failure is recorded.
- Day 2: The skill evolver distills "back up before modifying." The library updates.
- Day 3: The AI backs up files; the error no longer occurs.
- Every collected trajectory is tagged with a skill-generation version *g*.
- Support Data: failure trajectories collected under version *g*, used to drive skill evolution to *g+1*. They are consumed and discarded — never entering the RL buffer.
- Query Data: trajectories collected under version *g+1* (with new skills applied). Only these enter the RL buffer.
- Part I: 30 workdays with increasing task difficulty, focused on end-to-end execution reliability.
- Part II: 14 workdays focused on behavioral rule compliance (format specs, naming conventions).
- Skills alone significantly boost accuracy (up to +32%) but don't unlock end-to-end task completion.
- The full pipeline (skills + RL) truly breaks through: task completion improves 8.25x (from 2.0% to 16.5%).
- Kimi-K2.5 + MetaClaw (40.6%) nearly matches the GPT-5.2 baseline (41.1%) — showing that continual learning can bring a weaker model up to frontier-model performance.
- Stage retry rate reduced by 24.8%
- Refinement loops reduced by 40%
- Composite robustness score improved by 18.3%
- Paper: MetaClaw: Just Talk — An Agent That Meta-Learns and Evolves in the Wild
- Code: https://github.com/aiming-lab/MetaClaw
- Team: UNC-Chapel Hill, CMU, UC Santa Cruz, UC Berkeley
- Lead authors: Peng Xia, Jianwen Chen, Xinyu Yang, Huaxiu Yao, et al.
MetaClaw's core insight: adaptation should happen on two different timescales — and they should reinforce each other.
Layer 1: Skill-Driven Fast Adaptation
Like learning to ride a bike — after your first fall, your brain immediately derives a rule: "lean inward when turning." No neuron rewiring needed, just a trick you can apply right away.
MetaClaw's skill-driven fast adaptation works like this:
1. During conversations: MetaClaw acts as a proxy layer, intercepting every interaction between the AI assistant and the user. 2. Failure detection: Failed trajectories (e.g., deleting the wrong file, malformed JSON) are recorded. 3. Skill distillation: A dedicated "skill evolver" (another LLM) analyzes failures and extracts concise behavioral instructions, such as:
The skill library plays two roles:
Crucially, natural-language instructions are inherently transferable across tasks. A skill distilled from a file-operation failure ("back up before modifying") automatically applies to all file-related tasks.
Layer 2: Opportunistic Policy Optimization
Skills alone hit a ceiling: they don't change the underlying model weights. Like a student memorizing problem-solving tricks without understanding the math, novel problems still cause failure. That's where weight-level optimization comes in.
MetaClaw uses reinforcement learning (RL) + a process reward model (PRM) to fine-tune weights. But weight updates require "hot-swapping" the model, causing minutes of downtime — unacceptable in production.
The Opportunistic Meta-Learning Scheduler (OMLS)
MetaClaw's solution: train only when the user is away. OMLS monitors three idle signals:
1. Sleep windows: user-configured sleep times (e.g., 23:00–07:00). 2. System inactivity: no keyboard/mouse activity for 30+ minutes. 3. Calendar awareness: detects meetings via the Google Calendar API.
The training window opens when any signal indicates the user is away, and pauses when any signal indicates they're back. RL training supports pause/resume across fragmented idle windows — gradient steps can accumulate over several nights of sleep.
The Key Challenge: Preventing "Stale Reward Pollution"
Consider this scenario:
But when training with RL, you use Day-1 failure data (collected under old skills). The weights get optimized to avoid an error that skills have already fixed — like a swimming coach still punishing you for drowning after you've learned to swim. Such stale reward signals pollute gradient updates.
MetaClaw's Solution: Skill-Generation Versioning
When skills evolve from g to g+1, all samples with version ≤ g are purged, ensuring policy optimization is always based on behavior under current skills, not already-fixed old behavior.
Experimental Results: From 21.4% to 40.6%
MetaClaw-Bench
A continual-evaluation benchmark with 934 questions across 44 simulated workdays:
Results (Kimi-K2.5):
| Condition | Part I Acc. | Part II Acc. | File-Check Completion | |-----------|------------|-------------|----------------------| | Baseline | 21.4% | 21.1% | 18.2% | | +Skills | 28.3% (+32.2%) | 26.9% (+27.5%) | 33.8% | | +Skills+RL | 40.6% | 39.6% | 51.9% (+185%) |
Key findings:
AutoResearchClaw
A 23-stage autonomous research pipeline (literature search → hypothesis generation → experiment design → code synthesis → paper writing). With skill adaptation only (no RL):
This shows MetaClaw's skill mechanism transfers across domains, working even for completely different workloads (CLI tasks vs. autonomous research).
Architecture: The Proxy-Layer Design
1. MetaClaw runs as a lightweight proxy layer between your AI assistant (OpenClaw, CoPaw, IronClaw, etc.) and the LLM API. 2. Each conversation: relevant skills are retrieved and injected into the system prompt. 3. After each conversation: new skills are automatically summarized. 4. In the background: OMLS triggers RL training during idle periods. 5. After training: model weights are hot-swapped.
No local GPU required — RL training is done via cloud LoRA services (Tinker, MinT, Weaver).
Three Operating Modes
| Mode | Skills | RL | Scheduler | Use Case |
|------|--------|----|-----------|----------|
| skills_only | ✅ | ❌ | ❌ | Lightweight deployment, no GPU/cloud training |
| rl | ✅ | ✅ | ❌ | Immediate training, with downtime |
| madmax (default) | ✅ | ✅ | ✅ | Production, zero downtime |
Core Insights
1. Two-timescale adaptation: second-level skill evolution (gradient-free) + hour-level weight optimization (gradient-based). 2. Mutual reinforcement: better policies produce more informative failures for skill synthesis; richer skills produce higher-reward trajectories for policy optimization. 3. Opportunistic training: exploit user idle time for weight updates — zero downtime. 4. Version isolation: strictly separate "old data used to learn skills" from "current data used to train weights," preventing pollution.
Imagining the Future
Imagine a MetaClaw-powered personal AI assistant. In week one, it occasionally makes file-operation mistakes — but after each one it quietly notes: "Ah, that's how backups should be done." A month later, it rarely repeats mistakes. You don't even notice it evolving, because it always "learns" while you sleep, attend meetings, or step away from your computer. A year later, it has become your own personal expert — mastering not just general capabilities but your unique work habits, project structure, and naming preferences.
This isn't science fiction. It's the vision MetaClaw aims to realize.