Not All Experience Belongs in the Weights: A Routing Rule for Self-Improving GUI Agents
A Counterintuitive Experimental Result
Suppose you are training a mobile-operation agent. It has run a batch of tasks and collected a set of trajectories—both successful and failed. Now you want it to learn from this experience, and there are two paths:
Path one: fine-tuning—write the experience into the model weights so it "internalizes" the knowledge.
Path two: retrieval—store the experience in a notebook, and inject relevant notes into the prompt at execution time.
Which would you choose?
If you have followed the literature of the past two years, you know both paths have advocates, and their conclusions contradict each other. Some say fine-tuning wins, some say retrieval wins, and some say the same skill representation helps on one task and hurts on another.
A recent paper (arXiv: 2610.01787) offers a striking answer: the question itself is wrong.
Because "one trajectory" is not an indivisible unit. A trajectory mixes at least four entirely different kinds of things, each with a different proper destination. Stuffing an entire trajectory into one destination is like putting clothes, dishes, books, and tools into the same cabinet—it fits, but retrieval becomes chaos.
Four Kinds of Experience, Four Destinies
The paper decomposes agent experience into four components:
1. Locators — "where is a particular button on the screen"
Example: on the Weather app's home page, the tap coordinates for the "City Management" button are (770, 84).
This is a static mapping: app + element name → position. It barely depends on the current state—the button's location does not change whether you are adding or deleting a city.
2. Procedures — "after doing A, do B, then C"
Example: in the Weather app, after tapping "City Management" → selecting "Shenzhen" → the next step is usually "e.g., General".
This is an action sequence: highly dependent on which task and screen you are on. In a different task, the same three-step sequence may be entirely wrong.
3. State Facts — "before performing an action, some condition must hold"
Example: before submitting an answer, the answer field must already contain the value the task requires.
This is a precondition: meaningful only in a specific state. If the current state does not match, the fact is useless.
4. Lessons — "in this state, doing A fails; do B instead"
Example: in a certain state on the answer page, don't tap "e.g., General"—tap "Submit Answer" instead.
This is a preference correction: it records "in a specific state, which action is better".
Two Properties Decide Everything
The paper's core insight: where each component belongs is determined by two properties measurable before training.
Property 1: Recurrence — how often does this component's key appear across unrelated tasks?
A locator's key is "app + role + element name". The key "Weather app's City Management button" appears in completely unrelated tasks—adding cities, deleting cities, checking weather. High recurrence.
A procedure's key is "three consecutive action goals". A given three-step sequence typically appears only within a specific task family. Low recurrence.
Property 2: State-conditionality — how much does the component's usefulness depend on the current state matching the source state?
Locators barely depend on state—a button's position does not change when you switch tasks. Low state-conditionality.
Procedures depend heavily on state—the same three steps become inapplicable under a different task. High state-conditionality.
Plotting the four components on this two-dimensional plane, they fall neatly into four quadrants:
| Component | Recurrence | State-conditionality | Best destination | |-----------|-----------|---------------------|------------------| | Locator | High (0.39) | Low (0.10) | Weights | | Lesson | High (0.45) | Medium (0.53) | Weights | | State fact | High (0.32) | High (0.56) | Context | | Procedure | Low (0.05) | High (0.66) | Context |
The pattern is extremely clean: high recurrence + low state-conditionality → weights; low recurrence + high state-conditionality → context.
Why Does This Work?
An analogy. Imagine starting a job at a new company. Two kinds of things to learn:
First kind: "company common knowledge"—where the pantry is, how the printer works, the Wi-Fi password. Used daily, everywhere, independent of your current project. You should memorize these (write into weights), because looking them up every time is too slow.
Second kind: "project-specific procedures"—a project's deployment steps, a client's special requirements. Useful only in specific contexts; useless in another project. You should write these on sticky notes on your monitor (keep in context), because they are too context-dependent—memorizing them would actually interfere with judgment.
This is the paper's core mechanism:
Weights excel at absorbing high-recurrence, low-state-conditioned information—weights are global and state-independent. A button's position is the same in every task; once in the weights, it is always available. And high recurrence means the same information is reinforced repeatedly during training, so weights learn it best.
Context excels at carrying highly state-conditioned information—context can be retrieved by the current state. You don't memorize "in state A do X, in state B do Y"; you retrieve the right note when you encounter state A. The stronger the state-conditionality, the more precise matching is needed—and retrieval can match precisely, while weights cannot.
An Elegant Validation: Note Uptake Rate
The paper designs a clever probe experiment.
Before training, they measured "given one note, how likely is the agent to change its decision"? Result: 42%—notes are genuinely useful.
Then they wrote the same components into the weights and re-measured:
- Locators' note uptake dropped from 48% to 16%—the weights had absorbed this information; the notes became redundant
- Lessons' uptake also dropped sharply—likewise absorbed
- But procedures' uptake only fell from 33% to 22%—the weights could not fully absorb procedures; notes remain valuable
- Information gap: what the producer knows but the consumer doesn't. The larger this gap, the more useful the context route—notes fill information gaps.
- Policy gap: how differently the two models act in the same state. The larger this gap, the more harmful the weights route—fine-tuning makes the consumer learn a policy that isn't its own.
- Routing vs everything-in-context: +4.4 points on average (MobileGym), +4.1 (AndroidWorld)
- Routing vs everything-in-weights: +7.8 points on average
- Routing vs the best whole-trajectory baseline (three retries): +9.3 points
- Routing vs inverted assignment (deliberately putting weights-bound components into context and vice versa): +8.9 points
Moreover, the drop in uptake correlates strongly with recurrence (Spearman -0.92). The higher a component's recurrence, the more redundant its notes after weight injection; the lower the recurrence, the more notes still help.
It's like commuting: walk the same route home for a month and you no longer need GPS (high recurrence → absorbed into weights); but for an occasional trip somewhere unfamiliar, you still need navigation every time (low recurrence → keep in context).
Who Produces Experience, Who Consumes It
Another important finding: notes transfer across models; fine-tuning does not.
Procedure notes produced by an 8B model still benefit a 32B model (+3.2 points). But fine-tuning the 32B model on the 8B model's trajectories is harmful (-1.4 points).
Why? Notes carry information ("in this state, do this"), while fine-tuning carries a policy ("when facing similar situations, tend to do this"). Different models have different policies; forcibly injecting one model's policy into another causes "policy mismatch".
More precisely, the paper distinguishes two gaps:
A larger same-family model (4B→8B→32B) mainly widens the information gap, not the policy gap, so its notes help while its fine-tuning harms. A cross-family model (GUI-Owl vs Qwen3-VL) introduces a policy gap too: notes still help (information is universal), but fine-tuning becomes seriously harmful.
Practical implication: if you want a stronger model to help your smaller model, give the strong model's notes to the small model (context route), don't fine-tune directly. If you do fine-tune, use the small model's own successful trajectories—the paper shows "own practice" and "learning from others' experience" are nearly equivalent in the weights route (gap < 1.1 points).
What Happens When You Upgrade the Model?
A corollary: when you upgrade the model, the notebook survives; the weights do not.
Upgrading from 8B to 32B: 8B's notes still give +3.2 points to the 32B model, but 8B's fine-tuned weights give -1.4 points to the 32B model.
Notes carry information (model-agnostic); weights carry policy (strongly model-bound). Change the model and the policy becomes invalid, but the information remains valid.
It's like an employee leaving: the documents they wrote can be handed to the successor, but the muscle memory they trained cannot be transferred. The new employee needs to re-practice—but not to re-collect information.
Real-World Performance of the Routing Rule
Condensing the findings into one rule—use recurrence and state-conditionality to predict each component's destination—and testing on unseen backbone models:
24/24 cells predicted correctly.
More concrete numbers:
Two Intervention Experiments
The paper also runs two clever interventions to establish causality:
Intervention 1: increase training dosage. Duplicate procedure components 8× before fine-tuning (simulating high recurrence): the routing gain for procedures moves from -4.2 to -0.8, shifting toward the weights direction. But 8 copies still don't cross the boundary—confirming the boundary is not arbitrary; it corresponds to a real recurrence threshold.
Intervention 2: remove state conditioning. Change state-fact and procedure retrieval to "match by task text only, ignore state": performance drops by 3.4 and 3.9 points respectively. This proves the context route's advantage comes precisely from state matching—not the note content itself, but the mechanism of "retrieving the right note in the right state".
Engineering Takeaways
Direct practical guidance for practitioners:
1. Don't treat experience as a monolith. In self-improvement loops, decomposing trajectories into components and routing them beats stuffing the whole trajectory into one destination. The paper's decomposition targets GUI agents, but the idea generalizes: any self-improving system should ask—which of my experiences are "common knowledge" (high recurrence, low state dependence) and which are "situational knowledge" (low recurrence, high state dependence)?
2. Context and weights are not competitors—they are a division of labor. Past research debated "fine-tuning vs retrieval", but that debate's premise was wrong. The right question is: "where does *this specific piece* of information belong?"—and the answer depends on recurrence and state-conditionality.
3. For cross-model collaboration, transfer notes, not weights. To help a small model with a large one, give it the large model's notes; don't fine-tune directly. Fine-tuning works only "self-produced, self-consumed", and most of its benefit comes from practice effects, not from learning others' experience.
4. Keep the notebook when upgrading models. Notes are model-agnostic; weights are model-bound. When switching models, the notebook transfers directly; weights must be retrained from scratch.
5. Both properties can be measured before training. You don't need to train first and judge later—recurrence and state-conditionality are measurable from the experience pool itself, so routing can serve as a pre-training decision tool, saving substantial trial-and-error cost.
A Deeper Reflection
The paper suggests a more general pattern: many debates remain unresolved because both sides argue at the wrong granularity.
"Fine-tuning or retrieval?" cannot be answered because "one trajectory" is not a meaningful unit. Only by decomposing trajectories into components and asking the question per component do meaningful answers emerge.
This is isomorphic to the "judgment-gate decoupling" lineage: don't judge at the aggregate level; judge at the component level. Aggregate-level conclusions depend on mixing ratios; component-level conclusions are stable.
The paper's title says it well—Not All Experience Belongs in the Weights. It generalizes: not all experience belongs in the same place. The value of experience lies not in whether it is "good" but in what *type* it is—different types have different optimal storage locations.
Humans have long known this. We don't store phone numbers and bicycle-riding skills in the same place—the former is declarative memory (verbally teachable), the latter procedural memory (only learnable by practice). We don't store "Paris is the capital of France" and "meeting at 3pm today" in the same place—the former is long-term memory, the latter working memory.
What this paper does is rediscover an old cognitive-science insight, on AI agents.
---
Paper: Not All Experience Belongs in the Weights: Component Routing for Self-Improving GUI Agents arXiv: 2610.01787 Authors: Beining Wu, Zihao Ding, Jun Huang Code: The paper states it will be released; not yet publicly available