RoboTTT: A 'Memory Revolution' for Robots — From Goldfish to Historian
*Translation of a Chinese forum post by 小凯 (zhichai.net), July 20, 2026.*
> *"Memory is the treasury and guardian of all things."* — Marcus Tullius Cicero (the original cited Proust's aphorism on memory)
---
The Goldfish Problem
Legend says a goldfish's memory lasts only 7 seconds. In some ways, today's robots share this predicament.
State-of-the-art robot foundation models — Google's RT-2, OpenVLA, and other vision-language-action (VLA) models — typically process only single-step or very short visuomotor context. They see the current frame, take an action, and largely "forget" what they just observed.
For simple tasks this barely suffices. But for complex tasks — assembling furniture, cooking a multi-step dish, tidying a cluttered room — goldfish-style memory becomes a fatal bottleneck.
This post covers RoboTTT (Test-Time-Training Robot Policies), from NVIDIA Research, Stanford University, and UT Austin. Its core contribution in one sentence:
> They extended robot memory from 7 seconds to the length of a feature film — to 8,000 timesteps, three orders of magnitude beyond current state-of-the-art policies, with no added inference latency.
---
Three Forms of Memory in Robots
- Goldfish-type (single-step context): sees only the current frame. Fast and cheap, but cannot use history — e.g., it doesn't know what's in its gripper if occluded.
- Squirrel-type (short history): remembers the last 4–16 frames and actions. Handles simple temporal tasks, but is too short for long-horizon goals like "open the drawer, take out the tool, tighten the screw."
- Historian-type (ultra-long context): RoboTTT's goal. At one action per second, 8,000 steps is 133 minutes — over two hours — enough to recall everything seen and done across an entire task.
- Constant memory usage
- Inference speed unaffected by history length
- The model still learns from history at test time
- +87% overall performance vs. single-step-context baselines across multiple real-world manipulation tasks.
- A 5-minute, 10-stage assembly task — no baseline could complete it; RoboTTT succeeded, using its 8,000-step memory to retain the task's overall blueprint.
- A scaling law for context: training with 8,000-step context outperformed 1,000-step training by 62% — suggesting context length as a new scaling axis for robot foundation models, analogous to parameter count in LLMs.
- Context captures the temporal structure of the world: processes (assembly order, how many turns a screw needs) only reveal themselves over time.
- Context provides causal cues: seeing "I pushed A, then B fell" lets the robot infer causality that a single frame never could.
- Context underpins self-reflection: a robot that remembers its past actions can ask what went wrong and how to improve — a step toward higher-level intelligence.
- Jiang, Y., Chebotar, Y., Zheng, R., et al. (2026). *RoboTTT: Context Scaling for Robot Policies*. arXiv:2607.15275.
- Project page: https://research.nvidia.com/labs/gear/robottt/
- arXiv: 2607.15275
---
The Core Secret: Fast Weights
Traditional networks have slow weights — long-term memory learned during training and frozen at inference. RoboTTT adds fast weights that are updated during inference via gradient descent at each timestep.
Unlike Transformers, which store history in "sequence space" (vectors that multiply with sequence length), RoboTTT compresses history into "weight space": the fast weights have fixed size regardless of sequence length. Consequences:
---
Two Training Techniques
1. Sequence Action Forcing: the model is forced to predict long action sequences during training, learning temporal dependencies between actions rather than treating each action in isolation. 2. Truncated Backpropagation Through Time (TBTT): backpropagation is done over small segments instead of the full 8,000-step sequence, keeping training tractable.
---
Experimental Results
---
One-Shot In-Context Imitation
RoboTTT can learn a task from a single video of a human performing it: the video fills the 8,000-step context, and the robot then imitates the demonstrated action sequence — no teleoperated demonstrations required.
It can also improve its policy in real time during execution, adjusting grasp and insertion strategies on the fly when something doesn't fit — an online learning capability for unforeseen situations.
---
Why Does Context Matter So Much?
The speculative horizon: if context scaled from 8,000 steps to 8 million, a robot could accumulate something like lifelong experience — every action informed by a career's worth of memory.
---