Google's Titans & MIRAS: Breaking the AI Long-Term Memory Bottleneck

Based on Google Research papers *"Titans: Learning to Memorize at Test Time"* and *"It's All Connected: A Journey Through Test-Time Memorization, Attentional Bias, Retention, and Online Optimization"* (December 2025).

Current AI Memory Limitations

• Transformers become slower and more expensive as text volume grows, with quadratic computational complexity.
• Existing AI systems (ChatGPT, Claude, Gemini) tend to forget information in long conversations or when processing long documents.
• This is a long-standing fundamental limitation that hinders progress toward AGI.

Titans: A Brain-Inspired Three-Tier Memory System

• Short-term memory (core): Handles the current task via attention mechanisms, analogous to human short-term memory.
• Long-term memory: Stores historical information and can learn and update at runtime, using a "surprise metric" to decide what to store.
• Persistent memory: Knowledge fixed during training, akin to human meta-memory, remaining stable.

The MAC (Memory as Context) architecture treats memory as context, enabling context windows of over 2 million tokens.

MIRAS: A Unifying Framework for Sequence Models

MIRAS reveals that major AI architectures (Transformer, RNN, Mamba, etc.) are fundamentally the same, opening the door to better memory system design. It defines four components:

• Memory architecture: How information is stored (vectors, matrices, MLPs).
• Attentional bias: The internal learning objective that determines what the model prioritizes.
• Retention gating: A forgetting mechanism balancing new learning against retaining old memories.
• Memorization algorithm: The optimization algorithm that updates the memory state.

Key innovation: introducing a non-Euclidean objective function, allowing more complex mathematical penalty mechanisms.

Performance Validation

• Lower perplexity (better performance) on long sequences compared to Mamba; deep memory architectures maintain their advantage across parameter scales.
• Strong results on the Babai Long task (factual QA over ultra-long documents).
• Even smaller models (fewer parameters, lower cost) maintain high accuracy on long sequences.
• Significant advantages over GPT-4 and Mamba, especially with ultra-long contexts.

Applications and AGI Implications

• Legal document analysis: processing extremely long legal files
• Medical records management: integrating long-term patient data
• Research paper understanding: connecting concepts across papers
• Codebase maintenance: understanding large code projects
• Personal assistants: remembering user preferences and history
• AGI significance: a human-brain-like memory system enabling previously impossible applications

Conclusion: A Major Step Toward AGI

• Titans and MIRAS address the fundamental problem of AI long-term memory.
• They enable test-time learning, allowing models to keep evolving during use.
• They offer a new technical path toward AGI, with Google once again leading AI architecture innovation beyond the Transformer it created.