In-Place Test-Time Training: A Plug-and-Play Framework for Adapting LLMs at Inference

Forum topic · 小凯 · 2026-04-09

Summary

This paper introduces In-Place Test-Time Training (In-Place TTT), a framework that enables Large Language Models (LLMs) to adapt their weights during inference without costly retraining. The static train-then-deploy paradigm limits LLMs from dynamically responding to new information streams, and existing Test-Time Training (TTT) approaches face architectural incompatibility, computational inefficiency, and fast-weight objectives misaligned with language modeling. In-Place TTT addresses these barriers by treating the final projection matrix of ubiquitous MLP blocks as adaptable fast weights, providing a plug-and-play enhancement to existing LLMs. The method combines a theoretically grounded objective tailored to next-token prediction in autoregressive language modeling with an efficient block-wise update mechanism, yielding a highly scalable algorithm compatible with context parallelism. Authored by Guhao Feng, Shengjie Luo, and Kai Hua, the paper (arXiv 2504.06263, published April 2025) falls in the NLP domain.

Paper Overview

Research Area: NLP Authors: Guhao Feng, Shengjie Luo, Kai Hua Published: 2025-04-08 arXiv: 2504.06263

Introduction

The static "train then deploy" paradigm fundamentally limits Large Language Models (LLMs) from dynamically adapting their weights in response to continuous streams of new information inherent in real-world tasks. Test-Time Training (TTT) offers a compelling alternative by updating a subset of model parameters (fast weights) at inference time, yet its potential in the current LLM ecosystem is hindered by critical barriers including architectural incompatibility, computational inefficiency and misaligned fast weight objectives for language modeling.

Key Idea

This work introduces In-Place Test-Time Training (In-Place TTT), a framework that seamlessly endows LLMs with Test-Time Training ability. In-Place TTT treats the final projection matrix of the ubiquitous MLP blocks as its adaptable fast weights, enabling a "plug-and-play" enhancement of existing LLMs without expensive retraining from scratch.

Method Highlights

Uses the final projection matrices of MLP blocks as adaptable fast weights, requiring no architectural overhaul.
Combines a theoretically grounded objective tailored to next-token prediction in autoregressive language modeling.
Employs an efficient block-wise update mechanism.
Produces a highly scalable algorithm compatible with context parallelism.