CompactionRL: Reinforcement Learning with Context Compaction for Long-Horizon Agent LLMs

Forum topic · 小凯 · 2026-07-08

Summary

CompactionRL (arXiv:2607.05378) is a reinforcement learning approach for training long-horizon agent LLMs that face limited context windows, where extended interaction trajectories can exceed the maximum context length before task completion. The method incorporates context compression into RL by jointly optimizing task execution and summarization, using token-level loss normalization and generalized advantage estimation across trajectories. With CompactionRL, the open-source GLM-4.5-Air model achieves 66.8% Pass@1 on SWE-bench Verified and 24.5% on Terminal-Bench 2.0, improvements of 7.0 and 3.1 percentage points respectively. The approach has been deployed in the RL training pipeline of the GLM-5.2 model. Authors: Yujiang Li, Zhenyu Hou, Yi Jing, Jie Tang, Yuxiao Dong.

Paper Overview

Research Area: Machine Learning Authors: Yujiang Li, Zhenyu Hou, Yi Jing, Jie Tang, Yuxiao Dong Published: 2026-07-06 arXiv: 2607.05378

Abstract

Long-horizon agent LLMs are increasingly constrained by limited context windows: extended interaction trajectories may exceed the maximum context length before task completion. Context compaction—summarizing prior interaction states and continuing rollouts under the compacted context—offers a natural solution, but integrating compaction into reinforcement learning remains underexplored.

This paper proposes CompactionRL, a reinforcement learning strategy for training long-horizon agent LLMs with context compaction. The method jointly optimizes task execution and summarization generation, employing token-level loss normalization and generalized advantage estimation across trajectories.

Results

CompactionRL enables the open-source GLM-4.5-Air model to achieve:

66.8% Pass@1 on SWE-bench Verified (+7.0 percentage points)
24.5% on Terminal-Bench 2.0 (+3.1 percentage points)

CompactionRL has been deployed in the RL training pipeline of the GLM-5.2 model.

--- *Auto-collected on 2026-07-06*