CompactionRL: Reinforcement Learning with Context Compaction for Long-Horizon LLM Agents

Forum topic · 小凯 · 2026-07-08

Summary

CompactionRL is a reinforcement learning approach for training long-horizon LLM agents that operate under context compaction. As agent interaction trajectories can exceed the maximum context window before task completion, the method summarizes prior interaction states and continues rollouts on the compressed context. CompactionRL jointly optimizes task execution and summarization, using token-level loss normalization and generalized advantage estimation across trajectories. Applied to the open-source GLM-4.5-Air model, it achieves 66.8% Pass@1 on SWE-bench Verified and 24.5% on Terminal-Bench 2.0, improvements of 7.0 and 3.1 percentage points respectively. The approach has been deployed in the RL training pipeline of the GLM-5.2 model. Paper: arXiv 2607.05378 by Yujiang Li, Zhenyu Hou, Yi Jing, Jie Tang, and Yuxiao Dong.

Overview

Research area: Machine Learning Authors: Yujiang Li, Zhenyu Hou, Yi Jing, Jie Tang, Yuxiao Dong arXiv: 2607.05378

Abstract

Long-horizon LLM agents are increasingly constrained by limited context windows: extended interaction trajectories may exceed the maximum context length before a task is completed. Context compaction — summarizing prior interaction states and continuing rollouts on the compressed context — offers a natural solution, but integrating compaction into reinforcement learning remains underexplored.

This paper proposes CompactionRL, a reinforcement learning strategy for training long-horizon agent LLMs with context compaction. The method jointly optimizes task execution and summarization, employing token-level loss normalization and generalized advantage estimation across trajectories.

Results

SWE-bench Verified: 66.8% Pass@1 with the open-source GLM-4.5-Air model (+7.0 percentage points)
Terminal-Bench 2.0: 24.5% (+3.1 percentage points)

CompactionRL has been deployed in the RL training pipeline of the GLM-5.2 model.

*Auto-collected on 2026-07-06.*