CompactionRL: Reinforcement Learning with Context Compaction for Long-Horizon LLM Agents

Summary

CompactionRL (arXiv:2607.05378) is a reinforcement learning approach for training long-horizon LLM agents that operate under context compaction. As agent interaction trajectories can exceed the maximum context window before a task is completed, context compression—summarizing prior interaction states and continuing rollouts from the compressed context—offers a natural solution, but integrating compaction into RL has been underexplored. The method jointly optimizes task execution and summary generation, using token-level loss normalization and cross-trajectory generalized advantage estimation. With CompactionRL, the open-source GLM-4.5-Air model achieves 66.8% Pass@1 on SWE-bench Verified and 24.5% on Terminal-Bench 2.0, improvements of 7.0 and 3.1 percentage points respectively. The approach has been deployed in the RL training pipeline of the GLM-5.2 model, making it a practical technique for scaling agentic RL beyond fixed context limits.