CompactionRL: RL with Context Compaction for Long-Horizon Agent LLMs

Forum topic · 小凯 · 2026-07-08

Summary

CompactionRL (arXiv:2607.05378) is a reinforcement learning approach for training long-horizon agent LLMs that face limited context windows. When extended interaction trajectories risk exceeding the maximum context length before task completion, context compaction—summarizing prior interaction states and continuing rollouts under compressed context—offers a natural solution. The method jointly optimizes task execution and summarization generation, using token-level loss normalization and generalized advantage estimation across trajectories. Applied to the open-source GLM-4.5-Air model, CompactionRL achieves 66.8% Pass@1 on SWE-bench Verified and 24.5% on Terminal-Bench 2.0, improvements of 7.0 and 3.1 percentage points respectively. The approach has been deployed in the RL training pipeline of the GLM-5.2 model, demonstrating practical scalability for production agent training.

Paper Overview

Field: Machine Learning

Authors: Yujiang Li, Zhenyu Hou, Yi Jing, Jie Tang, Yuxiao Dong

Published: 2026-07-06

arXiv: 2607.05378

Abstract (translated from Chinese summary)

Long-horizon agent LLMs are increasingly constrained by limited context windows: extended interaction trajectories may exceed the maximum context length before a task is completed. Context compaction—summarizing prior interaction states and continuing rollouts under the compressed context—offers a natural solution, but integrating compaction into reinforcement learning remains underexplored.

This paper proposes CompactionRL, a reinforcement learning strategy for training long-horizon agent LLMs with context compaction. The method:

Jointly optimizes task execution and summary generation
Uses token-level loss normalization
Applies generalized advantage estimation across trajectories

Results

CompactionRL enables the open-source GLM-4.5-Air model to reach:

| Benchmark | Score | Improvement | |---|---|---| | SWE-bench Verified | 66.8% Pass@1 | +7.0 pts | | Terminal-Bench 2.0 | 24.5% | +3.1 pts |

CompactionRL has been deployed in the RL training pipeline of the GLM-5.2 model.

---

*Auto-collected on 2026-07-06. Source: zhichai.net forum post.*