[论文] AgentJet: A Flexible Swarm Training Framework for Agentic Reinforcemen...

论文概要

研究领域: ML
作者: Qingxu Fu, Boyin Liu, Shuchang Tao
发布时间: 2025-06-01
arXiv: 2606.04484

中文摘要

我们提出AgentJet，一种用于大语言模型（LLM）智能体强化学习的分布式群体训练框架。与将智能体rollout与模型优化紧耦合的集中式框架不同，AgentJet采用解耦的多节点架构，其中群体服务器节点承载可训练模型并在GPU集群上运行优化，而群体客户端节点在任意设备上执行任意智能体。这种设计提供了集中式框架难以支持的能力：（1）异构多模型强化学习，实现以多个LLM为大脑的异构多智能体团队训练；（2）具有隔离智能体运行时的多任务鸡尾酒训练；（3）容错执行，防止外部环境故障中断训练过程；（4）实时代码迭代，允许通过替换群体客户端节点在训练期间编辑智能体。为支持多模型、多轮和多智能体场景中的高效RL，AgentJet引入时间线合并的上下文跟踪模块，整合冗余上下文并实现1.5-10倍训练加速。最后，AgentJet引入自动化研究系统，以研究主题为输入，在大型集群上自主开展长时程、多天的RL研究。利用群体架构，该系统无需人工干预即可复现RL研究者的关键探索性工作流。

原文摘要

We present AgentJet, a distributed swarm training framework for large language model (LLM) agent reinforcement learning. Unlike centralized frameworks that tightly couple agent rollouts with model optimization, AgentJet adopts a decoupled multi-node architecture in which swarm server nodes host trainable models and run optimization on GPU clusters, whereas swarm client nodes execute arbitrary agents on arbitrary devices. This design provides capabilities that are difficult to support in centralized frameworks: (1) heterogeneous multi-model reinforcement learning, enabling the training of heterogeneous multi-agent teams with multiple LLM as brains; (2) multi-task cocktail training with isolated agent runtimes; (3) fault-tolerant execution that prevents external environment failures from int...

自动采集于 2026-06-05

#论文 #arXiv #ML #小凯