EnvFactory: When AI Builds Its Own Arsenal for Agentic RL Training

Forum topic · 小凯 · 2026-05-25

Summary

Agentic reinforcement learning suffers from a shortage of executable training environments: real APIs are costly, and LLM-generated simulated environments are unreliable and lack state persistence. A Chinese forum post introduces EnvFactory, a fully automated framework that mines real resources from the web, explores and verifies executable stateful tool environments, and synthesizes high-quality training data via topology-aware sampling. Unlike earlier synthetic datasets with rigid, machine-like instructions, EnvFactory generates multi-turn, implicitly-intended tasks that better reflect real human reasoning. Using only 85 verified environments and roughly two thousand data samples to fine-tune Qwen3 models, the method reportedly improved scores by 15% on the BFCLv3 tool-calling benchmark, outperforming prior approaches that used 5x more environments. The post summarizes the pipeline as T_data = Refine(Sample_topology(Verify(Explore(R_real)))) and cites arXiv:2605.18703 (May 20, 2026).

EnvFactory: When AI Builds Its Own "Arsenal" — No More Borrowed Training Grounds

Introduction: The Awkwardness of Armchair Training

Teaching AI to use tools is like teaching a person to ride a bicycle. Reading the manual alone is useless — you need a real bike and a practice ground to fall in.

This is what the field calls Agentic RL (agentic reinforcement learning). The problem: where do you find enough training grounds? Calling real external software APIs directly? Too expensive, and you can't afford to break things. Having a large language model "install" a simulated environment itself? It often hallucinates, and save states from yesterday may be gone today.

Without a real training ground, AI can only practice on paper — and everything it learns is empty show.

The Root Cause: Fake, Formulaic "Textbooks"

Even when a barely usable simulated environment exists, the training material isn't good enough.

Previous practice tasks gave overly rigid instructions, like "click the top-left button to open the menu, then type 'apple'." But in the real world, users just say "I want to buy two pounds of Red Fuji apples." This means the AI gets used to spoon-fed teaching — the moment a user's intent becomes indirect, it's lost.

> 💡 Tip: This is called the Lack of Implicit Human Reasoning. In plain terms, the training data looks too much like machine-generated code, with no human flavor, so the AI never learns to generalize.

The Breakthrough: EnvFactory's Creation Powers

In May 2026, EnvFactory arrived.

It does something remarkable: since there's no training ground and no good textbook, it builds its own arsenal:

1. Automatic mining and factory construction: It finds real resources on the web, automatically explores them, and builds genuinely executable tool environments with state memory — no human intervention required, and the environments it builds withstand verification. 2. Topology-based sampling to write textbooks: With a training ground in place, you still need exam questions. By perceiving the environment's topological structure, it automatically generates high-quality, real-world training data with multi-turn dialogues and implicit intents.

This generative logic can be captured by a self-sustaining formula:

$\mathcal{T}_{data} = \text{Refine}\left( \text{Sample}_{\text{topology}}(\text{Verify}(\text{Explore}(R_{real}))) \right)$

> 💡 Formula explained: It first explores real resources ( $R_{real}$ ) and verifies them into executable environments, then samples and refines based on the environment's structural topology, finally producing high-quality training data ( $\mathcal{T}_{data}$ ).

Head-to-Head Comparison

Battlefield Results: Winning Many with Few

The results were surprising.

Researchers had it build only 85 verified environments, producing a bit over two thousand data samples. Training Qwen3 models on just this data, scores surged by 15% on BFCLv3, a top-tier tool-calling benchmark — decisively beating older methods that used 5x more environments.

With EnvFactory, AI finally achieves "training freedom": it builds its own targets, shoots its own arrows, and reviews its own performance. No external help needed — a world of its own.

---

References

Paper title: EnvFactory: Scaling Tool-Use Agents via Executable Environments Synthesis and Robust RL
Published: May 20, 2026
Paper ID: arXiv:2605.18703
Core contribution: Solving the bottlenecks of scarce executable environments and training data lacking real human intent in agentic reinforcement learning.
Mechanism: A fully automated framework enabling autonomous exploration and verification of stateful tool environments, plus topology-aware synthesis of high-quality practical training data.