Overview
Envs-FORGE is a method for synthesizing training environments in agent reinforcement learning (RL). Rather than applying the same rewriting recipe to every seed task—as Few-shot, Self-Instruct, and Evol-Instruct do—Envs-FORGE estimates each seed's pass rate under the current policy and selects a customized rewriting strategy via mixed-integer programming (MILP).
Reference: arXiv:2608.14312 | Code: DataArc-SynData-Toolkit
The Problem with Fixed Recipes
Current environment-synthesis methods apply identical transformations regardless of whether a seed is too easy (90% pass rate) or too hard (5% pass rate). The pass-rate signal is ignored, so easy seeds get harder and hard seeds get even harder—or are diversified into tasks that may still be unreachable.
Envs-FORGE's core insight: estimate seed pass rate first, then decide how to rewrite it.
The 3×2 Action Space
Each seed task receives one of three projections:
- Increase — raise complexity
- Reduce — lower complexity, producing bridge tasks that step back from seeds beyond the current policy's reach
- Diversify — preserve difficulty while changing the scenario
- In-depth — deepen within the same domain
- In-breadth — extend across domains
- Paper: arXiv:2608.14312
- Code: github.com/DataArcTech/DataArc-SynData-Toolkit
- HTML: arxiv.org/html/2608.14312v1
Each projection runs along one of two directions:
This yields 3 × 2 = 6 combinations. The choice is computed by an MILP solver.
MILP as Strategy Selector
Envs-FORGE models environment synthesis as a MILP that optimizes simultaneously for:
1. Frontier matching — synthesized difficulty distribution should align with the policy's capability frontier 2. Skill coverage — synthesized environments should fill gaps in the target skill graph 3. Verifier feasibility — every synthesized environment must pass an executable test-based reward verifier
Fixed recipes are recovered as special cases via action masks: Evol-Instruct always Increases; Self-Instruct always Diversifies. The solver outputs 100 verified environments that feed directly into GRPO training.
Bridge Tasks and the Zone of Proximal Development
When a seed's pass rate is far below the policy's ability, training on it yields no learning signal. Envs-FORGE's Reduce projection creates intermediate bridge tasks—stepping back from an unreachable seed to a difficulty the policy can actually learn from. This mirrors Vygotsky's concept of the zone of proximal development: learning happens at the edge of capability, not far beyond it.
Results
GRPO training on Qwen 3.5 35B:
| Benchmark | Base | Envs-FORGE | Strongest Baseline | Gain | |-----------|------|-----------|--------------------|------| | tb-core | 40.0% | 49.2% | 46.8% | +9.2 pp | | tb-2.0 | 23.0% | 29.4% | 27.3% | +6.4 pp | | SWE-bench Verified | 73.4% | 77.1% | — | +3.7 pp |
Across 4B–35B model sizes, tb-core improves by 6.8–9.2 pp.
Crucially, all four synthesis methods produce 100 verified environments using 2.27M–2.88M synthesized tokens. Envs-FORGE's advantage comes from better strategy selection—not more data or compute.
Costs and Limitations
The paper reports MILP solver overhead transparently but does not fully quantify it. Appendix B provides a full audit trail of the branch-and-cut search, fallback boundaries, and solver decisions.
Evaluation scope is limited to Terminal-Bench subsets (CLI tasks) and SWE-bench Verified (code repository repair). Generalization to web interaction, multimodal agents, and broader task types remains open.
Broader Pattern
Envs-FORGE is an instance of a recurring principle: optimization granularity should match the granularity of what is being optimized. Where Heddle and CodeRescue raise decision granularity from single calls to trajectory/recovery actions, Envs-FORGE raises environment-synthesis granularity from fixed recipes to per-seed customization. Fixed recipes are uniform processing; MILP is on-demand allocation.