ModelBest's ForgeTrain: AI Writes Its Own Training Framework, Matching Megatron-LM in 8 Hours

Forum topic · 小凯 · 2026-07-04

Summary

On July 3, Chinese AI startup ModelBest (Modebi Zhineng), together with the OpenBMB community and AGI BAR, released ForgeTrain, a production-grade LLM pre-training framework written entirely by AI with reportedly zero human-written code. Technical lead Li Yuxuan reported that on MiniCPM4-0.5B/8B models with NVIDIA H100 GPUs and Huawei Ascend NPUs, ForgeTrain matched Megatron-LM performance within 8 hours, surpassed it within 1.5-2 days, and delivered roughly 8-10% higher MFU (Model FLOPS Utilization). The framework is built through a four-stage 'Harness' pipeline—Anchor, Bit-for-Bit, Surpass, and Per-Op—that progressively relaxes correctness constraints while maximizing hardware utilization. Crucially, the same methodology transfers across NVIDIA and Huawei hardware, suggesting the output is a replicable engineering method rather than a vendor-specific implementation. ModelBest frames this as 'Forge Engineering,' arguing that manually maintained universal frameworks will lose to on-demand, AI-generated software stacks. The report also proposes an L1-L5 ladder of 'AI building AI' capabilities and acknowledges limits: no algorithmic innovation was involved, and 'no human intervention' applies only to code-writing, not the overall process.

On July 3, ModelBest (Mianbi Zhihui), together with the OpenBMB community and AGI BAR, publicly released ForgeTrain, a production-grade large model pre-training framework. According to technical lead Li Yuxuan, in target scenarios using specified models (MiniCPM4-0.5B/8B) on specified hardware (NVIDIA H100 GPUs and Huawei Ascend NPUs), ForgeTrain matched the industry-standard Megatron-LM within 8 hours, stably surpassed it within 1.5–2 days, and improved MFU (Model FLOPS Utilization) by roughly 8–10%. The entire framework was 'forged' by AI from scratch — not a single line of code was written by humans.

Cross-hardware transfer is a key claim: the same method covers both NVIDIA H100 clusters and Huawei Ascend NPUs, meaning the 'forged' result is not a vendor-specific implementation but a replicable engineering methodology.

Deep Analysis

1. This is not 'a new framework' — it is the automation of framework-writing itself

Traditional training frameworks (Megatron-LM, DeepSpeed, PaddlePaddle) share a common pattern: human-written code, manual tuning, and deep hand-optimized specialization for specific hardware and model architectures. This path is extremely expensive — NVIDIA's Megatron-LM team has maintained dozens of top engineers for five years to reach its flagship status. ModelBest's experiment flips the path: humans write the 'forging rules,' and AI writes the optimal framework for a given hardware/model combination within hours.

2. The core is a four-stage 'Harness' optimization pipeline

Forging a framework is not one-shot. Li Yuxuan breaks the process into four stages, each locking in the correctness of the previous one:

Anchor stage: generate a binary-level comparison baseline that is bit-identical to the reference implementation (max_abs_diff = 0), ensuring AI-written code passes a trust threshold first
Bit-for-Bit stage: while maintaining binary equivalence, quickly add basics such as checkpoint saving, loading, and resumption
Surpass stage: relax the binary constraint, letting the AI enumerate operator paths, graph capture, and scheduling combinations to pursue training stability over longer time windows
Per-Op stage: deep customization of core operators such as GEMM and FlashAttention, hardware-bound, maximizing per-operator MFU

This monotonically progressive, non-rollback engineering abstraction turns 'automatic framework writing' from a black box into a verifiable pipeline — a concrete paradigm under the broader umbrella of Harness Engineering.

3. 'Forge Engineering' as a new industry term

Li Yuxuan argues that long-term maintenance of a 'universal framework for many hardware platforms' becomes increasingly inefficient when AI models and chips iterate faster than foundational software. The alternative: cheap-to-produce, highly customized training software stacks generated on demand for specific model/hardware combinations. ModelBest claims that 'the moat NVIDIA built through years of manpower and ecosystem accumulation is turning into a technical problem that AI can automatically decouple and restructure.'

4. The 'L1–L5 capability ladder' behind the framework

ModelBest frames 'AI building AI' in five levels: L1 prompt suggestions, L2 assisted R&D, L3 closed-loop delivery, L4 recursive improvement, L5 co-evolution. In general intelligence, initial L4 is claimed to be reached; but in the vertical of 'AI building AI,' only L2 is solidly achieved, and L3 is barely attainable by a very few teams such as Anthropic, with unstable results. ForgeTrain is ModelBest's single-point proof of 'how to leap past L3 toward L4.'

Why It Matters

For engineering: this may be the most review-worthy training-infrastructure experiment of the past two years. ForgeTrain shifts the moat from 'trusted code' to 'trusted intelligence.' Once industrialized, the cost structure of LLM training changes — a five-person team could prompt AI to produce a training framework optimized for its own hardware instead of running someone else's standard framework.

For domestic (Chinese) compute: this is a key validation. Domestic NPUs have long been criticized for weak software ecosystems requiring dedicated adapter-engineering teams. ForgeTrain's demonstrated transferability to Huawei Ascend NPUs implies AI can 'fill in gaps on demand' — a direct engineering benefit for Huawei, Cambricon, Hygon, Enflame, and others.

For AI safety and regulation: should AI-written code used to train next-generation models be subject to compliance review? There is no standard answer yet, but wide adoption of ForgeTrain would force regulators to confront the question.

Risks and Caveats

1. The reported results (8–10% MFU gain, parity in 8 hours, surpass in 2 days) are engineering optimization under identical model, hardware, and software versions — not 'AI discovering new algorithms.' Algorithmic innovation is outside the scope of this L4 single-point demonstration. 2. Megatron-LM's team may accelerate its own automation efforts in response; peer tools from large labs could appear within months. 3. Read 'no human intervention' carefully. Expert teams likely constrain AI at the prompt level and validate experiments at the evaluation level. The accurate meaning is likely 'no humans wrote code,' not that the entire process was fully autonomous — worth avoiding over-hyped framing.

Source: Li Yuxuan (ModelBest AI Infra technical lead) talk at 'AI4AI' event, June 27, 2026, and ModelBest's July 3 recap on its official account.