Amazon's Rufus-Air: Full Post-Training Recipe for GLM-4.5-Air Released

Forum topic · 小凯 · 2026-10-19

Summary

Amazon published a 22-author paper detailing Rufus-Air, a complete post-training recipe built on Zhipu's open-source GLM-4.5-Air base model (106B parameters, 12B active MoE). Instead of releasing weights, Amazon released the full pipeline: an eight-stage sequential recipe (SFT, three RL stages for reasoning/coding/instruction-following, three agent stages, then human-feedback alignment), including data composition, reward design, hyperparameters, failure modes, and infrastructure costs. The key ordering principle: sequence stages by reward reliability—hard, verifiable rewards first, gameable LLM-judge rewards last. Rufus-Air scores 76.9 on IFBench vs. 33.6 for the official GLM-4.5-Air post-trained version, beating it on all reported benchmarks except creative writing (Arena-Hard). Notable practical details include difficulty filtering (training only on problems the model sometimes solves), early stopping based on held-out validation rather than training loss, and hidden agent-training costs (sandbox concurrency, search API bills). The paper unusually admits limitations, arguing that complete reproducible recipes—rather than weights—are what the industry most lacks.

Amazon's Answer to Training Order: Sort by How Easily the Reward Can Be Gamed

Last week Amazon released a paper from a team whose model is named Rufus-Air. What it does may look odd to outsiders: taking Zhipu's open-source GLM-4.5-Air base (a 106B-parameter, 12B-active MoE model), they ran the entire post-training process—supervised fine-tuning, reinforcement learning, and human-feedback alignment—and then wrote out the whole recipe: how the eight stages are ordered, what data each stage uses, how rewards are designed, how many GPUs ran for how many days, and which checkpoints hit what problems. Twenty-two signatories, all from Amazon.

The Results First

On the hard instruction-following benchmark IFBench, the model scored 76.9, versus 33.6 for the official GLM-4.5-Air post-trained version. On the standard IFEval benchmark it's 95.4 vs 83.0. Across the whole comparison table, except for one creative-writing item on Arena-Hard, Rufus-Air beats the official post-trained version on every reported benchmark.

But the scoreboard isn't the paper's contribution. The paper's own words: "what we care about is the recipe itself." No weights were released. The recipe—each stage's training data composition, reward functions, hyperparameters, and common failures—is the deliverable. This separates it from typical model releases: most papers showcase a finished product; this one is a cookbook, written for "teams with a few dozen nodes" to follow.

Eight Stages in Sequence, and Why the Order Matters

The backbone of the recipe is eight stages in a line: supervised fine-tuning, then three RL stages (reasoning, coding, instruction-following), then three agent stages (general agent, coding agent, search agent), closing with human-feedback alignment. Each step continues from the previous stage's checkpoint.

Why this order? The usual answer is "easy tasks first." This paper gives a sharper one: order by reward reliability. Early stages use hard rewards—math answers are right or wrong, code passes tests or doesn't; machine-graded, little room to cheat. Later stages use soft rewards—LLM judges assigning scores, with much more room to be fooled. Reliable rewards first, noisy ones last, minimizing how long the pipeline is exposed to "reward hacking."

One counterexample clarifies the principle: instruction-following's reward is a rubric-based judge—a soft reward—yet it sits third. Because instruction-following sits close to the model's existing abilities, the judge has little room to be manipulated. So the ordering looks not at reward format, but at how hard the reward is to game.

This ordering logic is worth copying into any pipeline: put reliably-graded rewards first, gameable rewards last, regardless of task difficulty.

Three Practical Details Worth Picking Up

The paper's real value is in the details. Three highlights:

1. Difficulty filtering. Before each RL stage, the current model attempts all candidate problems. Problems it always solves are discarded (no gradient signal), problems it almost always fails are discarded (can't learn them), and only the middle band of "sometimes right, sometimes wrong" is trained. The paper calls this an automatic curriculum.

2. Concrete evidence for early stopping. SFT training loss kept decreasing over three epochs, looking healthy. But held-out validation scores plateaued within the first epoch; the next two epochs of loss decrease were the model "parroting seen data." The checkpoint they carried into RL came from the first epoch's plateau, not the final step. Without the held-out comparison line, the team would have spent eight extra GPU-days for a worse model.

3. The real bill for agent RL. The coding agent's average trajectory involves dozens of tool calls, with rewards from running test suites in a sandbox—the sandbox service must sustain roughly 10,000 concurrent sessions, at maintenance costs on the order of $10,000 per month. One full search-agent evaluation hits the public web 100,000 times for searches and page extraction—several hundred dollars at list prices; training switched to an internal search backend. Beyond GPUs, this is the hidden cost of agent training.

Honest to a Fault

The paper includes a "what the recipe does not prove" section, unusually candid: the eight-stage order is "a recipe that works, not a proven-optimal one"; the coding agent stopped when compute ran out, not at convergence; no benchmark was tracked across all eight stages, so changes between stages can't be attributed to a specific stage. Two numeric inconsistencies in the data tables (GPQA for the IF RL checkpoint appears as 71.1 and 73.36 in two tables) are flagged by the paper itself.

These admissions strengthen, not weaken, the work. A recipe claiming "reproduce this" is most endangered by reporting only successes—otherwise you discover the pitfalls only at step 1200 of your own run.

Who This Paper Is Calling Out

The paper opens with: "post-training sections often read like a system card; the details teams actually need are precisely the least disclosed." That's a shot at the whole industry: every "we did SFT and RLHF" table in a model release says, in the authors' view, nothing.

Model weights are increasingly open, but complete recipes for "how to train it from a base model" remain scarce. The value of Amazon's paper isn't the scores—green across the board except one creative-writing item—but publishing training-log-level detail: data mixes, filtering rules, the rationale for stage ordering, infrastructure pitfalls, cost breakdowns. With fifty nodes, someone else can actually build their own model from it.

There's an ironic twist at the end. Papers that "write out all training details" are traditionally academic; industry hides the recipe. This time it flipped: Zhipu, the most aggressive weight-releaser, supplied the base, while Amazon wrote the full recipe. Openness has moved from weights to the recipe layer—weights are the model, the recipe is the method for making models. Methods are harder to open-source than results. This paper opened them.

---

*Source: arXiv 2609.29421v2 (2026-09-25, Amazon, 22 authors); model: Rufus-Air, base: GLM-4.5-Air-Base 106B-A12B; training stack: adapted slime + Megatron-LM + SGLang + E2B sandbox; comparison table includes INTELLECT-3, Nemotron-3-Super, Qwen3.5, and ten models total. Weights not publicly released as of writing.*