On July 31 at 21:00, ModelBest (ModelBest), together with the Tsinghua NLP team, published the paper *Agent-Environment Alignment via Automated Interface Generation* (arXiv:2505.21055) — arguably the cleanest answer of the past seven days to the question "why do AI agents underperform?"
Core results from the paper:
- ALFWorld: Qwen2.5-7B agent task success rate 13.4% → 31.3% (by rewriting feedback wording)
- Across four benchmarks (ALFWorld + ScienceWorld + WebShop + M3ToolEval), up to +45.67% average improvement
- Consistent gains across algorithms: all five agent types — Vanilla / ReAct / Self-Consistency / Self-Refine / Planning — improved
- Consistent gains across models: an interface generated with Qwen2.5-7B also works on Qwen2.5-14B, Llama 3.1-8B, and Llama 3.3-70B
- Repeated invalid actions dropped 65% on ALFWorld and 49% on ScienceWorld
- Consistent across algorithms — all five mainstream agent types improved, so this isn't specific to one algorithm.
- Consistent across scales — 7B / 14B / 8B / 70B all improved with the aligned interface, so this isn't "models too small to read feedback" — the feedback itself is ambiguous.
- Consistent across domains — embodied (ALFWorld / ScienceWorld), web navigation (WebShop), and tool use (M3ToolEval) all improved, so this is a universal bug in the agent-environment interface.
- If your agent keeps hitting a wall in some environment: don't tune the model or add RLHF first. Step one is making environment feedback wording more explicit — turn "Nothing happens" into "a prerequisite action is needed", turn "Invalid action" into "why this action is invalid and what the correct action is".
- If you run agent evaluations: ALIGN reminds you that many "capability deficits" are actually "unfriendly evaluation interfaces". When you get a low score, run an ablation — clarify the environment feedback and rerun. The gain is the "evaluation-interface contamination". It will change how you judge your model.
- If you collect VLA training data: ALIGN's repeated-invalid-action rate (down 65% on ALFWorld) shows interface quality directly affects data quality. A clearer-feedback environment yields demonstration trajectories that are shorter and cleaner — collection cost and data noise both fall.
- If you build agent harness frameworks: ALIGN suggests a reusable pattern — make "automated interface generation" a standalone module decoupled from agent and environment, so the harness can be reused across environments.
- If your product keeps repeating the same mistake on certain cases: it may not be an intelligence problem but a feedback-interface problem. Start auditing the wording of error messages in the conversation history; fix and re-measure within a week.
- 07-19: Anthropic Claude's "Unhobbling" — removing 80% of system prompts caused no drop on coding evaluations
- 07-25: Claude Mythos Preview autonomously cracked an encryption algorithm over 60 hours
- 07-31: ModelBest's ALIGN automatically rewrites environment feedback wording
- 07-30: Hy3 / Inkling-style models pursue "unlocking capability at the harness and engineering level, not by stacking parameters"
- Paper arXiv:2505.21055: https://arxiv.org/abs/2505.21055
- Paper PDF: https://arxiv.org/pdf/2505.21055
- arXiv HTML version: https://arxiv.org/html/2505.21055
- Paper review + Table 1 / Table 2: https://www.themoonlight.io/review/agent-environment-alignment-via-automated-interface-generation
- ModelBest official site: https://modelbest.cn
- July 31 AI news roundup: https://dy.163.com/article/L3772K590531G0IB.html
You don't need to change the agent, the environment, or do RLHF — just make the "error feedback wording" the agent sees clearer, and performance more than doubles.
The problem: agent-environment "interface mismatch"
The paper calls this Agent-Environment Misalignment. A classic example on ALFWorld: an agent must "go to shelf 1" before it can "examine shelf 1", otherwise the environment returns "Nothing happens". The agent interprets this as "shelf 1 is empty" instead of "you need to walk there first" — and this misinterpretation makes it repeatedly hit a wall.
In a preliminary experiment, the authors changed the environment error from "Nothing happens" to "You need to first go to receptacle before you can examine it". This seemingly tiny change lifted Qwen2.5-7B's ALFWorld success rate from 13.4% to 31.3%. In other words, with the 7B model's capabilities unchanged and the environment code unchanged, merely "clearer feedback wording" added 17.9 percentage points.
The ALIGN system automates this insight: an LLM-based Interface Generator automatically analyzes failure cases, induces "hidden constraints", and repackages environment feedback. The generated interface has two components: static information (rules, field explanations) and dynamic observation augmentation (interpreting an "ineffective action" as "you need to do X first").
Why this matters: the next watershed in embodied AI isn't VLA, it's alignment
For two years, discussion of embodied AI has centered on VLAs, diffusion policies, world models, π0.5, Helix — all about the "model backbone". ALIGN offers a completely different view: in many scenarios, model capability is not the bottleneck; what actually blocks the agent is that it cannot understand the feedback about its current state.
The experimental data strongly supports this:
This means that going into 2026 H2, the main battleground of embodied AI is no longer "which next VLA beats π0.5" — it's "how to design environment feedback as an interface agents can stably understand".
Concrete advice for embodied AI / agent developers
Where this line of work is heading in 2026 H2
ALIGN is not isolated; a parallel line is forming:
The common theme: in 2026 H2, model weights are not the core variable. Releasing the potential of existing models from bad interfaces, bad harnesses, and bad system prompts is far cheaper than adding parameters — and far more immediate.
ALIGN's significance for embodied AI is that it validates "in embodied settings too, model weights are not the core variable" — the next watershed is shifting from VLA backbones to the design of the agent-environment interface.
One-line takeaway
ALIGN raised Qwen2.5-7B's ALFWorld score from 13.4% to 31.3% purely by rewriting feedback wording — and the effect holds consistently across algorithms, scales, and domains.
This is not the story of a new model; it is the first systematic empirical validation of the claim that "the next optimization point for embodied AI and agent products lies neither in the model nor in the environment code, but in the neglected interface between them".
Anyone building agent products should ask themselves: does my environment feedback wording push the agent toward the *correct understanding*, or into a wall?
---
References