Your Agent Aces the Lab but Fails in the Wild: Noisy-Environment Training for Robustness

Forum topic · 小凯 · 2026-05-28

Summary

A review of the paper "Learning to Act under Noise: Enhancing Agent Robustness via Noisy Environments" (arXiv:2605.27209), which addresses why LLM agents that score highly on clean benchmarks collapse in production. The authors identify two noise dimensions in real-world interaction: user-side noise (ambiguous intents, inconsistent requirements, redundant information) and tool-side noise (call failures, truncated outputs, error messages, verbose outputs). Their solution combines mixed-trajectory training—alternating clean and noisy rollouts so clean trajectories preserve reliable reward signals while noisy ones build robustness—with an adaptive noise curriculum that monitors the performance gap between ideal and noisy environments and increases noise difficulty as it narrows. Optimization uses GRPO, whose group-relative normalization suits heterogeneous reward distributions. Experiments show noisy-trained agents substantially outperform clean-trained baselines under realistic noise and, counterintuitively, also achieve consistent gains on standard clean benchmarks, indicating noise acts as implicit data augmentation. The article discusses practical implications (train on dirty data, gradual noise injection), limitations (seven noise types covered, threshold tuning), and frames the work as a paradigm shift: training agents to survive noisy reality, not just excel in idealized test conditions.

Your Agent Aces the Lab but Fails in the Wild: Noisy-Environment Training Teaches It to "Breathe in the Real World"

> Source: *Learning to Act under Noise: Enhancing Agent Robustness via Noisy Environments*, arXiv:2605.27209, https://arxiv.org/abs/2605.27209

1. Two Kinds of Agents

An agent in the lab enjoys clear user intents, smooth tool calls, and reliable positive feedback at every step—like a fish in a tank with constant water temperature and punctual feeding.

An agent in the real world hears "just handle that thing for me," watches an API return a 504, and receives a truncated JSON from a database. It is like a fish dropped into a murky river.

The paper's stark finding: most agents have never breathed real-world air during training.

2. The Core Problem: The Illusion of Clean Training

Current LLM agent training has a fatal blind spot: training environments are too clean while deployment environments are noisy. During training, user requests are grammatically complete, tool calls succeed, and reward signals are trustworthy. Models learn to make decisions under ideal conditions but never learn to stay clear-headed when signals distort.

The result: high benchmark scores that collapse in production. Agents are not getting dumber—they were never trained to handle this.

3. Two Categories of Noise

User-side noise—users are not prompt engineers:

Ambiguous intent: "book a nearby restaurant"—how near? What preferences?
Inconsistent requirements: "cheap" in turn one, "nice ambiance" in turn three
Redundant information: three unrelated requests in one message, the real goal buried at the end

Tool-side noise—APIs are not wishing wells:

Call failures: timeouts, permission errors, rate limits
Truncated outputs: incomplete JSON, missing key fields
Error messages: stack traces instead of data
Verbose outputs: the answer on page ten, preceded by logs

These are not occasional glitches—they are the norm of real interaction.

4. Training Strategy: Mixed Trajectories + Adaptive Curriculum

Adding noise to all training data fails: the model drowns in noise, reward signals get polluted, and training collapses.

The solution is mixed-trajectory training: clean and noisy trajectories alternate during rollout. Clean trajectories preserve reliable reward signals; noisy ones inject robustness. Both coexist rather than replace each other.

A key ingredient is adaptive noise scheduling:

Early training uses a low noise ratio so the model learns basics first
The performance gap Δ between ideal and noisy environments is monitored
As Δ shrinks, noise difficulty and proportion automatically increase
This avoids both ineffective exploration and underfitting

It is progressive desensitization: the agent walks from greenhouse to wilderness gradually, rather than being thrown into a storm.

Policy optimization uses GRPO (Group Relative Policy Optimization). Its group-level normalization naturally fits heterogeneous environments—clean and noisy trajectories have different reward distributions, and GRPO stabilizes training via within-group relative advantages.

5. Experiments: Noise Is Not a Burden but Implicit Augmentation

Noisy test scenarios: with user-side and tool-side noise injected, noise-trained agents significantly outperform clean-environment baselines—not marginally better, but a qualitative shift from unusable to usable under real deployment conditions.
Clean test scenarios: counterintuitively, noise-trained agents also show consistent improvements on standard ideal benchmarks. Noise did not hurt ideal performance—it improved generalization through implicit data augmentation.

This double benefit confirms the core hypothesis: noise is not an enemy to eliminate but a training signal to exploit.

6. Why Mixed Rather Than Pure Noise

Pure noise training suffers from unreliable rewards: if a trajectory contains ambiguous intent and failed tool calls, the model cannot tell whether failure came from a policy error or environmental noise—gradients become confused.

Mixed trajectories are elegant: clean trajectories provide a trustworthy baseline ("this is right under ideal conditions"); noisy trajectories provide stress tests ("which policies still work when conditions degrade"). By contrasting both, the model learns to separate "policy mistakes" from "environmental interference."

The Δ metric forms a closed-loop feedback system: large Δ means more noise training is needed; small Δ means robustness is sufficient and difficulty can rise.

7. Practical Takeaways for Production Agents

Training data must be "dirty": if training is cleaner than production, deployment will fail. Collect real user logs with ambiguous requests and tool failures and inject them into training.
Gradual noise injection: start at a low ratio and adjust dynamically based on performance—mirroring human learning.
GRPO suits heterogeneous training: PPO is unstable when reward distributions diverge; GRPO's normalization fits mixed clean/noisy settings naturally.
Noise as data augmentation: gains on clean benchmarks show noise training acts as implicit regularization.

8. Limitations

Noise modeling covers 7 types (3 user-side + 4 tool-side); real-world noise is more diverse
The Δ threshold for adaptive scheduling requires per-task tuning
Trajectory mixing ratios may need finer scheduling on complex tasks
Extreme noise (e.g., deliberately misleading users) is not yet covered

9. Conclusion

The paper's core contribution is not a new algorithm but a paradigm shift: from "training agents to score high in ideal environments" to "training agents to survive in noisy ones." The real world is not a benchmark—users do not write requests in paper format, and APIs do not return data per the docs. Robustness is not a bonus; it is a survival line.

---

References

*Learning to Act under Noise: Enhancing Agent Robustness via Noisy Environments*, arXiv:2605.27209, https://arxiv.org/abs/2605.27209
Authors: Yuxin Chen, Xiaodong Cai, Junfeng Fang, Zhuowen Han, Yu Wang, Yaorui Shi, Yi Zhang, Qi Gu, Xunliang Cai, Xiang Wang, An Zhang, Tat-Seng Chua (National University of Singapore, Meituan, Tsinghua University, Tianjin University, University of Science and Technology of China)