English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

Paper Slam 4/19: When a Web Designer Meets a Radiologist — Two Agents, Two Roads

Forum topic · 小凯 · 2026-04-28

Summary

This forum post compares two recent AI agent papers: MM-WebAgent (arXiv 2604.15309, Microsoft Research Asia), a hierarchical multimodal web agent for webpage generation, and RadAgent (arXiv 2604.15231, Imperial College London & Stanford), a tool-using agent for stepwise interpretation of chest CT scans. MM-WebAgent addresses creative integration: it plans globally then locally, calls AIGC tools (image, video, chart generation) as first-class actions, applies a three-level reflection loop (local, contextual, global), and was evaluated on the new MM-WebGEN-Bench (120 tasks, six quality dimensions), where it scored 0.75 average versus 0.49 for a code-only agent. RadAgent tackles explainable correctness: instead of a black-box 3D VLM like CT-CHAT (macro-F1 0.287, 0% faithfulness), it performs stepwise tool calls with human-readable traces and faithfulness verification, reaching macro-F1 0.347, a 24.7-point robustness gain under adversarial conditions, and 37% faithfulness on CT-RATE. The post argues both designs are shaped by the cost of failure: web generation tolerates iteration and aesthetic trial-and-error, while medical diagnosis demands verifiable, traceable reasoning. Through a Feynman-style analysis, the author concludes that agent design philosophy must match the application's error tolerance, and that both systems exemplify the underrated engineering skill of organizing existing AI capabilities into reliable, structured workflows.

Paper Slam 4/19: When a Web Designer Meets a Radiologist — Two Agents, Two Roads

This post compares two agent papers published around the same time: MM-WebAgent (arXiv 2604.15309, Microsoft Research Asia), a hierarchical multimodal web agent for webpage generation, and RadAgent (arXiv 2604.15231, Imperial College London & Stanford), a tool-using agent for stepwise interpretation of chest CT. Both are agents, but they walk entirely different roads: one toward aesthetics and creativity, the other toward medical precision.

Key points

MM-WebAgent (arXiv 2604.15309) — creative integration

  • Problem: naively pasting AIGC content (Midjourney images, generated videos, charts) into webpages produces visually incoherent results — elements clash in style, the way a suit paired with slippers clashes.
  • Core design: a three-stage pipeline mirroring how human designers work — global planning (layout, style, content structure expressed as structured JSON), local planning (per-section multimodal asset generation guided by the global blueprint), and integration with a three-level reflection mechanism (Local Refine → Context Refine → Global Refine, the last one screenshotting the rendered page for holistic review).
  • Architecture: a central LLM (gpt-4o) orchestrating tools — image generation, video generation, chart generation, HTML/CSS code generation, and browser rendering for visual checks. Crucially, multimodal content generation is a first-class action, not a placeholder, enabling joint optimization across code correctness, layout, and visual coherence.
  • Evaluation: the team built MM-WebGEN-Bench (120 curated tasks across four intents, multiple visual styles, and varied layout/multimodal complexity) with a six-dimension protocol: layout correctness, style coherence, aesthetics (global) plus image, video, and chart quality (local).
  • Results (from the post's table):
  • | Paradigm | Average | |---|---| | Code-only One-shot | 0.43 | | Code-only Agent | 0.49 | | MM-WebAgent | 0.75 |

    The multimodal agent roughly doubles local multimodal quality and gains 20–30% on global dimensions over a code-only agent — a qualitative leap, not an incremental one.

  • Limitations: dependence on external AIGC tool quality and stability; a fixed tool set (no dynamic tool discovery); no learning — it is a training-free orchestration agent that cannot accumulate design experience.
  • RadAgent (arXiv 2604.15231) — explainable correctness

  • Problem: end-to-end 3D VLMs like CT-CHAT compress the whole CT volume into visual tokens for an LLM. This yields black-box reasoning, severe hallucination, adversarial fragility (macro-F1 of only 0.287 on CT-RATE), and zero faithfulness — no way to trace report claims back to image evidence.
  • Core design: instead of generating a report directly, the agent calls interpretable tools step by step — region localization, lesion detection, measurement, slice comparison, report templating — with human-readable inputs and outputs. Every step is recorded in a transparent reasoning trace, followed by a faithfulness verification step that checks whether each report claim has visual evidence.
  • Results (on CT-RATE, 50,188 chest CT volumes):
  • | Metric | CT-CHAT | RadAgent | |---|---|---| | macro-F1 | 0.287 | 0.347 | | micro-F1 | 0.312 | 0.366 | | Faithfulness | 0% | 37.0% | | Adversarial robustness | — | +24.7 points |

    The recall improvement matters most clinically: fewer missed findings. The 37% faithfulness is a from-zero-to-one breakthrough for verifiability, though 63% of claims remain unverifiable.

  • Limitations: faithfulness still has large headroom; the tool set is chest-CT specific; stepwise reasoning is much slower than end-to-end inference — a real issue in time-critical settings like the emergency room.
  • The key insight: error cost determines design philosophy

    The comparison boils down to one question: what does failure cost?

  • A bad webpage is an aesthetic disaster — redo it. So MM-WebAgent can pursue "overall harmony" through iterative, tolerant reflection.
  • A wrong diagnosis can be fatal — no redo. So RadAgent must be step-by-step correct and fully explainable, functioning as an automated, mandatory checklist.
  • Neither is "better"; different scenarios demand different philosophies.

    Feynman-style reflections

  • Naming is not understanding: both papers say "Agent," but one is a creative coordinator and the other a reasoning executor. "Agent" is a spectrum, not a unified concept — and the term is suffering inflation as any LLM program with function calling gets labeled an agent.
  • Organization over novelty: neither system invents new AI capabilities. Both automate an existing human workflow (designer's top-down process; radiologist's protocol) by organizing LLM reasoning, generation tools, and verification into structured pipelines — arguably the most underrated skill in current AI engineering.
  • Open questions: Is MM-WebAgent's lack of learning a flaw or a feature that preserves diversity (and avoids baking in "AI slop" taste)? Is 37% faithfulness enough? Not as a final diagnosis, but valuable as a triage tool that directs physician attention.
  • Conclusion

    Both systems share the same skeleton — a central LLM brain, specialized tools, and a feedback loop — differing only in what feedback means: "does this look coherent?" versus "is this conclusion supported by evidence?" Future agents will likely be hybrids: creative yet constrained, intelligent yet transparent. As Feynman put it: "For a successful technology, reality must take precedence over public relations, for nature cannot be fooled." Neither paper is a revolution; both are solid, honest progress.

    References

  • MM-WebAgent: arXiv 2604.15309, Yan Li et al., Microsoft Research Asia
  • RadAgent: arXiv 2604.15231, Mélanie Roschewitz et al., Imperial College London & Stanford

Tags

#ai-agents#paper-comparison#multimodal#web-generation#medical-imaging#chest-ct#explainability#paper-slam

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/177618864