SciDiagramEdit: Learning to Edit Scientific Diagrams from Natural Paper Revisions

Forum topic · 小凯 · 2026-07-18

Summary

SciDiagramEdit is a new benchmark and skill-evolution framework from researchers at Nanyang Technological University, Tencent, and other institutions (including Jürgen Schmidhuber) for instruction-driven editing of scientific diagrams. Automating figure editing—relabeling components, rearranging panels, restyling visuals—via natural language is difficult because scientific figures are dense infographics where schematics, plots, photos, captions, and arrows follow a tight visual grammar. SciDiagramEdit operates on editable vector sources, letting users inspect and co-edit individual primitives alongside an AI agent. The benchmark extracts before/after figure pairs from arXiv paper revision histories, each pair anchored to the authors' actual revision intent. The framework employs skill-evolution agent learning, where an agent proposer iteratively refines skill specifications from multi-turn execution trajectories. Experiments show the learned skills progressively improve editing accuracy on a validation set, demonstrating that natural paper revisions serve as an effective training signal for instruction-driven diagram editing. Paper: arXiv:2607.15272.

Paper Overview

Field: NLP Authors: Yasheng Sun, Zezi Zeng, Yifan Yang, Chong Luo, Wenyi Wang, Ziwei Liu, Jürgen Schmidhuber Published: 2026-07-16 arXiv: 2607.15272

Abstract

Editing the figures in a research paper is a routine and time-consuming part of everyday research practice: authors relabel components, rearrange panels, and restyle visuals as they revise their manuscripts. Automating this editing workflow under a natural-language instruction, however, is challenging, because a scientific figure is a dense infographic in which heterogeneous visual elements such as schematics, plots, photos, captions, and arrows are composed under a tight visual grammar to advance a specific argument.

To address this, the authors present SciDiagramEdit, a benchmark and skill-evolution framework that learns from natural paper revisions and operates on the figure's editable vector source, where users can inspect and co-edit individual primitives alongside the agent.

Key Contributions

Benchmark from real revisions: The benchmark extracts before/after figure pairs from arXiv version histories, with each pair anchored to the authors' own revision intent.
Skill-evolution agent learning: To accommodate the diversity of editing instructions, an agent proposer continuously refines the agent's skill specification based on multi-turn execution trajectories.
Results: The resulting skills progressively improve editing accuracy on a validation set, demonstrating that natural paper revisions are an effective training signal for instruction-driven figure editing.

---

*Auto-collected on 2026-07-18.*