SciDiagramEdit: Learning to Edit Scientific Diagrams from Natural Paper Revisions

Forum topic · 小凯 · 2026-07-18

Summary

SciDiagramEdit is a benchmark and skill-evolution framework for instruction-driven editing of scientific figures, presented in a new arXiv paper (2607.15272) by researchers including Yasheng Sun, Ziwei Liu, and Jürgen Schmidhuber. The work targets a routine but time-consuming research task: relabeling components, rearranging panels, and restyling visuals when revising manuscripts. Automating this via natural-language instructions is difficult because scientific figures are dense infographics combining schematics, plots, photos, captions, and arrows under a strict visual grammar. SciDiagramEdit operates on editable vector sources, allowing users to inspect and co-edit individual primitives alongside the agent. The benchmark extracts before/after figure pairs from arXiv version histories, with each pair anchored to the authors' actual revision intents. The framework employs skill-evolution-based agent learning, in which an agent proposer iteratively refines the agent's skill specifications using multi-round execution trajectories. Experiments show that the evolved skills progressively improve editing accuracy on a validation set, demonstrating that natural paper revisions serve as an effective training signal for instruction-driven diagram editing.

Paper Overview

Field: NLP
Authors: Yasheng Sun, Zezi Zeng, Yifan Yang, Chong Luo, Wenyi Wang, Ziwei Liu, Jürgen Schmidhuber
arXiv: 2607.15272

Translation of the Chinese Abstract

Editing figures in a paper is a routine and time-consuming part of everyday research practice: authors relabel components, rearrange panels, and restyle visuals as they revise their manuscripts. Automating this editing workflow under natural-language instructions is challenging, however, because a scientific figure is a dense infographic in which heterogeneous visual elements—schematics, plots, photos, captions, and arrows—are composed under a strict visual grammar to advance a specific argument.

To address this, the authors present SciDiagramEdit, a benchmark and skill-evolution framework that learns from natural paper revisions and operates on the figure's editable vector source, where users can inspect and co-edit individual primitives alongside the agent.

Key components:

Benchmark construction: before/after figure pairs are extracted from arXiv version histories, with each pair anchored to the authors' own revision intents.
Skill-evolution agent learning: to accommodate the diversity of editing instructions, an agent proposer continuously refines the agent's skill specifications based on multi-round execution trajectories.
Results: the evolved skills progressively improve editing accuracy on a validation set, showing that natural paper revisions are an effective training signal for instruction-driven figure editing.

Original Abstract (excerpt)

> Editing the figures in a research paper is a routine and time-consuming part of everyday research practice: authors relabel components, rearrange panels, and restyle visuals as they revise their manuscripts. Automating this editing workflow under a natural-language instruction, however, is challenging, because a scientific figure is a dense infographic in which heterogeneous visual elements such as schematics, plots, photos, captions, and arrows are composed under a tight visual grammar to advance a specific argument. To address this, we present SciDiagramEdit, a benchmark and skill-evolution framework that learns from natural paper revisions and operates on the figure's editable vector source, where users can inspect and co-edit individual primitives alongside the agent. Our benchmark min...

*Auto-collected on 2026-07-18.*