SCAFFOLD: A Large-Scale Structured Dataset of Computer Science Research Figures for Diagram QA and Chain-of-Thought Reasoning

Forum topic · 小凯 · 2026-09-03

Summary

SCAFFOLD is a newly presented large-scale structured dataset of computer science research diagrams, designed to train vision-language models to understand figures such as architecture drawings, system flowcharts, and pipeline schematics. The dataset is built from arXiv computer science papers using layout detection and PDF parsing, with an AI-assisted question-generation step. Each sample is a tuple of (image, caption, context, question-answer, chain-of-thought), pairing diagrams with questions, answers, and step-by-step reasoning traces — a combination no prior public dataset provided. The release includes three scales: SCAFFOLD-157K with 157,387 pairs drawn from 29,887 figures across 3,058 papers, SCAFFOLD-37K with 36,797 pairs, and SCAFFOLD-12K with 12,000 pairs. The authors ran baseline experiments using SCAFFOLD-12K on Qwen2.5-VL-3B-Instruct. The paper (arXiv:2509.00007) falls in the computer vision research area.

Overview

Field: Computer Vision Authors: Ranjit Raut, Aarav Subedi, Sagun Rai Published: 2026-09-03 arXiv: 2509.00007

Abstract

Computer science papers rely heavily on diagrams: architecture drawings, system flowcharts, and pipeline schematics that often carry more information than the text around them. There is currently no public dataset that pairs this specific kind of figure with captions, context, questions, answers, and step-by-step reasoning, which is exactly what is needed to train a vision-language model to understand them.

We present SCAFFOLD, a large-scale structured dataset of computer science research figures with diagram QA and Chain-of-Thought reasoning traces. The dataset consists of (image, caption, context, question-answer, chain-of-thought) tuples from arXiv computer science papers, prepared using layout detection and PDF parsing, with an AI-assisted question-generation step.

Dataset Scales

SCAFFOLD-157K: 157,387 pairs, covering 29,887 figures from 3,058 papers
SCAFFOLD-37K: 36,797 pairs (medium scale)
SCAFFOLD-12K: 12,000 pairs (small scale)

The authors conducted baseline experiments on Qwen2.5-VL-3B-Instruct using SCAFFOLD-12K.

---

*Auto-collected on 2026-09-03*