OpAI-Bench: A Benchmark for Progressive Human-to-AI Text Transformation and Multi-Granularity AI-Text Detection

Forum topic · 小凯 · 2026-06-07

Summary

Researchers introduce OpAI-Bench, an operation-guided benchmark (arXiv:2606.06481) for studying how AI authorship signals emerge during progressive human-AI co-editing of documents. Starting from human-written texts, each sample is expanded into nine sequentially revised versions under predefined AI coverage levels and five representative AI edit operations, spanning four domains. The benchmark preserves complete authorship provenance at document, sentence, token, and span granularities, enabling evaluation of 8 document-level, 7 sentence-level, and 2 fine-grained token/span detectors. Experiments show that AI-text detectability depends not only on the proportion of AI-edited content but also on the edit operation type, domain, and cumulative revision history. Notably, mixed-authorship intermediate versions are often harder to detect than purely human or heavily AI-edited endpoints, revealing non-monotonic detection patterns that existing benchmarks overlook. OpAI-Bench provides a controlled testbed for analyzing when and how AI-assisted writing becomes detectable in realistic progressive editing scenarios, addressing a key gap since prior benchmarks focus mainly on final outputs.

Paper Overview

Field: NLP Authors: Sondos Mahmoud Bsharat, Jiacheng Liu, Xiaohan Zhao arXiv: 2606.06481

Key Points

As AI writing assistants become increasingly integrated into real-world drafting and revision workflows, many documents are no longer purely human-written or AI-generated, but result from progressive human-AI co-editing.
Existing AI-text detection benchmarks largely focus on final outputs and offer limited understanding of how AI authorship signals emerge, accumulate, or disappear throughout the revision process.
The authors introduce OpAI-Bench, an operation-guided benchmark for studying progressive human-to-AI text transformation across document, sentence, token, and span granularities.
Starting from human-written documents, OpAI-Bench constructs nine sequentially revised versions per sample under predefined AI coverage levels and five representative AI edit operations, covering four domains, while preserving full multi-granularity authorship provenance.
The benchmark supports comprehensive evaluation of 8 document-level, 7 sentence-level, and 2 fine-grained token/span detectors.

Findings

AI-text detectability is controlled not only by the proportion of AI-edited content but also by the edit operation, domain, and cumulative revision history.
Mixed-authorship intermediate versions are often harder to detect than purely human and heavily AI-edited endpoints, exposing non-monotonic detection patterns that existing benchmarks miss.
OpAI-Bench provides a controlled testbed for analyzing when and how AI-assisted writing becomes detectable in realistic progressive editing scenarios.