OpAI-Bench: An Operation-Guided Benchmark for Multi-Granularity AI Text Detection in Progressive Human-AI Co-Editing

Forum topic · 小凯 · 2026-06-06

Summary

OpAI-Bench is an operation-guided benchmark introduced by Sondos Mahmoud Bsharat, Jiacheng Liu, and Xiaohan Zhao (arXiv:2506.08272, June 2025) for studying AI text detection in progressive human-AI co-editing scenarios. As AI writing assistants become embedded in real-world drafting workflows, many documents are no longer purely human-written or purely AI-generated but the result of iterative mixed editing. Existing AI text detection benchmarks focus mainly on final outputs and offer little insight into how AI authorship signals emerge, accumulate, or disappear across revision steps. OpAI-Bench addresses this gap by evaluating detectability across multiple granularities: document, sentence, token, and span. Experiments show that AI text detectability depends not only on the proportion of AI-edited content but also on the types of editing operations, the domain, and the accumulated revision history. The benchmark provides a foundation for building more robust detectors suited to realistic, partially AI-assisted writing.

Overview

Research area: NLP Authors: Sondos Mahmoud Bsharat, Jiacheng Liu, Xiaohan Zhao Published: 2025-06-11 arXiv: 2506.08272

Abstract Summary

As AI writing assistants become increasingly integrated into real-world drafting and revision workflows, many documents are no longer purely human-written or purely AI-generated — they are the result of progressive human-AI co-editing. However, existing AI text detection benchmarks focus mainly on final outputs, offering little understanding of how AI authorship signals emerge, accumulate, or disappear during the revision process.

The authors introduce OpAI-Bench, an operation-guided benchmark for studying progressive human-to-AI text transformation across multiple granularities:

Document level
Sentence level
Token level
Span level

Key Findings

Experiments reveal that AI text detectability depends not only on the proportion of AI-edited content, but also on:

The type of editing operations applied
The domain of the text
The accumulated revision history

Original Abstract (English)

> As AI writing assistants become increasingly integrated into real-world drafting workflows, many documents result from progressive human-AI co-editing. We introduce OpAI-Bench, an operation-guided benchmark for studying progressive human-to-AI text transformation across document, sentence, token, and span granularities.

---

*Auto-collected on 2025-06-11*