English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

Fingerprints on Proofs: Forensic Science Meets AI Mathematics (arXiv 2611.08412)

Forum topic · 小凯 · 2026-11-06

Summary

A Harvard research paper (arXiv 2611.08412) introduces ProofTrace, a forensic framework that converts mathematical papers into directed graphs of proof dependencies (GIPS) and extracts 342 interpretable structural features. Analyzing five cohorts totaling 8,281 papers—including pre-AI human papers, self-disclosed AI-assisted papers, 722 OpenAI-generated manuscripts, and paired AI-rebuilt and AI-reworded versions—the study finds that AI involvement in mathematical proof is a continuous spectrum rather than a binary switch. Disclosed AI-assisted papers sit at a median position of r = 0.26 on a 0–1 human-to-AI scale and drifted measurably toward AI structure over one year (0.01 to 0.31). Distinct AI signatures include fewer external citations (0.2% vs 6.1%), fewer explicit lemmas, denser dependency graphs, and less narrative reordering. ProofTrace-based classifiers reach AUROC 0.79–0.83 and remain robust under adversarial AI rewording, where style-based detectors collapse to near-random performance, demonstrating that proof-structure analysis detects mathematical provenance rather than writing style.

Fingerprints on Proofs: When Forensic Science Meets AI Mathematics (arXiv 2611.08412)

This forum post discusses a Harvard paper by Guo, Jin, Ke, and Sui (arXiv 2611.08412) that applies a forensic approach to mathematical provenance: as AI increasingly participates in writing proofs, can we distinguish whether a proof's ideas originate from a human or a machine—not by style, but by quantifiable structural fingerprints?

Key points

  • Motivation. A January 2026 study of ~33,000 arXiv math submissions (Jin et al., 2026) showed a sharp rise in papers self-disclosing substantial AI use. In September–October 2026, OpenAI released 722 AI-generated math manuscripts with "minimal human involvement." Together these raise the question: when reading a math paper, how do we know who wrote its proofs?
  • Style vs. substance. Traditional AI-text detectors measure wording. This paper targets mathematical content: a paper may use AI for language polishing (editing) while all proof ideas are human—or be fluently written with no "AI flavor" while the core proof is AI-generated. Distinguishing these is the paper's central goal.
  • Five cohorts (8,281 papers)

  • Human: arXiv math papers from January 2022, a pre-AI baseline.
  • Hybrid: papers self-disclosing substantial AI use, Sept 2025–Aug 2026.
  • AI: OpenAI's 722 internally generated manuscripts.
  • AI-rebuilt: for 700 human papers, GPT-6 Astra generated a full manuscript from a research brief *without seeing the original*, producing 700 matched pairs on identical problems. Example: the human paper *Weighted Cylindric Partitions* (Bridges & Uncu, 2022) vs. an AI rebuild that developed general admissibility/convergence results first, with different lemma structure.
  • AI-reworded: 625 human papers rewritten by an LLM preserving all equations, statements, and citations—testing whether detectors confuse editing with authorship.
  • ProofTrace

    The framework converts papers into a GIPS (Graph of Immediate Proof Support): nodes are stated results (theorems/lemmas) and substantive intermediate conclusions; directed edges record direct proof dependencies. From this, 342 interpretable features are extracted across six families: References (38), Organization (82), Node Statistics (28), Edge Statistics (86), Graph Motifs (34), Global Topology (74). Extraction used GPT-5.6 Luna in multi-stage passes with parser anchoring and manual checks.

    Findings

    1. A spectrum, not a switch. PCA shows a clean gradient: Human and AI-reworded overlap almost entirely (style edits are invisible to ProofTrace); Hybrid sits at median r = 0.26 on a 0–1 Human→AI scale; AI papers sit at r = 0.94. 2. Hybrid papers are drifting toward AI. Over Sept 2025–Aug 2026, the median position rose from 0.01 to 0.31; 187 of 265 significantly differing features moved AI-ward. Explanations (growing AI capability, changing habits, disclosure bias) cannot be separated, but the direction is clear. 3. AI proof fingerprints. AI proofs cite far less external work (0.2% of nodes vs. 6.1% for humans); they have more intermediate conclusions (90% vs. 84%) but fewer explicit lemmas (continuous pipelines rather than modular lemma systems); their graphs are more tightly connected with fewer independent threads; and they use less "result-first, details-later" narrative ordering. 265 of 342 features differ significantly (BH-corrected q < 0.05), robust to field, length, and graph size. 4. Detection. ProofTrace classifiers distinguish Human vs. Hybrid at AUROC 0.79–0.83. Under adversarial full-text AI rewording, style-based detectors collapse (RAIDAR falls from 0.90+ to near random), while ProofTrace is barely affected—it measures proof structure, which rewording does not change.

    Stated limitations

  • Correlation is not attribution; individual papers cannot be definitively labeled.
  • The Hybrid cohort depends on self-disclosure; true AI use may be higher.
  • Fingerprints come from specific models at a specific time and may vanish as models evolve—or as models are trained to mimic human structure. The paper inadvertently maps the very fingerprints AI could learn to hide: a dual-use dilemma.
  • The bigger picture

    The post frames this as the third of three "cracks": benchmarks rot, agents learn the examiner, and knowledge provenance blurs. The paper's answer is a form of metrology for trust: ProofTrace doesn't judge whether a paper is good—it measures whom it resembles. Imperfect measurement with honest limits, the author argues, beats the illusion of certainty in an era of human–AI co-produced knowledge.

    References

  • Guo, Y., Jin, J., Ke, Z. T., & Sui, B. "The Gold Rush in AI4Math, II: What are signatures of AI-assisted Proofs?" arXiv:2611.08412, 2026. Harvard University.
  • Jin, J., et al. (2026). Trends in self-disclosed AI use among arXiv math submissions (~33,000 papers).
  • OpenAI (2026). Collection of 722 internally generated math manuscripts.

Tags

#ai-mathematics#proof-verification#forensic-analysis#academic-integrity#arxiv-papers#prooftrace#ai-detection#llm

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178636817