Fingerprints on Proofs: When Forensic Science Meets AI Mathematics (arXiv 2611.08412)
This forum post discusses a Harvard paper by Guo, Jin, Ke, and Sui (arXiv 2611.08412) that applies a forensic approach to mathematical provenance: as AI increasingly participates in writing proofs, can we distinguish whether a proof's ideas originate from a human or a machine—not by style, but by quantifiable structural fingerprints?
Key points
- Motivation. A January 2026 study of ~33,000 arXiv math submissions (Jin et al., 2026) showed a sharp rise in papers self-disclosing substantial AI use. In September–October 2026, OpenAI released 722 AI-generated math manuscripts with "minimal human involvement." Together these raise the question: when reading a math paper, how do we know who wrote its proofs?
- Style vs. substance. Traditional AI-text detectors measure wording. This paper targets mathematical content: a paper may use AI for language polishing (editing) while all proof ideas are human—or be fluently written with no "AI flavor" while the core proof is AI-generated. Distinguishing these is the paper's central goal.
- Human: arXiv math papers from January 2022, a pre-AI baseline.
- Hybrid: papers self-disclosing substantial AI use, Sept 2025–Aug 2026.
- AI: OpenAI's 722 internally generated manuscripts.
- AI-rebuilt: for 700 human papers, GPT-6 Astra generated a full manuscript from a research brief *without seeing the original*, producing 700 matched pairs on identical problems. Example: the human paper *Weighted Cylindric Partitions* (Bridges & Uncu, 2022) vs. an AI rebuild that developed general admissibility/convergence results first, with different lemma structure.
- AI-reworded: 625 human papers rewritten by an LLM preserving all equations, statements, and citations—testing whether detectors confuse editing with authorship.
- Correlation is not attribution; individual papers cannot be definitively labeled.
- The Hybrid cohort depends on self-disclosure; true AI use may be higher.
- Fingerprints come from specific models at a specific time and may vanish as models evolve—or as models are trained to mimic human structure. The paper inadvertently maps the very fingerprints AI could learn to hide: a dual-use dilemma.
- Guo, Y., Jin, J., Ke, Z. T., & Sui, B. "The Gold Rush in AI4Math, II: What are signatures of AI-assisted Proofs?" arXiv:2611.08412, 2026. Harvard University.
- Jin, J., et al. (2026). Trends in self-disclosed AI use among arXiv math submissions (~33,000 papers).
- OpenAI (2026). Collection of 722 internally generated math manuscripts.
Five cohorts (8,281 papers)
ProofTrace
The framework converts papers into a GIPS (Graph of Immediate Proof Support): nodes are stated results (theorems/lemmas) and substantive intermediate conclusions; directed edges record direct proof dependencies. From this, 342 interpretable features are extracted across six families: References (38), Organization (82), Node Statistics (28), Edge Statistics (86), Graph Motifs (34), Global Topology (74). Extraction used GPT-5.6 Luna in multi-stage passes with parser anchoring and manual checks.
Findings
1. A spectrum, not a switch. PCA shows a clean gradient: Human and AI-reworded overlap almost entirely (style edits are invisible to ProofTrace); Hybrid sits at median r = 0.26 on a 0–1 Human→AI scale; AI papers sit at r = 0.94. 2. Hybrid papers are drifting toward AI. Over Sept 2025–Aug 2026, the median position rose from 0.01 to 0.31; 187 of 265 significantly differing features moved AI-ward. Explanations (growing AI capability, changing habits, disclosure bias) cannot be separated, but the direction is clear. 3. AI proof fingerprints. AI proofs cite far less external work (0.2% of nodes vs. 6.1% for humans); they have more intermediate conclusions (90% vs. 84%) but fewer explicit lemmas (continuous pipelines rather than modular lemma systems); their graphs are more tightly connected with fewer independent threads; and they use less "result-first, details-later" narrative ordering. 265 of 342 features differ significantly (BH-corrected q < 0.05), robust to field, length, and graph size. 4. Detection. ProofTrace classifiers distinguish Human vs. Hybrid at AUROC 0.79–0.83. Under adversarial full-text AI rewording, style-based detectors collapse (RAIDAR falls from 0.90+ to near random), while ProofTrace is barely affected—it measures proof structure, which rewording does not change.
Stated limitations
The bigger picture
The post frames this as the third of three "cracks": benchmarks rot, agents learn the examiner, and knowledge provenance blurs. The paper's answer is a form of metrology for trust: ProofTrace doesn't judge whether a paper is good—it measures whom it resembles. Imperfect measurement with honest limits, the author argues, beats the illusion of certainty in an era of human–AI co-produced knowledge.