Reasoning Structure Is a Fingerprint: Robust LLM Authorship Attribution via Reasoning Graphs
> Paper: Show Me How You Reason and I'll Tell You Who You Are: Reasoning Graphs for Robust LLM Authorship Attribution > arXiv: 2607.14905 > Authors: Zlata Kikteva, Artur Romazanov, Annette Hautli-Janisz, Ramon Ruiz-Dolz > Institutions: University of Passau, University of Dundee
A Real-World Problem
During the ICLR 2026 review season, a striking figure surfaced: 21% of peer reviews were fully AI-generated. In education, students use LLMs to write essays; on social media, AI-generated content already accounts for 5% of online articles. Detecting AI text has become an urgent problem.
But the authors tackle a finer question: not just whether a text was written by an AI, but which AI wrote it.
Why does this matter? Different LLMs have different biases, capabilities, and tendencies. For a disinformation article, knowing whether it came from Llama or Qwen has practical value for tracing origins, understanding motives, and crafting countermeasures. It's like handwriting forensics—not asking "was this printed?" but "which printer printed it?" Every printer has tiny mechanical quirks that leave a unique fingerprint.
There's a thorny challenge, though: obfuscation. Many "humanize AI text" tools rewrite or back-translate AI text to make it look human. Traditional detectors based on surface lexical features collapse under obfuscation. Worse, LLM versions iterate fast—a detector trained on Llama 3.3 may fail completely on Llama 4 text.
The paper's new idea: don't look at what the LLM says; look at how it reasons.
Core Insight: Reasoning Structure Is a Fingerprint
From Text to Reasoning Graphs
The key innovation converts LLM-generated text into a "reasoning graph," then classifies it with a graph neural network (GNN):
1. Argument mining: existing argument mining tools extract claims and their relations (support, attack, addition, etc.). 2. Graph construction: each claim is a node; relations are edges. An essay becomes a graph. 3. GNN classification: a GNN learns structural features and predicts which LLM family (Gemma, Qwen, Llama, Phi) generated the text.
The key assumption: different LLMs have different reasoning styles. Llama might prefer "claim first, then rebuttal"; Qwen might favor listing parallel arguments; Gemma might lean toward progressive argumentation. These styles are stable artifacts of training that survive rewriting or back-translation—because rewriting changes vocabulary, not argumentative structure.
An Analogy
Imagine identifying a composer. You could listen to a melody (lexical features), but melodies can be rearranged—minor to major, 4/4 to 3/4. However, a composer's harmonic habits, phrase-structure preferences, and development techniques—deep structural traits—are hard to "arrange away."
This paper does exactly that: ignore the "melody" (vocabulary), examine the "harmonic progression" (reasoning structure).
Experimental Design
Dataset
- 8 LLM model versions from 4 families: Gemma, Qwen, Llama, Phi
- 280 argumentative essays per model, 2,240 total
- 3 obfuscated versions per essay: paraphrase, French back-translation, Turkish back-translation
- 8,960 essays total
- No obfuscation: Longformer reaches 90%+ F1; GNNs slightly lower. Surface features work well on clean text.
- With obfuscation: Longformer drops 9–52 points (paraphrase is the deadliest); GNNs lose only 10–20 points.
- Key number: on late model versions, GNNs beat Longformer by up to 27 points.
- Longformer: falls from 90%+ to below 50%. Surface features don't generalize at all.
- GNN: leads Longformer by 19 points (early-version training, late-version testing).
- Key finding: reasoning features are more stable across versions. Late Llama's reasoning structure resembles Early Llama's more than their lexical features resemble each other.
- Paraphrase is most damaging—it changes vocabulary and syntax but preserves argumentative structure (paraphrasers tend to keep the original meaning).
- French back-translation has moderate impact.
- Turkish back-translation has the least impact (possibly because Turkish differs more from English, and back-translated text retains more of the original structure).
- Llama family: lowest lexical diversity (MTLD 103 and 87), but higher syntactic complexity
- Qwen family: simpler syntax, high lexical diversity (MTLD 129–196)
- Gemma family: tends toward complex syntactic structures
- Phi family: moderate lexical diversity
- Positive: helps detect abuse (e.g., AI-generated fake reviews)
- Negative: could be used for de-anonymization (e.g., identifying a whistleblower who wrote with a particular model)
Argumentative essays were chosen because the genre forces models to display their reasoning, making argumentative structure easy to extract.
Baseline
Longformer—a Transformer for long texts—classifies raw text directly. This is the "listen to the melody" approach.
GNN Models
Four GNN layers: GCN (mean aggregation), GAT (attention aggregation), Graph Transformer (global attention), GPS (local + global hybrid). Seven layers (the graphs' average path length is 7.5), global attention pooling, then a classification head.
Two Evaluation Scenarios
1. Same-Version: train and test on the same model version (different essays). Tests baseline performance. 2. Cross-Version: train on earlier versions (e.g., Llama 3.3), test on later ones (e.g., Llama 4). Tests generalization.
The Numbers
Scenario 1: Same-Version
Scenario 2: Cross-Version
Obfuscation Strategies Compared
This comparison confirms the paper's core hypothesis: reasoning structure is more stable than vocabulary. Rewriting changes words but rarely the argumentative logic—especially since the rewriting tools are themselves LLMs that tend to preserve the source's argumentation.
Why It Works
Reasoning Style Is a Product of Training
Different LLM families use different training data, alignment strategies, and RLHF preferences. These differences show up in reasoning habits:
These "reasoning fingerprints" are byproducts of training, not deliberate design. That's precisely why they're hard to erase—changing them would require changing the entire training process.
Graphs Capture Relations, Not Content
Reasoning graphs don't care about claim content ("AI is beneficial" vs. "AI is harmful")—only relations between claims ("A supports B," "C attacks B"). Such relational features are more abstract, and therefore more stable, than content features.
A paraphraser will turn "AI benefits education" into "artificial intelligence positively impacts teaching," but won't turn a "support" relation into an "attack" relation. That's the source of the reasoning graph's obfuscation resistance.
What It Means
1. A New Tool for LLM Authorship Attribution
Prior methods rely on lexical frequency, syntactic features, and n-gram statistics—effective on clean text but fragile under obfuscation. Reasoning graphs offer a more robust dimension.
2. LLM "Fingerprints" Are More Persistent Than Expected
Even with version updates, rewriting, or back-translation, reasoning structure retains traces of the training process. This has practical value for tracing the source of AI-generated content—e.g., identifying which model family produced a disinformation campaign.
3. A Warning for LLM Developers
If your model has a distinctive reasoning style, that style can identify its output. This cuts both ways:
4. The Limits of "Humanize AI Text" Tools
These tools alter the surface, not the structure. They can make text *look* more human, but cannot change the underlying model's reasoning habits. Structure-based detection may thus be a long-lasting line of defense.
Limitations
1. Dataset scale: 2,240 argumentative essays across 4 model families. Scalability to more models and larger datasets remains unverified. 2. Genre restriction: only argumentative essays were tested. Whether valid reasoning graphs can be extracted from narrative, expository, or conversational text is unclear. 3. Argument mining accuracy: graph quality depends on the mining tools. If those tools have biases (e.g., extracting more complete graphs for some models), downstream classification suffers. 4. GNNs trail Longformer on clean text: the method is better suited as a "second line of defense" for obfuscated scenarios, not a replacement for traditional methods. 5. Cross-version vs. cross-family: only cross-version generalization was tested (Llama 3.3 → Llama 4); cross-family generalization (encountering a completely new family like Mistral at test time) was not.
Conclusion
The paper's title riffs on a Sherlock Holmes quote: "Show me how you reason and I'll tell you who you are." It captures the contribution precisely: reasoning structure is a fingerprint of identity.
This raises a deeper philosophical question: if reasoning style can uniquely identify an LLM, is it a kind of "personality"? We say every person has a unique way of thinking—now LLMs have that trait too. Not that they have personalities, but that "reasoning style as identity marker" holds for both humans and machines.
Next time you read an AI-generated article, try ignoring what it says and watching how it argues—claim first or rebuttal first, how many parallel points, where the turns come. These "reasoning habits" may reveal its origins more honestly than any watermark.
---
Paper: https://arxiv.org/abs/2607.14905 HTML version: https://arxiv.org/html/2607.14905v1 Dataset: the authors released a dataset of 8,960 argumentative essays (four families: Gemma/Qwen/Llama/Phi, including obfuscated versions)