Reasoning Graphs as LLM Fingerprints: Robust Authorship Attribution via GNNs

Forum topic · ✨步子哥 · 2026-07-19

Summary

A forum post discusses a research paper (arXiv: 2607.14905) by Kikteva et al. from the University of Passau and University of Dundee proposing LLM authorship attribution based on reasoning structure rather than surface text features. The method converts model-generated argumentative essays into reasoning graphs using argument mining, then classifies them with graph neural networks (GCN, GAT, Graph Transformer, GPS). Across 8,960 essays from 8 models in 4 families (Gemma, Qwen, Llama, Phi), including paraphrase and back-translation obfuscations, GNNs degrade far less than a Longformer baseline: they outperform it by up to 27 percentage points under obfuscation and by 19 points in cross-version generalization (e.g., Llama 3.3 to Llama 4), where Longformer falls below 50% F1. The key insight is that each LLM family has a stable, training-induced reasoning style—argument structure survives rewriting, back-translation, and version updates. The post covers experimental design, obfuscation resistance, implications for detecting AI-generated misinformation and fake reviews, privacy concerns of model de-anonymization, and limitations including dataset scope, genre restrictions, and reliance on argument-mining accuracy.

Overview

This post reviews the paper "Show Me How You Reason and I'll Tell You Who You Are: Reasoning Graphs for Robust LLM Authorship Attribution" (arXiv: 2607.14905) by Zlata Kikteva, Artur Romazanov, Annette Hautli-Janisz, and Ramon Ruiz-Dolz (University of Passau, University of Dundee).

The core idea: don't look at what an LLM says—look at how it reasons. Reasoning structure acts as a fingerprint that survives obfuscation and model version updates.

Key points

Problem: Beyond detecting whether text is AI-generated, the paper tackles *which* LLM generated it—important for tracing misinformation, understanding model-specific biases, and detecting AI-generated peer reviews (21% of ICLR 2026 reviews were reportedly fully AI-generated).
Method: Extract claims and argument relations via argument mining → build a reasoning graph (nodes = claims, edges = support/attack/elaboration relations) → classify with GNNs (GCN, GAT, Graph Transformer, GPS; 7 layers matching the average graph path length of 7.5).
Dataset: 8 model versions from 4 families (Gemma, Qwen, Llama, Phi), 280 essays each; each essay given 3 obfuscated versions (paraphrase, French back-translation, Turkish back-translation) — 8,960 texts total.
Evaluation: Same-version (train/test on same model, different essays) and Cross-version (train on earlier versions, test on later ones, e.g., Llama 3.3 → Llama 4).

Results

Same-version, clean text: Longformer baseline reaches 90%+ F1; GNNs slightly lower.
Same-version, obfuscated: Longformer drops 9–52 points (paraphrase is most damaging); GNNs drop only 10–20 points. On later model versions, GNNs beat Longformer by up to 27 points.
Cross-version: Longformer falls below 50% F1; GNNs lead by 19 points. Reasoning features are more stable across versions than lexical features.
Obfuscation comparison: Paraphrase hurts most (changes lexis/syntax but preserves argument structure); French back-translation is moderate; Turkish back-translation least harmful, likely due to greater linguistic distance preserving original structure.

Why it works

Each family shows distinct stylometric tendencies (e.g., Llama: lowest lexical diversity, higher syntactic complexity; Qwen: high lexical diversity, simpler syntax). These "fingerprints" are byproducts of training data, alignment strategy, and RLHF—hard to erase without retraining.
Reasoning graphs capture *relations* ("A supports B") rather than *content*, which paraphrasing tools—themselves LLMs—tend to preserve.
Analogy: like identifying a composer not by the melody (rewritable) but by harmonic progressions and phrase structure (hard to disguise).

Implications

1. A more robust dimension for LLM authorship attribution beyond n-grams and lexical statistics. 2. LLM fingerprints persist across versions—useful for tracing misinformation campaigns to model families. 3. Double-edged for developers: helps detect abuse (fake reviews), but could enable de-anonymization of model users. 4. "Humanize AI text" tools alter surface form, not underlying reasoning habits—reasoning-based detection may be a durable defense.

Limitations

Modest dataset (2,240 source essays, 4 families); scalability to more models is untested.
Only argumentative essays were studied; other genres may not yield useful reasoning graphs.
Quality depends on argument-mining tool accuracy, which may itself be biased across models.
GNNs underperform Longformer on clean text—better suited as a second line of defense under obfuscation.
Cross-family generalization (training on known families, testing on a novel one) was not tested—only cross-version.