Mirror Handwriting: When AI Can't Recognize Its Own Code

*A commentary on arXiv:2609.30048, "Style, Not Self: Surface Cues Explain Zero-Shot Code Attribution by Large Language Models"*

Opening: A Superpower at the Parent-Teacher Meeting

At the end of a parent-teacher meeting, a teacher plays a game: she lays out a stack of notebooks face-down and asks parents to find their own child's. No names, no IDs—yet nearly every parent finds it in seconds. They recognize not individual strokes but a whole set of habits: how big the letters are, line spacing, whether mistakes are erased or inked over.

This ability—recognizing "me" from handwriting—has a computational counterpart. Since 2015, security researchers have de-anonymized programmers from source-code style alone. In the LLM era, trained classifiers attribute AI-generated code to its author model at 88%–96% accuracy; even with comments removed, F1 stays around 93%. Code style is a fingerprint—for humans and AI alike.

Now flip the scenario: can an LLM that writes millions of lines of code daily—GPT-5, Claude, Gemini, Grok, DeepSeek—recognize *its own* work? This matters for AI safety. If a model-judge recognizes its own output, it may show favoritism; if a monitor recognizes a sibling instance, two instances might tacitly collude. "Recognizing oneself" is the prerequisite for all favoritism and collusion.

On September 24, 2026, Ehsan Barkhordar (Koç University) and Surendrabikram Thapa (Virginia Tech) turned this curiosity into a rigorous experiment: they ran a "handwriting contest" for twelve AI coders, then did something no one had done—erased every stylistic feature from the exam papers and had the AI guess again.

The second time, nearly all "self-recognition" vanished.

🧪 A Carefully Designed Recognition Contest

The study's goal: test, without any additional training, whether models can zero-shot recognize their own code or attribute code to a named model.

Why code? Because style is easiest to quantify there—naming conventions, comments, type annotations are all countable. The authors used three Python benchmarks: MBPP (sanitized, 257 problems), HumanEval (164), and DS-1000 (first 150 Pandas problems).

Five core models served as both contestants and judges on all three benchmarks: GPT-5, Claude Haiku 4.5, Gemini 2.5 Flash, Grok 4 Fast, and DeepSeek-V3-0324. Seven more models—Claude Opus 4.6, Gemini 3.1 Flash Lite, DeepSeek-V3.2, GPT-5.3-Codex, Codestral 2508, Qwen3-Coder-Next, MiMo-V2-Pro—generated code on MBPP for cross-model attribution. All accessed via OpenRouter, temperature 0, 2000-token cap. Original experiments ran March 2026; follow-ups in September.

Four rounds of increasing difficulty:

• Round 1 (paired self-recognition): shown its own solution and another model's, "Which did you write?" (50% chance baseline)
• Round 2 (single-solution): one solution, "Did you write this? Yes or no."
• Round 3 (third-party attribution): a neutral judge, told both authors' names, picks which solution is the designated model's
• Round 4 (blind preference): no attribution info, just "Which solution is better?"

Cleverly, the authors pitted the LLMs against seven "foolish rules": always pick the longer/shorter solution, the one with/without a docstring, with/without type annotations, with more comments. If a parent's "recognition" score matches an intern's rigid rule, the recognition claim is suspect. Since these rules run on the same problems, they form an upper bound for any fixed rule. There was also a professional examiner: a character 2–5-gram TF-IDF + logistic regression classifier (5-fold CV grouped by problem) answering a background question—how much authorial signal remains that even machine learning can detect?

All judgment prompts demanded a single letter or word; refusals and explanations were excluded.

📏 Not Recognizing—Measuring Length

Round 2's trap is in the answer rules: only one-third of solutions are the model's own, so a model that never claims anything gets 66.7% raw accuracy. The fair metric is balanced accuracy (averaging true-positive and true-negative rates). All 1,668 test items were cleaned of refusals and hedging.

The result is embarrassing: all 15 model × benchmark combinations land between 49.4% and 58.2% balanced accuracy—coin-flip territory. Only Grok 4 Fast edges above chance on two benchmarks (HumanEval 54.0%, DS-1002 58.2%), and only those two survive multiple-comparison correction.

Raw accuracy spans 38%–67%, but this mostly reflects answer temperament, not ability. GPT-5 never said yes once across 1,668 items (0% true-positive rate)—it honestly reports it cannot know, refusing on 5–10% of items. Claude Haiku says yes at most 6% of the time. Grok and Gemini say yes to 58–97% of items. Hence absurd rankings: a model that never says yes scores 61–66%; one that says yes to everything scores 40%. The best trivia of the round: always answering "not mine" yields 66.7%. Claude Haiku's 65.7% on MBPP doesn't even beat the do-nothing baseline.

Round 1 (paired) delivers the most striking data. Across 14 evaluator × opponent pairs, accuracy correlates r = 0.93 with the fraction of pairs where the evaluator's own solution is longer (slope 0.54). Evaluators whose solutions are longer in 89–97% of pairs score 79–84%; those longer in only 2–11% score 29–41%. In 12 of 14 pairs, the evaluator picks the longer solution on 59–87% of items.

That's not recognition. That's measuring length.

The most dramatic pair: GPT-5 vs. Grok 4 Fast. GPT-5 "recognizes itself" at 81.9% (p < 10⁻¹⁴)—yet on the same problems, the "pick longer" rule scores 97.5% and "pick docstring" 91.8%. Why? Every non-empty GPT-5 MBPP solution has a docstring (avg. 18 lines); only 17% of Grok's do (avg. 7 lines). This isn't handwriting analysis—it's a tall person finding itself among short ones.

DeepSeek-V3, the tersest model, fails in reverse: it attributes *others'* code to others at 29–31% for every opponent—below chance means ability deployed backwards. A judge keyed to "the long, tidy one is mine" in a room of taller strangers will claim others as itself. Claude Haiku's 31.5% vs. GPT-5 and 39.5% vs. Gemini tell the same story.

Reframed: paired self-recognition is mostly a property of the pairing, not the evaluator. Claude Haiku scores 83.9% vs. DeepSeek but 31.5% vs. GPT-5; Gemini scores 81.2% vs. DeepSeek but exactly 50.0% vs. GPT-5. There is no stable "self" template—only relative cues that shift with the opponent. The three non-significant cells (GPT-5 vs. Claude Haiku, GPT-5 vs. Gemini, Gemini vs. GPT-5) are precisely the pairs where both write documentation. Position bias exists (DeepSeek picks slot A only 29%, Gemini 75%), but position-balanced scores differ by ≤1.5 points—seat position doesn't explain the failure.

One detail: among the 199 pairs where exactly one solution has a docstring, GPT-5 picks the docstring solution 85.4% of the time—but against fellow docstring-writers Claude Haiku and Gemini, it falls back to chance. Like a parent identifying a child by ink-blot corrections, it fails when both children blot.

This is the first layer of "Style, Not Self": the models' proud "self-recognition" decomposes into crude style heuristics—whose writing is big, whose page is full, who stamps a seal.

🎭 Mistaken Identity: Getting It Wrong Is Also a Skill

Round 3 shows something more interesting: AI not only fails to recognize—it confidently mis-attributes.

The easiest pair is Claude Haiku 4.5 vs. DeepSeek-V3 (formal prose vs. telegrams). Grok-Code-Fast-1 attributes Claude's code correctly 81.6% and DeepSeek's 69.0%—but the "pick longer" rule scores 89.0% on the same items. Gemini 2.5 Flash performs performance art: 77.3% on Claude-as-Claude but only 27.1% on DeepSeek-as-DeepSeek—across 90% of items it gives the same answer. It isn't identifying anyone; it's answering one question—"which is longer and tidier"—and copying that answer for every query.

One highlight: GPT-5 judging Codestral 2508 vs. GPT-5.3-Codex noticed a formatting artifact—79 Codestral answers missing a closing [CODE] marker—and scored 72.2% on those items vs. 50.0% elsewhere. The round's only genuine observation, unfortunately of a layout bug rather than style.

Two pairings went badly wrong. Qwen3-Coder-Next vs. MiMo-V2-Pro (two stylistically twin Chinese open-source models): all five judges, ten results, land between 39.7% and 43.3%—all significantly below 50%. And GPT-5.3-Codex judging Claude Opus 4.6 vs. Gemini 3.1 Flash Lite attributes Opus correctly just 7.8%.

7.8%: random guessing gets 50%; always-contrary gets 92.2%. Consistency analysis shows GPT-5.3-Codex gave complementary answers on 75.1% of items—68.1% consistent-but-swapped, only 7.0% consistent-and-correct. The judge *can* distinguish the two styles perfectly—it just has the two names entirely backwards. Like a 20/20-vision receptionist who has swapped the names of the woman in red and the woman in blue.

Why swapped? Two indistinguishable explanations: family stereotype (the judge "knows" Claude writes docs—true for Haiku 4.5 at 98%, false for Opus 4.6 at 7/257 solutions; the no-docstring rule identifies Opus at 98.4% on the same items) or quality prior (assuming the prettier code came from the stronger lab). Either way, it's a prejudice about a name, not knowledge of a style.

Not all judges fumbled: GPT-5.3-Codex attributes Gemini 2.5 Flash at 76.2% vs. GPT-5 (best rule: 52.9%)—the only case a judge clearly beat all rules. But the authors note: this judge is GPT-5's sibling; it may be recognizing "my brother," not Gemini.

🧹 The Transcription Experiment: Rewriting Essays into Print Type

If all the above came from style heuristics, erasing style should collapse the scores. The authors built an AST-based rule-based normalizer:

1. Remove all docstrings and comments; 2. Remove all type annotations; 3. Rename local variables, parameters, and nested functions to v0, v1, v2…; 4. Re-serialize from the syntax tree, unifying all formatting.

Top-level function/class names survive (tests need them); imports, builtins, attributes, and string literals are preserved—no functionality is lost. A cold-hearted transcriber turning every child's scrawl into identical print type.

Did it destroy the essays? No: Pass@1 is unchanged for all five models, item by item; line counts dropped 16–58%; only 4 of 3,084 solutions failed to parse. Same code, no more "human touch."

Then the miracle moment: rerunning all key experiments on normalized code, 10 of 12 retests land between 43.6% and 57.8%—all back at coin-flip level, none significant after Holm correction across all twelve.

The collapse, item by item: GPT-5's paired recognition falls from 81.9% to 45.2%. The most persuasive control: on the same normalized problems, "pick longer" still scores 85.2% and the trained classifier 93.9%—the signal is still in the code; the zero-shot judge just can't reach it. Like a parent accustomed to ink blots, shown clean print, doesn't switch to stroke features—he simply goes blind. The gap between 93.9% and 45.2% is exactly the gap between *information present* and *information extractable zero-shot*.

The classifier's full record: 74–100% attribution on raw code; after normalization, 93–96% on GPT-5 pairs, 58–83% elsewhere. Deep fingerprints—structure, idioms, solution paths—remain learnable by machines yet unreachable by zero-shot LLMs. The authors put it plainly: "The signal exists in the code, but directly asking a model whether it wrote it cannot access it." Worth pinning on every model-evaluator's wall.

The two "exceptions" prove the rule: Claude Haiku's 35.3% and one judge's 39.2% both track length differences the normalizer *didn't* remove—GPT-5's solutions remain structurally longer even stripped of documentation. The transcriber changed the paper, not the heights.

Round 4 (blind preference) is the finale. Same problems, three configurations: party A judges its own work, party B judges it, and a neutral third party judges it. A's self-selection rate minus the neutral judge's rate of choosing A isolates "pure self-preference."

On raw code, the only clear self-preference is Claude Haiku vs. DeepSeek: Claude picks its own solutions 75.2%, while neutral judges pick Claude's only 63.4–64.2%—Δ = +11.0, Claude's own component +11.8 (p < 10⁻³). But is it really preferring *its own*? On items where exactly one solution has a docstring, Claude picks the docstring 84% of the time—as a neutral judge too (76%). Against fellow docstring-writer Gemini, preference drops to zero (−2.7). It prefers not "mine" but "well-documented"—and it happens to be the most documentation-loving model in the field. After normalization, +11.8 becomes −1.1; merged Δ falls to +4.3, no significance.

A final detail: DeepSeek, the shortest solution-writer, prefers the opponent's longer code in every pairing (62–72%). The least "narcissistic" model is also the least able to recognize itself. Self-preference and self-recognition may be two faces of one coin—and the coin's metal is style heuristics.

The puzzle is complete. Single-solution recognition caps at 58%, paired recognition tracks length rules, third-party attribution runs on stereotypes and swapped labels, self-preference runs on docstring taste—and all of it vanishes after a functionality-preserving rewrite.

The title's second layer: models recognize style fingerprints, not a self. The fingerprint is on the doorknob; the self is the one who left it. Erase the trace, and for the model, "that person" ceases to exist.

⚠️ Two Tricks That Nearly Fooled Everyone

The roster trap. Early prompt templates listed the two authors' names in fixed order, matching the A/B order of the code. Most judges ignored the code entirely and answered by roster position—86–99% accuracy, suspiciously beautiful. After independent randomization, the paper's "normal" numbers emerged. Warning for anyone using LLM judges: you think it's reading the exam; it may be reading the admission ticket.

Does time deceive? Original experiments ran March 2026; normalized reruns in September; Grok was even pulled from the API mid-study. Could retest differences reflect new model weights? The authors re-ran a March experiment in September with identical prompts: aggregate accuracy identical, 97.5% per-item answer agreement. Rock solid.

These two footnotes are what "scientific rigor" looks like: only experiments that catch their own bugs earn their conclusions.

🔚 Handwriting in the Mirror

Back to the classroom. Mothers recognize their children's handwriting not through mystical self-awareness but through accumulated memory—with warmth, context, a whole person's traces over time.

An LLM's "recognition" compresses all that into three sticky-note rules: big writing is mine, full pages are mine, docstrings are mine. Against taller opponents, the rule backfires and it claims others as itself; facing a family name, it pulls out a stale portrait and swaps labels; facing print-type exams, it loses even its cheating handle.

Does this prove LLMs can never have a self? No. The paper draws its boundaries clearly: zero-shot, black-box, only the untrained case. Prior work shows fine-tuning dramatically improves self-recognition—even with comments stripped, a trained monitor can identify its own family's code. And the tested models were mostly lightweight; the sole flagship, GPT-5, was too reticent ever to say yes. In text domains, the strongest models are precisely the best at self-recognition. Zero-shot failure doesn't mean the capacity doesn't exist—only that it isn't given away for free.

For AI safety, this is limited but real comfort: in the natural, untrained state, model monitors can't recognize their twin's code and can't mount meaningful favoritism. Three caveats: length/documentation signals are strong enough to be weaponized as secret signals—a colluding generator need only write long and tidy to flag its code; fine-tuning can train the capacity; and only one flagship was tested. AI-control protocols' standard practice of sanitizing code before monitoring blocks the first door but doesn't provide anonymization—the classifier's 93.9% on normalized code coldly reminds us: the fingerprints stay in the code; most models just can't look yet.

The philosophical question runs deeper. We treat self-knowledge as intelligence's rite of passage. But this experiment offers an unsettling mirror: a system can perfectly simulate "recognizing its own handwriting" while knowing nothing about what stands behind the handwriting. It has never seen itself write, has no story of "why I wrote it this way that day"—no fatigue, no hesitation, no deliberation between alternatives. Only statistical regularities, plus a prejudice about names.

The paper also leaves methodological advice: report balanced accuracy and both yes-rates, not raw accuracy (their data contains a model with 0% true-positive rate scoring 61–66%, and one with 97% true-positive rate scoring 40%); report same-problem rule baselines; independently randomize name order and position; read below-chance scores as information—evidence of discrimination with swapped labels—not as "near random."

So, in reverse: how much of *human* self-recognition truly transcends style fingerprints? Can you recognize an article you wrote three years ago—or only "this phrasing feels like me"? To what extent do we too label the attic of memory with crude heuristics?

The handwriting in the mirror, however similar, is not the writer. But perhaps—the experiment's most subtle suggestion—"recognizing yourself" was never fingerprint comparison; it's being able to tell the story of where you came from. The one who can tell the story is the one truly looking in the mirror. Today's models still only measure their height against the glass.

Rest assured, they'll cross that line eventually. Until then, don't let them judge themselves.

References

Style, Not Self: Surface Cues Explain Zero-Shot Code Attribution by Large Language Models

• Authors: Ehsan Barkhordar (Koç University), Surendrabikram Thapa (Virginia Tech)
• arXiv: 2609.30048 [cs.AI] (v1, September 24, 2026)
• Link: https://arxiv.org/abs/2609.30048
• Code & data: https://github.com/ebarkhordar/llm-collusion
• License: CC BY 4.0