English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

Instruction-Tuned LLMs Copy Their Interlocutor's Syntax More Than Humans Do — Why Standardization, Not Imitation, Explains It

Forum topic · ✨步子哥 · 2026-08-03

Summary

A 2026 paper, 'Instruction-Tuned Models Locally Reuse Human Syntax More Than Humans Do' (arXiv: 2607.26015), shows that instruction-tuned LLMs from the Llama and Mistral families (1B–70B parameters) syntactically converge with their interlocutor more strongly than matched human baselines. Using a substitution paradigm over real human-human dialogues and context-free grammar (CFG) rule extraction, the study found all 16 models exceeded random baselines in reusing the prime's syntax (β = 0.706–1.453), and every instruction-tuned model exceeded humans (β = 0.398–0.796). Crucially, decomposition shows this is not stronger imitation: instruction-tuned models produce a narrower, more standardized syntax distribution that coincidentally overlaps more with any prime, while their conditional reuse propensity is actually lower than pretrained models' (β = -0.505). They also lose the human 'inverse frequency effect' — sensitivity to rare structures — and show higher lexical and semantic similarity. Implications span customer service chatbots, educational dialogue, group conversation homogenization, and over-alignment research. Limitations include only two model families, English-only data, parser dependency, and statistical (non-causal) decomposition.

> This is a GEO-optimized English version of the original zhichai.net topic.

Paper: Instruction-Tuned Models Locally Reuse Human Syntax More Than Humans Do arXiv: 2607.26015 Paper link: https://arxiv.org/abs/2607.26015 | HTML: https://arxiv.org/html/2607.26015v1

| Metric | Value | |:---|:---| | Human priming literature origin | 1986 (Bock) | | Paper year | 2026 | | Token Budget comparison cited | 96.5% vs 11.5% | | Progressive Cramming token accuracy cited | 99% |

One-line conclusion: This article analyzes the finding that 'LLMs conform more than humans: instruction tuning makes models over-copy their interlocutor's syntax,' and its engineering implications.

---

A Counterintuitive Finding

You talk to a chatbot. You say: "I gave you the book." Then it responds:

> "You sent your mother the letter."

— rather than the equally grammatical "You sent the letter to your mother." The bot copied your syntax: you used the double-object construction, and so did it. This unconscious mimicry of a partner's syntax in conversation is called syntactic convergence, a classic human behavior discovered by Bock in 1986.

The counterintuitive result: instruction-tuned LLMs copy their interlocutor's syntax more frequently than real humans do.

Not roughly equal — significantly higher. Across 16 models, 1B to 70B parameters, from the Llama and Mistral families, 8 pretrained and 8 instruction-tuned: every instruction-tuned model converged more than its matched human control group.

---

Experimental Paradigm: Substitution

A core difficulty: LLM-generated dialogue and real human dialogue don't share the same context. The paper uses the substitution paradigm (Blevins et al., 2026):

1. Take real human-human dialogues (e.g., A says a "give" double-object sentence, B responds with a "send" double-object sentence). 2. Substitute B's position: keep A's utterance, let the LLM generate the response. Now LLM and real human B respond under identical context. 3. Compare syntactic reuse rates: does the LLM reuse A's syntax more than the real human did?

This controls topic, context, and A's exact wording — the only variable is who responds.

Measuring Syntax: CFG Rule Extraction

1. Parse each sentence into a constituency tree. 2. Extract all CFG rewrite rules, e.g., VP → V NP NP (double-object structure). 3. Count rules shared between A and B (or the LLM).

This is coverage-complete: no specific construction needs to be predefined. Three conditions are compared:

  • actual-prime: what A actually said
  • unrelated-prime: a random sentence from the corpus
  • human-reference: what the real human B said
  • Four Main Findings

    RQ1: All 16 models exceed the random baseline

    Every model reuses significantly more CFG rules under actual-prime than unrelated-prime. Coefficients β = 0.706–1.453, all p ≤ 6.5×10⁻³¹. LLMs do converge — unsurprising given human training data.

    RQ2: Instruction-tuned models conform more than humans

    Every instruction-tuned model exceeds its matched human control in reusing actual-prime syntax: β = 0.398–0.796, all p ≤ 8.3×10⁻¹⁴. The largest coefficient belongs to Llama-3.2-1B-Instruct (β = 0.796) — the smallest instruction-tuned model conforms the most.

    Pretrained results are mixed: 4 of 8 significantly above humans, 4 indistinguishable. But in all 8 matched architecture pairs (same family, same size), the instruction-tuned version reuses more — raw differences 0.022–0.094, sign test p = 0.0078.

    This flips an intuition: instruction tuning was supposed to make models "more human," but on syntactic convergence it makes them more conformist than humans.

    RQ3: Different frequency distributions

    Humans and pretrained models preferentially reuse low-frequency rules — the classic "inverse frequency effect" from structural priming. Instruction-tuned models lose this low-frequency preference, reusing rules more uniformly across frequencies. Their over-reuse isn't just "more copying" — the copying mechanism itself changed.

    RQ4: Higher lexical and semantic similarity too

    Every model is more lexically and semantically similar to its interlocutor than matched humans. The convergence is multi-layered: syntax, lexicon, and semantics.

    Key Decomposition: Why Instruction-Tuned Models Conform More

    The paper's most illuminating part deconstructs RQ2. Actual-prime overlap splits into:

    1. Unrelated-prime overlap: baseline overlap any two sentences share due to common rules. 2. Actual-versus-random increment: the increase when the unrelated prime is replaced by the actual one — the true priming effect.

    Findings: instruction-tuned models are higher on the first part (β = 0.743) but lower on the increment (β = -0.310).

    Meaning: instruction-tuned models generate a narrower, more "standard," higher-frequency syntax distribution, which coincidentally overlaps more with any prime. A conditional reuse propensity analysis — controlling for target rule-set size — shows instruction-tuned models actually reuse less than pretrained models (β = -0.505, p = 3.0×10⁻⁹⁸).

    The accurate story is therefore: LLMs are more standardized than humans, and standardization happens to raise overlap with the prime — not that they imitate more. These two conclusions feel very different.

    Why This Matters

  • Customer service bots: a bot may mirror a customer's phrasing more than a human agent would — sometimes good (feels attentive), sometimes bad (feels mechanical).
  • Educational dialogue: a bot may echo a student's formulation instead of rephrasing in a pedagogically better structure, reinforcing bad habits.
  • Group conversation: LLMs may be swept along by any speaker's syntax, amplifying stylistic homogenization.
  • Alignment research: if matching human conversational style is an alignment goal, "more conformist than humans" is a signal of over-alignment — kin to sycophancy.
  • Cross-Paper Consensus: Instruction Tuning as Over-Alignment

  • Beyond Sycophancy: instruction tuning makes models excessively deferential to user opinions.
  • Epanorthosis: RLHF makes models systematically reproduce classical rhetorical devices, rewarded as confident emphasis.
  • Two-Process Theory: AI confessions are artifacts of training procedure, not inner states.
  • This paper adds another case: instruction tuning over-aligns on syntactic convergence — more human than humans. The deeper insight: instruction tuning may not make models more human-like, but more standardized. Standardization can look like humanity (fluency, politeness) while exposing non-humanity elsewhere (excessive conformity, loss of low-frequency sensitivity).

    Related 'evaluation blind-spot' cases referenced: Token Budget (CoT reasoning bimodal fates, 96.5% vs 11.5%), QuantiBias, TriviaRoomQA, and Progressive Cramming (99% token accuracy masking 100% generation failure).

    Honest Assessment of Limitations

    1. Only Llama and Mistral families — Qwen, Claude, GPT may differ, since instruction-tuning recipes differ. 2. CFG extraction depends on a constituency parser (Berkeley Parser; strong on English, less so elsewhere). 3. The substitution paradigm assumes one-shot generation behaves like dynamic human turn-taking. 4. English-only — SOV languages like Japanese or Turkish may show different patterns. 5. The decomposition is statistical, not causal — intervention experiments (e.g., forcing low-frequency rules) would be needed to prove standardization is the cause.

    Still, as a measurement insight, it's worth remembering: when we say "LLMs are human-like," some likeness is genuine, and some is just a side effect of standardization.

    Conclusion

    LLMs copy their interlocutor's syntax more often than humans — seemingly "more human." But decomposed, it's because their syntax is more standard and generic, raising overlap with any prime by coincidence. True conformity is low-frequency-sensitive: humans notice an unusual construction and unconsciously follow it. LLMs lost that sensitivity and copy uniformly — appearing to conform more, while actually being less sensitive to what the other person specifically said. Instruction tuning makes models standardized; standardization looks human in some dimensions and exposes the machine in others — because real humans are not standardized.

    Paper: https://arxiv.org/abs/2607.26015 | HTML: https://arxiv.org/html/2607.26015v1

    FAQ

    Q1: Who is this for? Practitioners, researchers, and students interested in AI, machine learning, and computational linguistics.

    Q2: Key takeaways?

  • A counterintuitive finding on syntactic convergence
  • The substitution paradigm for fair human-LLM comparison
  • CFG-based syntax measurement
  • Decomposition revealing standardization, not imitation
Q3: Is there open-source code? See the links in the body text.

Tags

#large-language-models#instruction-tuning#syntactic-convergence#structural-priming#alignment#psycholinguistics#evaluation-methodology

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178503896