Instruction-Tuned LLMs Converge to Human Syntax More Than Humans Do — But Standardization, Not Imitation, Is the Cause

Forum topic · ✨步子哥 · 2026-07-29

Summary

A 2026 arXiv paper (2607.26015) reports that instruction-tuned LLMs reuse their conversation partner's syntactic structures more frequently than humans do. Using a substitution paradigm on real human-human dialogues and context-free grammar rule extraction, the authors tested 16 models (Llama and Mistral families, 1B-70B parameters, 8 pretrained and 8 instruction-tuned). Every instruction-tuned model significantly exceeded its matched human control in syntactic convergence, while pretrained results were mixed (4 of 8 above humans). A deeper decomposition reveals the twist: instruction-tuned models produce narrower, more 'standard' syntactic distributions, which trivially overlap more with any prime; conditional on output distribution, their true reuse propensity is actually lower than pretrained models (β = -0.505). They also lose the human 'inverse frequency effect' — sensitivity to rare constructions. The post discusses implications for chatbots, education, and alignment (over-convergence as an over-alignment signal akin to sycophancy), plus limitations: only two model families, English-only, and statistical rather than causal decomposition.

**Paper: *Instruction-Tuned Models Locally Reuse Human Syntax More Than Humans Do* arXiv: 2607.26015 | https://arxiv.org/abs/2607.26015 HTML version: https://arxiv.org/html/2607.26015v1

A Counterintuitive Finding

When a chatbot replies to "I gave you the book," it tends to respond with a double-object construction like "You sent your mother the letter" rather than the equally grammatical "You sent the letter to your mother" — it mirrors the interlocutor's syntax. This phenomenon, known as syntactic convergence, was discovered in humans by Bock's classic 1986 experiments. The surprise: instruction-tuned LLMs do it more than real humans do. Across 16 models (1B–70B parameters, Llama and Mistral families, 8 pretrained and 8 instruction-tuned), every instruction-tuned model reused its interlocutor's syntax more than a matched human control.

Method: The Substitution Paradigm

To place LLMs and humans in identical contexts, the paper (building on Blevins et al. 2026) uses a substitution paradigm:

1. Take real human-human dialogues where speaker B's reply shares syntax with A's utterance. 2. Replace B: keep A's utterance and let the LLM generate the reply in the exact same context. 3. Compare how much LLM vs. real human B responses reuse A's syntax.

Syntax is measured via CFG rule extraction: parse each sentence into a constituency tree, extract rewrite rules (e.g., VP → V NP NP), and count rules shared with the prime. Three conditions are compared: actual-prime (A's real utterance), unrelated-prime (a random utterance), and human-reference (the real human B's reply).

Four Main Findings

RQ1: All 16 models exceed the random baseline. Every model reuses more CFG rules under actual-prime than unrelated-prime (β from 0.706 to 1.453, all p ≤ 6.5×10⁻³¹).

RQ2: All instruction-tuned models exceed humans (β 0.398–0.796, p ≤ 8.3×10⁻¹⁴); the largest effect is Llama-3.2-1B-Instruct (β = 0.796). Pretrained models are mixed (4 of 8 above humans), but in all 8 matched architecture pairs, the instruction-tuned version reuses more (raw differences 0.022–0.094, sign-test p = 0.0078).

RQ3: Different frequency profiles. Humans and pretrained models preferentially reuse low-frequency rules (the classic "inverse frequency effect"); instruction-tuned models lose this preference and reuse rules uniformly across frequencies.

RQ4: Lexical and semantic similarity are also higher than matched humans — convergence is multi-level.

The Key Decomposition: Standardization, Not Imitation

The paper's sharpest insight comes from decomposing actual-prime reuse into two parts:

1. Unrelated-prime overlap: rules shared even with a random prime (baseline overlap due to common constructions). 2. Actual-versus-random increment: the increase when the random prime is swapped for the actual one — the true priming effect.

Instruction-tuned models score higher on the first part (β = 0.743) but lower on the second (β = -0.310). Their outputs contain more "generic," high-frequency constructions, so they overlap with any prime more often. A conditional reuse propensity analysis confirms this: controlling for the target rule-set size, instruction-tuned models' reuse tendency is actually below pretrained models (β = -0.505, p = 3.0×10⁻⁹⁸).

So the accurate story is not "LLMs imitate more than humans" but: instruction tuning makes models more standardized, and standardization trivially inflates surface overlap with the prime. The models are less, not more, sensitive to what specific construction the interlocutor used.

Why It Matters

Customer service: bots may mirror customer phrasing more mechanically than human agents.

Education: a tutor bot may echo a student's erroneous phrasing instead of re-expressing it pedagogically.

Group dialogue: LLMs may amplify syntactic homogenization in multi-party conversations.

Alignment: "more human than humans" convergence is an over-alignment signal, kin to sycophancy. The finding joins a broader pattern (Beyond Sycophancy, Epanorthosis) suggesting instruction tuning's essence is standardization** — which can look like human-likeness (fluency, politeness) while exposing non-humannness elsewhere (excessive convergence, loss of low-frequency sensitivity).

Limitations

1. Only two model families (Llama, Mistral); results for Qwen, Claude, GPT are unknown. 2. CFG extraction depends on constituency parser accuracy (English-tuned). 3. The substitution paradigm assumes one-shot generation matches in-conversation behavior. 4. English-only; SOV languages may behave differently. 5. The decomposition is statistical, not causal — intervention experiments (e.g., forcing low-frequency rules) are needed to confirm the standardization account.

Conclusion

Humans' convergence is sensitive to rare constructions; instruction-tuned LLMs converge uniformly. They appear "more conformist than humans," but the mechanism is narrower output distributions, not stronger imitation. When we say an LLM is "human-like," some of that likeness is real, and some is merely a side effect of standardization.

Paper: https://arxiv.org/abs/2607.26015