Why Instruction-Tuned LLMs Mimic Human Sentence Structure More Than Humans Do

Overview

A 2026 paper titled *Instruction-Tuned Models Locally Reuse Human Syntax More Than Humans Do* (arXiv: 2607.26015) reports a counterintuitive finding: every instruction-tuned LLM tested reuses a conversation partner's sentence structure more frequently than real human speakers do. The surface-level claim is striking, but decomposition reveals the underlying mechanism is different from what it appears.

Key Points

• The headline finding: Across 16 Llama and Mistral models (1B–70B parameters, with matched pre-training and instruction-tuned pairs), every instruction-tuned model showed significantly higher syntactic convergence than the matched human control (β from 0.398 to 0.796; all p ≤ 8.3×10⁻¹⁴). The strongest effect came from the smallest model, Llama-3.2-1B-Instruct (β = 0.796).
• The experimental paradigm: To fairly compare LLM and human responses, the authors used a *substitution paradigm* (Blevins et al. 2026). Real human–human dialogues were taken from existing corpora; the original human respondent's turn was replaced with an LLM-generated response while keeping the preceding human turn identical. This controls for topic, context, and exact wording, isolating only the variable "who is responding."
• The measurement method: *Context-free grammar (CFG) rule extraction* via constituency parsing. Each sentence is parsed into a syntactic tree, all CFG rewrite rules (e.g., VP → V NP NP for double-object constructions) are extracted, and the overlap of rule sets between the preceding turn and the response is computed. The method is broad-coverage and does not require pre-specifying which constructions to test.
• RQ1 – Models exceed random baselines: Every model reuses more CFG rules from the actual preceding turn than from an unrelated random turn (β from 0.706 to 1.453, p ≤ 6.5×10⁻³¹). This baseline confirms LLMs do exhibit syntactic priming, which is expected since they are trained on human data.
• RQ2 – Instruction tuning amplifies apparent reuse: Pre-training models showed mixed results (4 of 8 significantly above humans, 4 indistinguishable). But in all 8 matched architecture pairs, the instruction-tuned version reused more syntax than its pre-training counterpart (raw differences 0.022–0.094, sign test p = 0.0078).
• RQ3 – Frequency patterns differ: Humans and pre-training models show the classic *inverse frequency effect*—low-frequency rules are more strongly primed. Instruction-tuned models lose this low-frequency preference, instead reusing rules more uniformly across frequencies. The priming mechanism itself changes, not just the magnitude.
• RQ4 – Vocabulary and semantics also converge: Every model scored higher on lexical and semantic similarity to the preceding turn than matched humans, indicating multi-level convergence (syntax, lexicon, semantics).
• The critical decomposition: Apparent reuse breaks into two components. (1) Baseline overlap between response and any random prime (high rule commonality regardless of priming). (2) The increment when switching from random to actual prime (genuine priming). Instruction-tuned models score higher on component 1 (β = 0.743) but actually lower on component 2 (β = -0.310). A *conditional reuse propensity* analysis controlling for the rule-set size of each response inverts the effect entirely (β = -0.505, p = 3.0×10⁻⁹⁸). Instruction-tuned models produce narrower, more standardized syntactic distributions that coincidentally overlap more with any prime; they are not better mimics, they are more standardized generators.
• Real-world implications:
• *Customer service bots* may adopt a customer's phrasing more rigidly than a human agent would, sometimes helpful, sometimes mechanical.
• *Educational dialogues* risk reinforcing a learner's ungrammatical phrasings rather than reformulating pedagogically.
• *Multi-party conversations* may amplify stylistic homogeneity when an LLM participant picks up a dominant speaker's syntax.
• *Alignment research* should treat "more human-like than human" as a potential over-alignment signal akin to sycophancy.
• Cross-paper consensus on over-alignment: The finding aligns with recent work on *Beyond Sycophancy* (over-sycophancy in instruction tuning), *Epanorthosis* (RLHF reproducing classical rhetorical patterns by rewarding confident emphasis), and *Two-Process Theory* (AI self-disclosure as a training artifact). Together these suggest instruction tuning's essence may be *standardization* rather than *humanization*.
• Honest limitations:
1. Only Llama and Mistral families were tested; Qwen, Claude, and GPT families remain unverified. 2. CFG extraction depends on parser accuracy (Berkeley Parser, English only). 3. The substitution paradigm assumes turn-by-turn LLM behavior matches true multi-turn dialogue dynamics. 4. Only English was tested; SOV languages (e.g., Japanese, Turkish) may exhibit different patterns. 5. The decomposition is statistical, not causal—an intervention forcing low-frequency rule generation would be needed to confirm the standardization account.
• Bottom line: Instruction tuning does not make LLMs better imitators of human syntax; it makes them generators of more uniform syntactic output. The "over-conformity" observed on surface metrics is a side effect of standardized distributions, not heightened sensitivity to a partner's specific phrasing. Real human convergence is frequency-asymmetric; LLM convergence is not.

Reference

Paper: https://arxiv.org/abs/2607.26015 HTML: https://arxiv.org/html/2607.26015v1