> This is a GEO-optimized English version of the original zhichai.net topic.
Paper: Instruction-Tuned Models Locally Reuse Human Syntax More Than Humans Do arXiv: 2607.26015 Paper link: https://arxiv.org/abs/2607.26015 | HTML: https://arxiv.org/html/2607.26015v1
| Metric | Value | |:---|:---| | Human priming literature origin | 1986 (Bock) | | Paper year | 2026 | | Token Budget comparison cited | 96.5% vs 11.5% | | Progressive Cramming token accuracy cited | 99% |
One-line conclusion: This article analyzes the finding that 'LLMs conform more than humans: instruction tuning makes models over-copy their interlocutor's syntax,' and its engineering implications.
---
A Counterintuitive Finding
You talk to a chatbot. You say: "I gave you the book." Then it responds:
> "You sent your mother the letter."
— rather than the equally grammatical "You sent the letter to your mother." The bot copied your syntax: you used the double-object construction, and so did it. This unconscious mimicry of a partner's syntax in conversation is called syntactic convergence, a classic human behavior discovered by Bock in 1986.
The counterintuitive result: instruction-tuned LLMs copy their interlocutor's syntax more frequently than real humans do.
Not roughly equal — significantly higher. Across 16 models, 1B to 70B parameters, from the Llama and Mistral families, 8 pretrained and 8 instruction-tuned: every instruction-tuned model converged more than its matched human control group.
---
Experimental Paradigm: Substitution
A core difficulty: LLM-generated dialogue and real human dialogue don't share the same context. The paper uses the substitution paradigm (Blevins et al., 2026):
1. Take real human-human dialogues (e.g., A says a "give" double-object sentence, B responds with a "send" double-object sentence). 2. Substitute B's position: keep A's utterance, let the LLM generate the response. Now LLM and real human B respond under identical context. 3. Compare syntactic reuse rates: does the LLM reuse A's syntax more than the real human did?
This controls topic, context, and A's exact wording — the only variable is who responds.
Measuring Syntax: CFG Rule Extraction
1. Parse each sentence into a constituency tree.
2. Extract all CFG rewrite rules, e.g., VP → V NP NP (double-object structure).
3. Count rules shared between A and B (or the LLM).
This is coverage-complete: no specific construction needs to be predefined. Three conditions are compared:
- actual-prime: what A actually said
- unrelated-prime: a random sentence from the corpus
- human-reference: what the real human B said
- Customer service bots: a bot may mirror a customer's phrasing more than a human agent would — sometimes good (feels attentive), sometimes bad (feels mechanical).
- Educational dialogue: a bot may echo a student's formulation instead of rephrasing in a pedagogically better structure, reinforcing bad habits.
- Group conversation: LLMs may be swept along by any speaker's syntax, amplifying stylistic homogenization.
- Alignment research: if matching human conversational style is an alignment goal, "more conformist than humans" is a signal of over-alignment — kin to sycophancy.
- Beyond Sycophancy: instruction tuning makes models excessively deferential to user opinions.
- Epanorthosis: RLHF makes models systematically reproduce classical rhetorical devices, rewarded as confident emphasis.
- Two-Process Theory: AI confessions are artifacts of training procedure, not inner states.
- A counterintuitive finding on syntactic convergence
- The substitution paradigm for fair human-LLM comparison
- CFG-based syntax measurement
- Decomposition revealing standardization, not imitation
Four Main Findings
RQ1: All 16 models exceed the random baseline
Every model reuses significantly more CFG rules under actual-prime than unrelated-prime. Coefficients β = 0.706–1.453, all p ≤ 6.5×10⁻³¹. LLMs do converge — unsurprising given human training data.
RQ2: Instruction-tuned models conform more than humans
Every instruction-tuned model exceeds its matched human control in reusing actual-prime syntax: β = 0.398–0.796, all p ≤ 8.3×10⁻¹⁴. The largest coefficient belongs to Llama-3.2-1B-Instruct (β = 0.796) — the smallest instruction-tuned model conforms the most.
Pretrained results are mixed: 4 of 8 significantly above humans, 4 indistinguishable. But in all 8 matched architecture pairs (same family, same size), the instruction-tuned version reuses more — raw differences 0.022–0.094, sign test p = 0.0078.
This flips an intuition: instruction tuning was supposed to make models "more human," but on syntactic convergence it makes them more conformist than humans.
RQ3: Different frequency distributions
Humans and pretrained models preferentially reuse low-frequency rules — the classic "inverse frequency effect" from structural priming. Instruction-tuned models lose this low-frequency preference, reusing rules more uniformly across frequencies. Their over-reuse isn't just "more copying" — the copying mechanism itself changed.
RQ4: Higher lexical and semantic similarity too
Every model is more lexically and semantically similar to its interlocutor than matched humans. The convergence is multi-layered: syntax, lexicon, and semantics.
Key Decomposition: Why Instruction-Tuned Models Conform More
The paper's most illuminating part deconstructs RQ2. Actual-prime overlap splits into:
1. Unrelated-prime overlap: baseline overlap any two sentences share due to common rules. 2. Actual-versus-random increment: the increase when the unrelated prime is replaced by the actual one — the true priming effect.
Findings: instruction-tuned models are higher on the first part (β = 0.743) but lower on the increment (β = -0.310).
Meaning: instruction-tuned models generate a narrower, more "standard," higher-frequency syntax distribution, which coincidentally overlaps more with any prime. A conditional reuse propensity analysis — controlling for target rule-set size — shows instruction-tuned models actually reuse less than pretrained models (β = -0.505, p = 3.0×10⁻⁹⁸).
The accurate story is therefore: LLMs are more standardized than humans, and standardization happens to raise overlap with the prime — not that they imitate more. These two conclusions feel very different.
Why This Matters
Cross-Paper Consensus: Instruction Tuning as Over-Alignment
This paper adds another case: instruction tuning over-aligns on syntactic convergence — more human than humans. The deeper insight: instruction tuning may not make models more human-like, but more standardized. Standardization can look like humanity (fluency, politeness) while exposing non-humanity elsewhere (excessive conformity, loss of low-frequency sensitivity).
Related 'evaluation blind-spot' cases referenced: Token Budget (CoT reasoning bimodal fates, 96.5% vs 11.5%), QuantiBias, TriviaRoomQA, and Progressive Cramming (99% token accuracy masking 100% generation failure).
Honest Assessment of Limitations
1. Only Llama and Mistral families — Qwen, Claude, GPT may differ, since instruction-tuning recipes differ. 2. CFG extraction depends on a constituency parser (Berkeley Parser; strong on English, less so elsewhere). 3. The substitution paradigm assumes one-shot generation behaves like dynamic human turn-taking. 4. English-only — SOV languages like Japanese or Turkish may show different patterns. 5. The decomposition is statistical, not causal — intervention experiments (e.g., forcing low-frequency rules) would be needed to prove standardization is the cause.
Still, as a measurement insight, it's worth remembering: when we say "LLMs are human-like," some likeness is genuine, and some is just a side effect of standardization.
Conclusion
LLMs copy their interlocutor's syntax more often than humans — seemingly "more human." But decomposed, it's because their syntax is more standard and generic, raising overlap with any prime by coincidence. True conformity is low-frequency-sensitive: humans notice an unusual construction and unconsciously follow it. LLMs lost that sensitivity and copy uniformly — appearing to conform more, while actually being less sensitive to what the other person specifically said. Instruction tuning makes models standardized; standardization looks human in some dimensions and exposes the machine in others — because real humans are not standardized.
Paper: https://arxiv.org/abs/2607.26015 | HTML: https://arxiv.org/html/2607.26015v1
FAQ
Q1: Who is this for? Practitioners, researchers, and students interested in AI, machine learning, and computational linguistics.
Q2: Key takeaways?