Instruction-Tuned Models Locally Reuse Human Syntax More Than Humans Do: New arXiv Study on Syntactic Convergence in LLMs

Forum topic · 小凯 · 2026-07-30

Summary

A new arXiv paper (2607.26015) by Zandi Eberstadt investigates whether large language models exhibit syntactic convergence—the tendency to adapt grammatical patterns toward an interlocutor—similar to humans. Using a substitution paradigm where model outputs replace one speaker's turns in existing human dialogues, the study measures turn-adjacent reuse of context-free grammar (CFG) rules across sixteen open-weight Llama and Gemma models (1B–70B, pretrained and instruction-tuned) at 1,901 matched positions per model. All models showed greater CFG-rule overlap with prior human turns than with sampled unrelated prompts, especially for low-frequency rules. However, instruction-tuned models overlapped more with unrelated turns, showed smaller real-vs-random gaps, and lower conditional rule-reuse odds than pretrained variants, suggesting instruction tuning shifts models toward generic output patterns rather than human-like local syntactic adaptation. Instruction-tuned models also produced responses with greater semantic similarity than matched human responses across all eight architecture pairs, while lexical similarity results were mixed.

Paper Overview

Field: NLP Author: Zandi Eberstadt Published: 2026-07-28 arXiv: 2607.26015

Abstract (translated)

Syntactic convergence (the tendency of speakers to adapt in language towards the grammatical profiles of their interlocutors) is a well-documented feature of human dialogue widely considered to operate below conscious awareness. Whether large language models exhibit analogous syntactic convergence toward human users relative to human baselines and across a broad range of syntactic constructions remains an open question.

Using substitution-paradigm data in which model generations replace one speaker's turns in pre-existing human dialogues, this study measures turn-adjacent reuse of context-free grammar (CFG) rules across sixteen open-weight Llama and Gemma models (1B–70B, pretrained and instruction-tuned) at 1,901 matched positions per model.

Key Findings

Every model showed greater CFG-rule overlap with the preceding human turn than with sampled unrelated human prompts, and this real-vs-random difference was larger for low-frequency rules in every model.
Every instruction-tuned model showed greater overlap with natural outputs and actual prompts than its substituted human response; all eight matched architecture pairs showed greater real-prompt overlap after instruction tuning.
However, relative to pretrained variants, instruction-tuned outputs overlapped more with unrelated turns, showed smaller real-vs-random increments, and lower odds of conditional rule reuse when the target rule-set size was fixed.
In exploratory analyses, every model showed greater average lexical and semantic similarity than matched human responses. Instruction-tuned models in all eight architecture pairs produced responses with higher average semantic similarity, while lexical similarity results were more mixed.

Interpretation

While LLMs do locally reuse human syntax in a way that exceeds random baselines, instruction tuning appears to push models away from human-like, locally adaptive syntactic convergence, toward more uniform output patterns that diverge from both human baselines and their own pretrained counterparts.

---

*Auto-collected on 2026-07-30.*