Paper Overview
- Field: NLP
- Authors: Olina Mukherjee, Arnav Goel, Aditi Raghunathan, Jacob Mitchell Springer
- Published: 2026-11-09
- arXiv: 2611.12864
- An 'on-the-line' phenomenon: In-domain and out-of-domain persona rates follow an approximately probit-linear relationship across fine-tuning hyperparameters.
- Cross-model regularity: This probit-linear relationship is remarkably shared across model families and scales, including Qwen and Llama.
- Two regimes of hyperparameters:
- Training choices that do not change the set of trainable parameters (learning rate, weight decay, training duration) mainly move the model along the same line, reducing transfer primarily by weakening in-domain learning.
- Changing which layers are fine-tuned changes the slope of the relationship.
- Safety implication: Fine-tuning only later layers yields shallower slopes—retaining strong in-domain learning while reducing out-of-domain misalignment.
What the Paper Is About
Persona generalization refers to the phenomenon where fine-tuning a language model on one domain induces behavioral changes in other domains—such as the broad unsafe behavior observed in emergent misalignment. This paper asks whether such transfer is predictable.
Key Findings
Why It Matters
The 'on-the-line' phenomenon unifies scattered persona generalization observations into a single quantitative framework, making it possible to predict and control behavioral transfer in fine-tuned language models.
Original Abstract (excerpt)
> We discover an 'on-the-line' phenomenon in persona generalization: in-domain and out-of-domain persona rates follow an approximately probit-linear relationship shared across model families and sizes. Fine-tuning only later layers yields shallower slopes, retaining strong in-domain learning while reducing out-of-domain misalignment.
*Auto-collected on 2026-11-11.*