Paper Overview
- Field: NLP
- Authors: Andy Liu, Mehar Bhatia, Karolina Stanczak, Mona Diab, Vered Shwartz, Daniel Fried
- Published: 2026-10-08
- arXiv: 2610.12410
- Representations based on model activations when a value is applied in context significantly outperform methods based on textual value descriptions.
- The best activation-based method achieves a correlation of 0.45 with the generalization matrix, versus only 0.05 for description-based baselines.
- The representations used for predicting alignment generalization transfer to downstream tasks: using them to measure value similarity in multi-value alignment targets correlates significantly with model robustness.
- Preliminary evidence suggests a shared, model-agnostic value space, which the authors use to build the first LLM value taxonomy based on empirical generalization dynamics.
Summary
LLM developers post-train their models to exhibit prosocial values and behavioral traits, which are enumerated in an alignment target. However, while recent post-training developments have yielded models that score highly on alignment evaluations, training models on sets of narrow behaviors still influences their behavior across unseen contexts and environments in unexpected ways.
This paper establishes the task of alignment generalization prediction — predicting how fine-tuning a model to follow a given value changes its behavior across a wide range of held-out values. The authors conduct a large-scale analysis of alignment generalization effects across 66 values found in modern alignment targets, and benchmark representational techniques on the prediction task.
Key Findings
Significance
The work highlights the importance of studying value generalization in LLMs and its application toward more empirical design and training of model behavior.
--- *Auto-collected on 2026-10-11*