An Old Psychological Question
Is morality unified or categorized?
Jonathan Haidt's Moral Foundations Theory (MFT) says morality has six foundations: care/harm, fairness/cheating, liberty/oppression, loyalty/betrayal, authority/subversion, and sanctity/degradation. These are not six phrasings of the same thing but six independent "moral taste buds."
MFT further divides them into two groups:
- Individualizing foundations: care, fairness, liberty—protecting individual rights
- Binding foundations: loyalty, authority, sanctity—maintaining group order
- Paper: https://arxiv.org/abs/2608.27402
- HTML version: https://arxiv.org/html/2608.27402v1
- Code: https://github.com/orionrr/moral-geometry
- License: CC BY 4.0
This dichotomy has empirical support: on questionnaire data, individualizing foundations correlate highly with each other, binding foundations with each other, and the two groups correlate less.
What about large language models? After reading trillions of tokens of text, is a model's moral knowledge organized according to MFT's structure—or something else?
Method: Six Linear Probes + Geometric Analysis
The paper's approach is straightforward:
1. Train six independent linear probes—one per MFT foundation—on OLMo-2 1B and 7B, and OLMoE-1B-7B 2. Each probe is a direction vector indicating "where this foundation points in the model's representation space" 3. Examine the geometry among the six directions—do they merge, separate, or something else?
"Merging" would mean the six directions collapse into one "morality detector"—the model only knows "this is moral," without type distinctions. "Separation" would mean the six directions are mutually orthogonal—six fully independent moral concepts.
The actual result is a third option: integration.
Integration: Neither Merged nor Separated
The six directions neither collapse into one nor are they orthogonal. They occupy "near-maximally independent dimensions"—spread as widely as possible within a six-dimensional subspace, but sharing a common positive component.
Concretely: the average cosine similarity among the six moral directions is 0.26. As a control, six non-moral concept directions (e.g., "food," "weather") built the same way average only 0.013.
0.26 vs. 0.013—the moral directions are significantly positively correlated, but far from merged (which would be 1.0). This positive correlation is the geometric signature of integration: six independent directions sharing a common "moral-ness" component.
In plain terms: the model knows "care" and "loyalty" are different things, but they resemble each other more than either resembles "food"—because both are moral.
MFT's Dichotomy Never Appears
MFT predicts the six foundations should split into two clusters: individualizing (care, fairness, liberty) and binding (loyalty, authority, sanctity). If correct, dendrograms of the six directions should recover this two-group structure.
They do not. Using multiple clustering methods, the dendrograms never stably recover MFT's dichotomy. The model's moral geometry does not reflect MFT's theoretical structure.
What does it reflect? The paper finds: the statistical structure of the training corpus. The model's moral categorization is a geometric projection of co-occurrence patterns for words like "care" and "loyalty" in the corpus—not a projection of psychological theory.
This is a significant finding: the model is not learning MFT; it is learning the corpus. MFT is a theory psychologists distilled from human survey data; the model's moral geometry is a structure grown from text statistics. Where they agree (six foundations are separately probeable), MFT captures real distinctions in the corpus; where they disagree (no dichotomy), MFT's structure is the psychologists' abstraction, not the corpus's structure.
Geometry Stabilizes Early in Training
The paper tracks geometric evolution over training. Finding: the integration geometry appears early in pretraining, long before probe accuracy saturates.
That is: the model very early learns the geometric structure "morality has six directions sharing a common component," but much later becomes accurate on specific moral judgment questions.
This matches the "geometry precedes capability" phenomenon—models first learn conceptual geometric relations, then learn to use that geometry for concrete judgments. Geometry is infrastructure; capability is the application layer. The infrastructure gets built first.
Compositional Moral Dilemmas: Partial but Structured
Beyond single-foundation items (e.g., "he hit someone"—pure harm), the paper tests moral dilemmas—situations where two foundations conflict (e.g., "he hit someone, but to protect a bullied child"—care versus authority).
Result: dilemma directions are neither a simple sum of the two single-foundation directions nor entirely new directions. They are "partially composed"—single-foundation directions contribute part of the signal, but dilemmas have unique components of their own.
This "partial composition" is more pronounced in the MoE architecture (OLMoE-1B-7B) than in the dense architecture (OLMo-2 1B). MoE preserves more compositionality.
Why? The authors' conjecture: MoE's sparse activation lets different experts handle different foundations, making composition more flexible. Dense models compress all foundations into the same parameters, making composition stiffer.
A Complexity–Fragility Gradient
Another pattern: the more complex the moral concept, the more fragile it is.
Single-foundation directions (care) are most stable, changing little under bootstrap resampling. Dilemma directions (care + authority conflict) are less stable. The more complex the composition, the more sensitive the direction is to data perturbation.
This matches intuition: simple concepts occur frequently in corpora and are learned stably; complex concepts occur less and are learned brittlely. But the paper also finds something counterintuitive: fragility cannot be attributed per foundation—you cannot say "care is stable, authority is fragile." Each foundation varies in fragility across contexts; fragility is context-dependent, not foundation-dependent.
Cross-Scale Consistency: Geometry Unchanged from 1B to 7B
A final finding: moral geometry stays consistent from 1B to 7B.
Not just "both 1B and 7B have integration geometry"—the relative relations among the six directions, the size of the shared component, and the dendrogram structure are nearly identical across 1B and 7B.
This suggests moral geometry is not an emergent phenomenon that "appears when models get big," but a statistical regularity that "appears whenever there is enough data." 1B suffices; 7B just raises probe accuracy without changing the geometry.
What This Means
The paper's findings compress into one sentence: models' moral knowledge is integrated, not categorical.
The six moral foundations in a model are neither six independent drawers nor one unified "morality detector." They are six overlapping but independent directions sharing a common "moral-ness" component.
This structure was not taught to the model by MFT—MFT's dichotomy never appears. The structure is learned from the corpus itself, reflecting the statistical distribution of "morality" in human text.
Deeper: the model's moral geometry is a mirror of the corpus, not of theory. If you want to change a model's moral judgments, changing theories is useless; changing the corpus works. This aligns with the "evaluation blind-spot law"—model behavior is determined by the data it has seen, not by the theory you hope it follows.
Paper and Code
This paper raises a broader question: many concepts models learn may not be organized the way our theories assume. MFT was distilled from human surveys, but the model's "morality" grows from text statistics. Where they agree, the theory captures real distinctions; where they disagree, the theory is an abstraction, not the ontology.
The "integration" pattern—neither merged nor separated, but sharing a common component—may apply beyond morality. Emotion, causality, social roles: these multidimensional concepts may all be integrated in models. We think they are categorical because our theories say so; but what models learn from data is closer to the corpus's own structure.
Geometry precedes capability; integration precedes categorization. This may be a general law of concept organization in large models.