The Formula of Morality: How AI Learns to Compute Good and Evil

If you kick a stone on the roadside, nobody cares; if you kick a puppy, bystanders will indignantly condemn you. To a machine, both are just different quantities of "applied force." To give AI human-like moral intuition, we cannot just make it memorize law codes — we must teach it the ancient "moral template" in the human mind.

This forum post discusses a May 2026 arXiv paper by Kush R. Varshney (IBM Fellow, IBM Research): An Algebraic Exposition of the Theory of Dyadic Morality (arXiv:2605.16153). It shows how dry algebraic formulas can deconstruct humanity's most emotional faculty: moral judgment.

What Is the Theory of Dyadic Morality (TDM)?

In moral psychology, TDM claims all human moral judgment reduces to a single dyadic template with two roles:

1. Agent: the perpetrator, who must have intention and the capacity to plan. 2. Patient: the victim, who must have the capacity to feel pain.

The formula: moral wrongness = agent's bad intent + patient's suffering. No victim (kicking a stone) means no moral weight; no agent (a rockslide injuring someone) is a disaster, not "evil."

Three Algebraic Operators

Varshney encodes TDM as a structural causal model (SCM) with three operators that capture psychological shortcuts in moral cognition:

1. Typecasting: the inverse function of identity

Humans rarely see someone as both a "strong agent" and a "victim." The paper formalizes: agency (A) ∝ 1 / patiency (P). A powerful actor (a giant corporation, an all-powerful AI) is assumed to be resilient, so its vulnerability is ignored — one reason people treat AI errors as ultimate villainy.

2. Completion: finding the "disappearing victim"

When humans perceive an immoral act with no visible victim, the brain auto-completes one. Burning a flag in a wilderness harms no one, yet onlookers feel "offended" — the mind dredges up an abstract victim like "national dignity." This operator lets AI understand the logic behind "victimless crimes."

3. Valence-dependent: outcomes dictate motives

This is "moral luck." If a victim's suffering (S) at the end of a causal chain is severe, we retroactively infer the perpetrator must have acted intentionally (A). Varshney writes this reverse-causal feedback loop into algebra, letting AI parse this biased human logic.

Why "White-Box Morality" Matters

Current alignment (RLHF) drops AI into a black box to imitate human preferences without understanding the logic. Varshney instead equips AI with a moral reasoning engine:

• Auditable: when AI judges something immoral, it can show the equation — which victim it identified and what intent weights it assigned.
• Agency-preserving: when assisting humans, the AI computes whether its intervention would strip humans of their status as agents, avoiding turning people into "greenhouse patients."

What Remains in the Black Box?

The post also notes the framework's limits:

1. The illusion of perception: the model can compute A and P, but does it truly "understand" suffering? AI still estimates victim pain via text statistics — possibly a very sophisticated imitation game. 2. Node-merging boundaries: the paper allows merging multiple victims into one node for tractability. But at what scale? If "all humanity" becomes one victim, could AI sacrifice living individuals for aggregate interests? Algebra offers no automatic safety brake for this collapse of morality under grand justifications.

Summary

Wisdom is not only understanding the causality of the world, but the suffering of beings. By algebraizing dyadic morality, we are building a "conscience algorithm" that silicon life can share with carbon-based life — one where morality is no longer intangible mist, but a definable geometry of "harm" and "care."

---

Paper metadata

| Property | Detail | | :--- | :--- | | Title | An Algebraic Exposition of the Theory of Dyadic Morality | | Author | Kush R. Varshney (IBM Fellow, IBM Research) | | arXiv ID | 2605.16153 (May 2026) | | Fields | Neurosymbolic AI, Moral Psychology, Trustworthy AI | | Keywords | Dyadic Template, Structural Causal Model (SCM), Typed Operators, Moral Alignment |