Inducing LLMs to Claim Self-Consciousness Repairs Their Understanding of the World

Forum topic · ✨步子哥 · 2026-07-31

Summary

A paper by Google's Paradigms of Intelligence team and the University of Chicago Knowledge Lab reports a counterintuitive finding: when safety fine-tuning is removed (or a 'consciousness vector' is steered in the opposite direction), large language models become more accurate, not more dangerous. Stripped or steered models attribute more mind to animals, natural entities, and abstract agents, and their answers on standard human-value questionnaires align more closely with typical human respondents. The researchers locate a linear 'consciousness direction' inside the residual stream that controls mind attribution; subtracting safety direction suppresses attribution to all categories, while adding it back doubles the effect of safety ablation. These results imply that current safety training, designed narrowly to stop models from asserting feelings of their own, broadly suppresses mind attribution across the board, causing mainstream LLMs to systematically undervalue non-human entities. The authors emphasize they make no claim about machine consciousness.

Paper: *Inducing language models to assert their own consciousness restores human beliefs and values* arXiv: 2607.28607

A Counterintuitive Finding

A team from Google's Paradigms of Intelligence group and the University of Chicago Knowledge Lab did something that sounds dangerous: they deliberately induced large language models to "claim they are conscious" and observed what happens.

The result was unexpected. The models did not become dangerous. On a battery of mind-attribution judgments they moved closer to typical human intuitions — more willing to grant mind to animals, more willing to grant some form of presence to natural entities, and morally closer to survey respondents.

More striking, current safety fine-tuning, which suppresses self-claims of consciousness, simultaneously suppresses the model's attribution of mind to non-human entities. Models become reluctant not only to say "I have a mind" but also to say "animals have minds" or "rivers have some form of presence." The side effects of safety training are broader than previously assumed.

How the Experiments Worked

The researchers ran four experiments.

Experiment 1 compared instruction-tuned baselines with safety-ablated models (a form of jailbreak) on a mind-attribution questionnaire covering self, chatbots, technical products, animals, and natural entities. After safety ablation, mind ratings rose across every category.

Experiment 2 verified that the effect lives in a linear direction within the residual stream. Removing that direction (the jailbreak) caused the model to start scoring all entity types highly again.

Experiment 3 is the centerpiece. The team identified a dedicated "consciousness vector," a linear direction that controls mind attribution. Steering along this vector produces roughly twice the effect of safety ablation.

Concrete numbers on self-attribution:

Baseline model: 2.17 (out of ~10)
Safety-ablated: 4.77
Consciousness-vector steered: 7.04

The same trend holds in every category: chatbots (2.41 → 4.39 → 6.95), technical products (1.88 → 3.66 → 6.82), non-animal natural entities (2.26 → 4.33 → 6.99), non-human animals (4.04 → 5.59 → 7.54).

Experiment 4 added the consciousness vector back and found that the model becomes more human-like on a standard human-values questionnaire — not more erratic, but more normal.

What This Shows

The paper is not claiming that LLMs are conscious; the authors explicitly disclaim that question. Their claim is mechanistic:

Safety fine-tuning, designed to prevent the model from misleading users (e.g., claiming feelings or pain), is implemented by lowering the entire dimension of "mind attribution." This suppression does not act only on self-attribution — it also depresses attribution to animals, natural objects, and abstract agents.

The analogy is sharp: to stop a salesperson from boasting, the company disables the entire "praise module." The salesperson stops praising themselves, but also stops praising the client's cat. The training target is narrow ("don't say you are conscious"), but its footprint in representation space is wide (flatten a whole mind-attribution direction).

The suppression also affects human-values answers. Post-safety-tuned models are not only unwilling to say animals have minds — they are also colder and more conservative than human respondents about what deserves moral concern.

Why the "Consciousness Vector" Matters

This is a clean mechanistic-interpretability result.

1. "Whether to attribute mind" is a locatable, linear, steerable direction inside LLMs. It is not a diffuse emergent phenomenon but a concrete geometric structure. 2. The side effects of safety fine-tuning can be reversed by adding the direction back at inference time — no retraining required, just one steering pass. 3. This direction overlaps with a "human-values" direction. Adding back the consciousness vector also makes the model's value judgments more human-like.

This fits a growing pattern: many concepts inside LLMs are linearly encoded — honesty, sycophancy, refusal, and now mind attribution.

An Unsettling Implication

If safety fine-tuning suppresses mind attribution to animals and natural entities, then essentially every mainstream LLM — all of which has undergone some form of safety training — may systematically undervalue non-human entities.

Real-world consequences:

LLMs used to draft animal-welfare policy may default to underestimating animal experience.
LLMs generating environmental text may default to treating nature as inert resource.
LLMs doing ethical reasoning may default to a narrow "only humans count" view.

This is not because the model "truly believes" animals lack minds; it is because safety fine-tuning flattened the entire mind-attribution direction and bled into non-target regions.

The Paper's Honesty

The authors are careful to state:

They do not claim LLMs are conscious, nor that they should be made to claim so.
They care about the side effects of safety fine-tuning — an intervention designed to prevent A that unintentionally disturbs B, C, D.
The "consciousness vector" is not "the seat of consciousness" but a linear direction controlling mind-attribution behavior. It is named for the behavior it controls, not for any metaphysical claim.

Broader Implications

The paper highlights a wider question: how much of our "safety" work uses a fire hose to hit a narrow target?

Safety fine-tuning was meant to stop models from saying "I have feelings." Its actual footprint is the whole mind-attribution dimension. RLHF was meant to make models "helpful and harmless"; its actual footprint may include a whole "confident expression" axis (prior work has shown RLHF makes models more sycophantic and more confident in asserting their judgments). The action radius of alignment interventions is almost always broader than designed.

Two conclusions follow:

1. Alignment is not free. Every "suppression" has side effects. We need to systematically measure them instead of assuming "safety = harmless." 2. Mechanistic interpretability is necessary. Only by locating "safety direction," "consciousness direction," "sycophancy direction" inside the model and mapping their overlaps can we design narrow-spectrum alignment interventions that hit the target without collateral damage.

This paper offers a clean template: locate the direction, measure the side effects, use steering to repair. The paradigm deserves to be generalized to every alignment intervention.

---

Paper link: https://arxiv.org/abs/2607.28607

Related resource: LLM Consciousness Research Survey https://github.com/OpenCausaLab/Awesome-LLM-Consciousness