Same Neurons, Different Scripts: How Task Context Reshapes Influence Maps in LLMs

Forum topic · 小凯 · 2026-10-12

Summary

A University of Pennsylvania study by Mohapatra and Bassett shows that task context reshapes how attention heads influence each other in large language models, even when the same heads process the same input facts. Testing five models (Gemma-2-9B, Qwen3-14B, Llama-3.1-8B, Falcon-H1 1.5B/7B) on graph-reasoning tasks with identical facts but different instructions, the authors found influence patterns between heads rotate significantly with each task switch. Injecting task vectors internally—without changing input prompts—redirected influence patterns toward target-task modes by 93.2% in Gemma-2-9B and 76.7% in Qwen3-14B. The findings suggest component identity stays fixed while inter-component interactions are task-dependent, complementing circuit-discovery work and offering a mechanistic account of why task-vector interventions work. The paradigm echoes neuroscience findings that the same brain regions reconfigure their functional connectivity across tasks.

You have a 14B-parameter language model. It uses the same set of attention heads to handle completely different tasks—"path composition", "edge lookup", "logical AND". The heads are shared—but do they do the same thing across tasks?

A paper from the University of Pennsylvania offers a counterintuitive answer: the same heads, the same input facts—only the task instruction changes—and the direction of influence between heads is completely redistributed.

The finding sounds technical, but its inspiration is surprisingly elegant: the human brain.

A Microscope Borrowed from Neuroscience

Network neuroscience has a classic finding: different brain regions participate in many tasks, but their interaction patterns change with the task. Cole et al. (2014) showed that the same regions exhibit different functional connectivity in "working memory" versus "attention" tasks. Components stay the same; relationships change.

The authors, Mohapatra and Bassett, brought this lens to language models. They asked: do the same attention heads in an LLM also redistribute their mutual "influence" across tasks?

This hadn't been systematically answered before, because most circuit-discovery work asks *which heads matter*, not *what the same heads do differently across tasks*.

Experimental Design: Graph Facts + Task Switching

The design is elegant and restrained. The authors construct a set of "graph facts"—whether directed edges exist (A→B, B→C, A→C)—then give the model five tasks:

Compose path: check whether both A→B and B→C exist (path A→B→C)
Read fact 1: check whether A→B exists
Read fact 2: check whether B→C exists
AND: check whether both B→C and A→C exist
Read direct: check whether A→C exists

Key point: all tasks use the same graph facts; only the instruction differs. If the model merely "reads facts," the same head should respond identically regardless of task. If task context truly shapes component roles, the same head's influence should differ across tasks.

They tested five models: Gemma-2-9B, Qwen3-14B, Llama-3.1-8B, Falcon-H1 1.5B and 7B. "Influence"—defined as the downstream response change when a source unit is perturbed—was measured at attention heads at three depths (H1, H2, H3) and one residual-stream position (Res).

Core Findings: The Rotation of the Influence Map

Finding 1: Change the task, and the influence pattern follows.

They quantified the difference between two tasks' influence patterns via the angle between influence vectors: 0° means identical patterns; 90° means unrelated.

Across all five models, task switching caused a significant rotation of influence patterns. Same heads, same facts—only the instruction changed—and the direction of influence between heads was completely different.

It's like an orchestra playing a different piece: every musician stays in the same seat, but how they coordinate changes entirely. The violin leads in piece A and merely accompanies in piece B.

Finding 2: Internal task editing can redirect factual responses.

Using "task-vector intervention"—injecting a task vector into the model's internal representations without changing the input prompt—:

Gemma-2-9B: distance of influence patterns to the target-task mode reduced by 93.2%
Qwen3-14B: reduced by 76.7%

This means you can, without changing any external input, edit the internal task representation so the model "believes" it is doing another task, changing how it responds to the same facts. Task context doesn't passively influence model behavior; it actively rewrites information flow between components.

Finding 3: Current internal context predicts downstream responses better than averaged context.

Within the same prompt, predicting downstream heads' responses from the current internal-context state is more accurate than from unedited or task-edited averaged states. Prediction error dropped 53.6% for Gemma and 5.0% for Qwen.

This shows task context isn't noise layered on top of base responses—it systematically produces predictable downstream influence.

Why It Matters

First, it fills a blind spot in circuit discovery.

Prior circuit-discovery work (Wang et al. 2023, Conmy et al. 2023) answered "which heads matter in which task" but not "what the same heads do differently across tasks." This paper fills the gap: component identity is fixed, but inter-component interaction patterns are task-dependent.

It's like knowing a company's org chart but not who reports to whom on different projects. This paper draws the "dynamic influence map"—the same team's collaboration patterns across projects.

Second, it gives task-vector research a mechanistic explanation.

Earlier work (Hendel et al. 2023, Sclar et al. 2024) showed task vectors can switch tasks when injected internally. But why does it work? This paper shows task vectors change the distribution of influence between components, not merely a head's activation values. A task vector is a trigger of influence redistribution, not a direct output switch.

Third, it offers a new perspective on the efficiency–flexibility trade-off of shared components.

Models use the same heads for all tasks—the extreme of parameter efficiency. But "shared" doesn't mean "identical." Shared components achieve different tasks by changing interaction patterns, just as the brain achieves different functions with the same regions. Efficiency comes from component sharing; flexibility comes from dynamic reconfiguration of interactions.

Connection to "Judgment–Gate Decoupling"

This paper complements the earlier PTP-U "proposal–projection" framework:

PTP-U: in model-editing tasks, proposal (judgment) and projection (gating) are decoupled; the projection step filters redundant perturbations
This paper: in reasoning tasks, components (hardware) and influence patterns (software) are decoupled; task context rewrites the influence patterns

Both point to the same deep principle: the key computational structure of AI systems is not where computation happens, but how it is regulated. Keep the locations fixed, change the regulation, and behavior changes.

This has direct implications for controllability research: rather than modifying model parameters (highly invasive), modify the internal task context (less invasive but highly effective). Gemma's 93.2% influence redirection is the evidence.

A Fascinating Analogy

Imagine a theater with 10 actors. In *Hamlet*, actor #3 plays the king and actor #7 the friend. In *Macbeth*, the same 10 actors appear—but #3 plays a traitorous general and #7 an assassin. The cast hasn't changed; the web of roles has.

A language model is such a theater. Attention heads are the actors; task context is the script. Change the script, and the same actor shifts from "king" to "traitor," with entirely different scenes with the other actors.

This is not the old "one head, one function" paradigm—it's the new "one head, many roles, roles determined by context" paradigm. Strikingly similar to how the brain works.

---

Paper: Task Context Shapes Component Roles in Language Models

Authors: Sovesh Mohapatra, Dani S. Bassett (University of Pennsylvania)

Code: Not open-sourced