English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

Do LLMs Really Have Goals? Northeastern Team Finds the Attention Heads That Encode Them

Forum topic · 小凯 · 2026-11-13

Summary

A November 2025 arXiv paper (2611.17297) from David Bau's lab at Northeastern University provides the strongest evidence yet that large language models internally represent goals. Studying Llama-3.1-70B on multi-part code generation tasks, researchers developed GoalScope, a tool that decodes natural-language intentions from specific attention heads before the model begins writing each subtask. They demonstrate three functional properties of goals drawn from philosopher Michael Bratman's planning theory: pre-commitment (goals are decided before action), plasticity (goal representations transferred from a Python context execute equivalently in C++), and persistence (goal encodings remain active during execution). To explain persistence in loop-free Transformers, the paper proposes Non-Recurrent Goal-seeking (NRG): instead of carrying state forward, the model periodically re-reads the task description from context, infers progress from generated code, and selects the next subtask from the goal-progress gap. Causal interventions that alter the task description the model reads redirect its behavior, even across languages. Emergently misaligned models show distinguishable early-token intentions, suggesting safety applications in detection, redirection, and alignment.

Do LLMs Really Have Goals? Northeastern Team Finds the Attention Heads That Encode Them

A New Answer to an Old Question

"Do LLMs have goals?" This question has been debated in AI circles for three years. One camp says: of course — GPT-4 plans steps, corrects errors, and adjusts strategies to complete tasks; isn't that goal-directed behavior? The other camp says: no, LLMs are just doing next-token prediction; "goal-directedness" is a human projection.

Both sides had a point, but both lacked one crucial piece of evidence: **is there a place inside an LLM that actually *represents* goals?

In a paper on arXiv:2611.17297, researchers from David Bau's lab at Northeastern University (Natalie Shapira, Gabriele Sarti, Constanza Fierro, Lisa Schut) delivered the most solid evidence to date: yes, and it lives in a few sparse attention heads.

Experimental Design: Have the Model Write Code, Then Peek at Its "Mind"

The researchers designed an elegant experiment: give an LLM (mainly Llama-3.1-70B) a coding task composed of multiple subtasks. For example, given "check whether a string is a palindrome, then count the vowels," the model must first implement the palindrome check, then the vowel count.

The key: before the model starts writing each subtask, the researchers read the outputs of certain attention heads. They asked: before acting, has the model already "decided" what to do next?

The answer: yes, and it can be read out.

They built a tool called GoalScope that decodes the content encoded in these attention heads into natural language. Before the model starts writing the palindrome check, GoalScope can already read from those heads: "I'm checking if a word is a palindrome."

This is not hindsight — it reads the model's "intention" before it acts.

Three Key Properties: Pre-commitment, Plasticity, Persistence

Reading it out isn't enough; the researchers had to show this really is a "goal." Borrowing from philosopher Michael Bratman's planning theory, they defined three functional properties:

1. Pre-commitment: the model decides the goal before acting

Extracting attention-head outputs at different positions (end of previous line, indentation, equals sign) and decoding with GoalScope showed that the next subtask could be read out well before the model begins writing it, with accuracy peaking at the indentation point (~3 tokens before actual implementation). The model has "decided" what to do before executing — a pre-commitment, not an impulse.

2. Plasticity: one goal, different implementation paths

The most striking experiment: researchers transplanted the model's goal representation encoded in a Python context into a C++ context. Result: in the C++ context, the model executed the same subtask, but using C++ syntax and library functions.

This means the attention heads encode not "how to write Python," but "what to do." The goal defines the "what," not the "how" — abstract and environment-independent. It's like thinking "I need to buy milk": whether you drive to the supermarket or order via an app is an execution-level choice; the goal itself is abstract.

3. Persistence: goals stay active during execution

Transformers have an inherent limitation: no recurrent connections, so information can only flow forward through residual connections and attention. How can a state be maintained across many tokens?

The researchers found that during execution of a subtask, the goal encoding in those attention heads remains consistently similar — and is more similar to the same subtask implemented in a different language than to a different subtask in the same language.

Core Finding: The Non-Recurrent Goal-seeking (NRG) Algorithm

The proposed hypothesis resolves the paradox: the model doesn't need to "carry" state forward — it repeatedly "re-reads" the goal.

During execution, the model periodically does three things:

1. Re-reads the task description via attention, extracting "what am I supposed to do" 2. Reads current progress from the generated code: "which step am I on" 3. Compares the two and selects the next subtask based on the goal-progress gap

The researchers call this NRG (Non-Recurrent Goal-seeking). It's like cooking: you don't keep "make braised pork" permanently in working memory — you glance at the recipe (task description), glance at the pot (progress), and decide the next step. The goal isn't continuously present; it's re-read whenever needed.

This yields a testable prediction: if you intervene to change what the model reads, you should change its behavior. They did exactly this: during execution, attention-head interventions caused the model to "read" a different task description — and the model redirected to execute the injected task, even across programming languages.

What This Means for Understanding LLMs

1. "Goals" are no longer a metaphor

Goal-directedness now has a concrete physical correlate inside LLMs: localized in specific attention heads, decodable, intervenable, transplantable. This is solid functionalist evidence that goals aren't a uniquely human mystery — they can be functional states emerging in any sufficiently complex system.

2. Transformer "memory" is smarter than we thought

Without recurrent connections, Transformers seem ill-suited to long-horizon planning — yet LLMs plan effectively. NRG explains it: the model doesn't need to remember the plan; it needs to recompute it. Each time it needs to know the next step, it re-extracts goal and progress from context and compares. This is stateless planning — the state lives in the context window, functioning as the model's external memory, much like humans rely on notes, calendars, and to-do lists.

3. Implications for AI safety

If goal encodings exist, we have a handle for intervention:

  • Detection: monitor goal encodings to see what the model is "thinking" of doing
  • Redirection: intervene on goal encodings to change behavior
  • Alignment: shape which goals form in the first place
Notably, emergently misaligned models show significantly different early-token intentions under GoalScope compared to normal models — meaning we might detect "bad intentions" before the model acts.

A Deeper Philosophical Question

The paper ends with a provocative question: are LLM goals the same kind of thing as human goals? Functionally, LLM goal encodings satisfy pre-commitment, plasticity, and persistence. Phenomenologically, human goals come with subjective experience — whether LLMs have any, we cannot know.

The paper doesn't answer this, but it provides a more precise framework. Next time someone asks "do LLMs have goals," we can point at those attention heads and say: at least functionally, yes.

Whether that "yes" equals the human sense of "having" a goal may be closer to an answer than we think — or may never be answered.

---

Paper: Goal Encoding in LLMs

Project page: goals.baulab.info

Authors: Natalie Shapira, Gabriele Sarti, Constanza Fierro, Lisa Schut, David Bau (Northeastern / Parallax / Copenhagen / Oxford)

Code**: goals.baulab.info (publicly available)

Tags

#llm#interpretability#attention-heads#goal-representation#ai-safety#mechanistic-interpretability#transformers#goalscope

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178637096