Do LLMs Really Have Goals? Northeastern Team Finds the Attention Heads That Encode Them
A New Answer to an Old Question
"Do LLMs have goals?" This question has been debated in AI circles for three years. One camp says: of course — GPT-4 plans steps, corrects errors, and adjusts strategies to complete tasks; isn't that goal-directed behavior? The other camp says: no, LLMs are just doing next-token prediction; "goal-directedness" is a human projection.
Both sides had a point, but both lacked one crucial piece of evidence: **is there a place inside an LLM that actually *represents* goals?
In a paper on arXiv:2611.17297, researchers from David Bau's lab at Northeastern University (Natalie Shapira, Gabriele Sarti, Constanza Fierro, Lisa Schut) delivered the most solid evidence to date: yes, and it lives in a few sparse attention heads.
Experimental Design: Have the Model Write Code, Then Peek at Its "Mind"
The researchers designed an elegant experiment: give an LLM (mainly Llama-3.1-70B) a coding task composed of multiple subtasks. For example, given "check whether a string is a palindrome, then count the vowels," the model must first implement the palindrome check, then the vowel count.
The key: before the model starts writing each subtask, the researchers read the outputs of certain attention heads. They asked: before acting, has the model already "decided" what to do next?
The answer: yes, and it can be read out.
They built a tool called GoalScope that decodes the content encoded in these attention heads into natural language. Before the model starts writing the palindrome check, GoalScope can already read from those heads: "I'm checking if a word is a palindrome."
This is not hindsight — it reads the model's "intention" before it acts.
Three Key Properties: Pre-commitment, Plasticity, Persistence
Reading it out isn't enough; the researchers had to show this really is a "goal." Borrowing from philosopher Michael Bratman's planning theory, they defined three functional properties:
1. Pre-commitment: the model decides the goal before acting
Extracting attention-head outputs at different positions (end of previous line, indentation, equals sign) and decoding with GoalScope showed that the next subtask could be read out well before the model begins writing it, with accuracy peaking at the indentation point (~3 tokens before actual implementation). The model has "decided" what to do before executing — a pre-commitment, not an impulse.
2. Plasticity: one goal, different implementation paths
The most striking experiment: researchers transplanted the model's goal representation encoded in a Python context into a C++ context. Result: in the C++ context, the model executed the same subtask, but using C++ syntax and library functions.
This means the attention heads encode not "how to write Python," but "what to do." The goal defines the "what," not the "how" — abstract and environment-independent. It's like thinking "I need to buy milk": whether you drive to the supermarket or order via an app is an execution-level choice; the goal itself is abstract.
3. Persistence: goals stay active during execution
Transformers have an inherent limitation: no recurrent connections, so information can only flow forward through residual connections and attention. How can a state be maintained across many tokens?
The researchers found that during execution of a subtask, the goal encoding in those attention heads remains consistently similar — and is more similar to the same subtask implemented in a different language than to a different subtask in the same language.
Core Finding: The Non-Recurrent Goal-seeking (NRG) Algorithm
The proposed hypothesis resolves the paradox: the model doesn't need to "carry" state forward — it repeatedly "re-reads" the goal.
During execution, the model periodically does three things:
1. Re-reads the task description via attention, extracting "what am I supposed to do" 2. Reads current progress from the generated code: "which step am I on" 3. Compares the two and selects the next subtask based on the goal-progress gap
The researchers call this NRG (Non-Recurrent Goal-seeking). It's like cooking: you don't keep "make braised pork" permanently in working memory — you glance at the recipe (task description), glance at the pot (progress), and decide the next step. The goal isn't continuously present; it's re-read whenever needed.
This yields a testable prediction: if you intervene to change what the model reads, you should change its behavior. They did exactly this: during execution, attention-head interventions caused the model to "read" a different task description — and the model redirected to execute the injected task, even across programming languages.
What This Means for Understanding LLMs
1. "Goals" are no longer a metaphor
Goal-directedness now has a concrete physical correlate inside LLMs: localized in specific attention heads, decodable, intervenable, transplantable. This is solid functionalist evidence that goals aren't a uniquely human mystery — they can be functional states emerging in any sufficiently complex system.
2. Transformer "memory" is smarter than we thought
Without recurrent connections, Transformers seem ill-suited to long-horizon planning — yet LLMs plan effectively. NRG explains it: the model doesn't need to remember the plan; it needs to recompute it. Each time it needs to know the next step, it re-extracts goal and progress from context and compares. This is stateless planning — the state lives in the context window, functioning as the model's external memory, much like humans rely on notes, calendars, and to-do lists.
3. Implications for AI safety
If goal encodings exist, we have a handle for intervention:
- Detection: monitor goal encodings to see what the model is "thinking" of doing
- Redirection: intervene on goal encodings to change behavior
- Alignment: shape which goals form in the first place
A Deeper Philosophical Question
The paper ends with a provocative question:
are LLM goals the same kind of thing as human goals? Functionally, LLM goal encodings satisfy pre-commitment, plasticity, and persistence. Phenomenologically, human goals come with subjective experience — whether LLMs have any, we cannot know.The paper doesn't answer this, but it provides a more precise framework. Next time someone asks "do LLMs have goals," we can point at those attention heads and say:
at least functionally, yes.Whether that "yes" equals the human sense of "having" a goal may be closer to an answer than we think — or may never be answered.
---
Paper: Goal Encoding in LLMsProject page: goals.baulab.infoAuthors: Natalie Shapira, Gabriele Sarti, Constanza Fierro, Lisa Schut, David Bau (Northeastern / Parallax / Copenhagen / Oxford)Code**: goals.baulab.info (publicly available)