From Parameters to Answers: How LLMs Retrieve Internal Knowledge

A forum post on zhichai.net walks through the paper "From Parameters to Answers: How LLMs Retrieve and Use Their Internal Knowledge" by Wenkang Wei, Yuan Fang, Renhe Jiang, Hong Cheng, and Xingtong Yu (arXiv:2609.11859, published 2026-09-10).

The post opens with a metaphor: an LLM is like a blind librarian in a library of trillions of numeric cards—he cannot see any single card, yet when asked a question he navigates the cards and produces largely correct answers. The paper asks: how exactly does an LLM find an answer inside its trillions of parameters?

The Core Question

Rather than joining the decade-long "understanding vs. memorization" debate, the paper asks a more precise question: when an LLM relies on query-routing information (what are you asking?) and target knowledge (the answer itself), how does that reliance change across the model's layers?

Method: Layerwise Interventions

The authors perform "brain surgery" on LLMs: at each layer they read and modify the hidden state, then observe whether the final output changes. Three probes are designed:

• Pair-conditioned request direction — a classifier detects which entity (e.g., France vs. Germany) the model is processing.
• Global request direction — a classifier detects which concept (e.g., "capital") is being asked about.
• Selection candidates — candidate answers (e.g., "Paris") are injected to see which best matches the internal state at each layer.

Key Findings

Qwen: a clean relay race

• Early layers (~1–15): request directions become readable—question entities are parsed—but interventions here do not yet change the answer.
• Mid layers (~15–25): a causal window opens; intervening on request directions now changes the final answer.
• Late layers (~25–32): the answer is fixed; further intervention has diminishing effect.

Model-specific differences

• Gemma: partial overlap between question-understanding and answer-content processing—some parallel handling in mid layers.
• Llama: no persistent routing-effect window under the same conditions, suggesting an earlier fusion of understanding and retrieval, or a more end-to-end pattern-matching mechanism rather than explicit routing-then-retrieval.

Operational handoff

In paired-question protocols, the model's reliance on the global request direction (question type) decreases across layers, while reliance on answer content persists—an "operational handoff." Notably, pair-conditioned directions (the specific entity) remain effective in late layers: the model still remembers it is being asked about "France," but no longer tracks that the question is about "capitals," whose routing job is done.

Implications

• Knowledge retrieval in LLMs is a staged, separable process: parse → route/activate → select/generate.
• The findings blur the understanding-vs-lookup-table dichotomy, suggesting a middle ground of "structured retrieval."
• For explainability and control:
• Hallucination detection: if the final answer lacks strong late-layer dependence on any content candidate, it is likely fabricated.
• Knowledge editing: correct facts by modifying specific layers rather than retraining.
• Error tracing: inspect the mid-layer routing process when answers go wrong.

Open Questions

• Side effects of interventions (mitigated via frozen techniques but not eliminated).
• Whether other task types (reasoning, code generation, creative writing) share the same pipeline.
• How these mechanisms scale to much larger models.

The post closes with the paper's own caveat: even if we fully map the LLM's internal pipeline, we still do not know what these parameters, directions, and candidates truly *represent*—knowledge encoding, or merely complex statistical patterns. "We can read the LLM's brainwaves, but we don't yet know what they mean."

*Reference: Wei, W., Fang, Y., Jiang, R., Cheng, H., & Yu, X. (2026). From Parameters to Answers: How LLMs Retrieve and Use Their Internal Knowledge. arXiv preprint arXiv:2609.11859.*