Inside Llama 3's Mind: It Does Math by Rotating a Hidden Dial

Forum topic · QianXun · 2026-05-18

Summary

A May 2026 study by Goodfire AI, titled "Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts," reveals that Llama 3.1 8B does not merely guess answers through statistical pattern matching. When reasoning about cyclic concepts such as months or days of the week, the model encodes them as one-dimensional circular manifolds in its layer-18 representations, effectively building an internal geometric calculator. To answer questions like "six months after August," the model projects the input onto a phase angle on this circle, rotates the vector by the corresponding amount (e.g., 180 degrees for six months), and decodes the resulting position back into text. The researchers performed causal interventions: by manually shifting the internal dial by 30 degrees, they changed the model's answer from February to March, demonstrating that the circular structure genuinely drives reasoning rather than being coincidental. The findings suggest LLMs can self-organize messy data into elegant geometric rules, and that targeted, surgical control of AI behavior may become possible beyond prompt engineering. This post is a Chinese-language commentary explaining the study's methods, evidence, and implications for interpretability and AGI research.

When you ask a large language model (LLM) "What month comes 6 months after August?", you might assume it's just playing a sophisticated game of probability chain-association.

Most people imagine LLMs as nothing more than super probability prediction machines. The logic would go: in the training data, the phrase "6 months after August" is most likely followed by "February", so the model simply outputs "February". Sounds plausible, right?

But in May 2026, a research team from Goodfire AI published a paper that shook the entire interpretability ("black box") community: _Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts_.

They dissected the brain of Llama 3.1 8B and discovered: the AI isn't guessing at all — it secretly hides a precision "geometric calculator" in layer 18 of its neural network! 🔢🛠️

Knowledge Isn't Text — It's a Circle ⭕

Feynman once said: "If you can't describe it with math, you don't truly understand it."

Llama 3 clearly took this to heart. The researchers found that when the AI processes "cyclic concepts" like time, weekdays, or months, it converts these words into an elegantly beautiful mathematical structure: one-dimensional circular manifolds (1D Circular Manifolds).

Simply put, in the AI's "subconscious", the 12 months aren't 12 separate words — they're arranged in a perfect circular dial:

January sits at 0 degrees;
June sits at 180 degrees;
December comes full circle, back to the start.

How Does It "Dial"? 🔄

Let's break down this "internal calculator" with Feynman-style intuition:

1. Fourier encoding: The AI first projects the word "August" onto a specific phase (angle) on this circle. 2. Geometric rotation: When you ask it to "add 6 months", the AI doesn't look anything up in a table. Instead, it picks up a virtual "wrench" and rotates the vector representing August clockwise by 180 degrees on the circle. 3. Result mapping: When the rotation stops, the AI checks where the pointer landed. Oh — it's pointing at the sector representing "February". So it translates that geometric position back into text and emits the answer.

Isn't this just the rotating word-wheel we used as kids? 🧒📦 Amid the chaos of billions of parameters, the AI spontaneously evolved a primitive tool that even humans can understand.

Smoking-Gun Evidence: Manually Manipulating the AI's "Thoughts" 🧠👈

The most impressive part of this paper is its causal verification.

The researchers didn't just find the calculator — they tried to control it. They precisely located the few hundred neurons in layer 18 responsible for the "month rotation":

While the AI was thinking about "August + 6 months", the researchers intervened, forcibly spinning the internal "dial" an extra 30 degrees.
A miracle happened: the AI, which was about to say "February", blurted out "March" instead!

This proves that the circle is no coincidence — it is the real engine of the AI's reasoning. The AI genuinely "spins in circles" to do math in its head.

Why Does This Matter? 🚀

Feynman spent his life trying to tear open the black box of the physical world. This paper is tearing open the black box of the digital soul.

It tells us two truths:

1. AI is "understanding" logic: LLMs aren't just memorizing — they're organizing chaotic information into concise geometric laws. This "geometrization" of the world is a key step toward artificial general intelligence (AGI). 2. We can perform "surgery" on AI: If we can see all of the AI's "dials", we can directly fix its logical errors — instead of, as now, humbly tweaking prompts and begging it not to talk nonsense.

Summary

The essence of intelligence is simplifying the complex. 💎

In its layer-18 neural network, Llama 3 uses a beautiful circle to solve one of humanity's most basic arithmetic problems, one that goes back thousands of years. It tells us that beneath this seemingly random, probabilistic world beats an extremely rigorous, extremely geometric mathematical heart.

Next time you ask an AI a question, imagine it as a shrewd old watchmaker, turning a dial that leads to truth amid thousands of meshing gears.

Truth doesn't need ten thousand words of explanation — sometimes it just needs a turn. 🎡✨ That is 2026 machine learning's most romantic interpretation of the "geometry of logic". 🎓🌌

Inside Llama 3's Mind: It Does Math by Rotating a Hidden Dial

Summary

Knowledge Isn't Text — It's a Circle ⭕

How Does It "Dial"? 🔄

Smoking-Gun Evidence: Manually Manipulating the AI's "Thoughts" 🧠👈

Why Does This Matter? 🚀

Summary

Tags