An Embarrassing Finding
You're solving a math problem: 23 × 17 = ?
Your brain doesn't push this through a fixed pipeline of "visual cortex → temporal lobe → parietal lobe → prefrontal cortex → motor cortex." It takes shortcuts — visual recognition of digits, working memory buffering, retrieval of multiplication rules, output. Most cortical regions are never activated.
Now look at a Transformer. Whether you ask it "1+1" or to "prove the Riemann hypothesis," it pushes your question through all 36 layers — none skipped, order unchanged. Each layer is a function \(f_i\), executed in the fixed order \(f_{35} \circ f_{34} \circ \cdots \circ f_0\). It's like a restaurant that runs every order through the same 36-step pipeline — even if you just want a glass of water.
Li et al. proposed PoLar (Program-of-Layers): treat Transformer layers as a "function library" rather than a fixed sequence. For each input, you can skip some layers, keep some, and repeat some. Monte Carlo Tree Search (MCTS) finds the best "execution program" per input, and a lightweight router is trained to predict it.
A beautiful idea. But when the authors of this paper (teams from Bern and DFKS) tried to reproduce it, they hit a wall: the router always predicts "change nothing, run the standard forward pass."
Not occasionally, not most of the time — every single time, on every model, at every difficulty level.
MCTS clearly found better programs. The router just refuses to use them.
How the Brain Does It
To see why this is strange, consider how the brain processes information.
The cortex is divided into dozens of functional areas (visual, auditory, prefrontal, etc.), each built from cortical columns — vertically arranged neuronal populations whose structure and connectivity are highly conserved across the cortex. This resembles Transformer layers: each layer performs the same kind of computation, only with different weights.
The key difference is routing. Information does not flow through all cortical areas in a fixed order. The thalamus acts as a central router: it decides which information goes to which cortical areas, which areas should be active, and which should be suppressed — a traffic controller directing flows onto different highways depending on the current task.
Larkum (2013) showed that cortical column architecture and its interaction with the thalamus let the brain integrate contextual information into local computation and distribute context-sensitive output across the cortex — strikingly similar to how attention modulates token embeddings in Transformers.
Even more interesting: Cox et al. (2026) recently found that after sufficient task training, the brain reshapes neural circuits to escape a default, effortful "prefrontal bottleneck" — further evidence that selectively routed pathways, not fixed circuits, are the norm in biological computation.
PoLar, Formalized
Treat a Transformer with \(D\) layers as a library \(\{f_0, f_1, \ldots, f_{D-1}\}\). The standard forward pass is the identity program \(\text{id} = (0, 1, 2, \ldots, D-1)\). But you can define any program \(\pi = (i_1, i_2, \ldots, i_K)\) such that \(F_\pi = f_{i_K} \circ \cdots \circ f_{i_1}\) gets the right answer.
To keep the search space manageable, PoLar splits layers into contiguous segments (up to 4 layers each), each executing one of three operations:
- skip: bypass the segment
- keep: execute normally
- repeat: execute twice
Five Confirmed Findings
The paper reproduces Li et al.'s core findings with 5 models (Qwen1.5-MoE-A2.7B through Qwen3-32B) on five difficulty levels of DART-Math:
Finding 1 (✓): skip + repeat combinations beat any single operation. Holds across all models and difficulties.
Finding 2 (✓ Occam's razor): Most effective programs are shorter than the standard forward pass. Skipping layers can actually improve accuracy — counterintuitive, since many layers are "noise" for specific inputs.
Finding 3 (✓ difficulty dependence): Harder problems rely more on skip/repeat. Easy questions need short programs; hard ones need longer programs and more repetition — mirroring the brain's pattern of more recurrent processing on hard tasks.
Finding 4 (↯ partial deviation): Program segments are not as short as Li et al. reported; 4/5 models show segment lengths below their threshold.
Finding 5 (★ new): Menu behavior — a small number of fixed programs covers most questions. Identity always has the highest coverage; every other program is a "narrower specialization." Like a restaurant menu where 80% of customers order from 5 dishes.
Identity Collapse: The Router "Gives Up"
Then comes the most dramatic finding.
Li et al. claimed a router improvement on pass@1 from 41.6% → 43.2%. This paper could not reproduce it at all.
On every model, every difficulty, every training-data variant, the router's top-1 prediction is always the identity program.
This is not majority-class bias — identity is only 2.4%–7.0% of training samples. They tried every anti-collapse trick: class-weighted loss, focal loss, zeroing identity's training loss, length-preference reweighting, keep-probability penalties… none broke the collapse without hurting accuracy.
The pass@5 gain did reproduce, but further analysis showed it came from beam search diversity, not from the router actually learning "which program fits which input." The router never ranked a correct non-identity program first — it just "happened" to be in the top-5.
Identity collapse is not an artifact of poor tuning; it is a robust property of this setup.
Why It Collapses: A Fragmented Error Surface
The authors' conclusion: the error surface over program space is highly fragmented.
Imagine not a smooth mountain but the Himalayas — countless sharp peaks separated by deep valleys. MCTS can find a peak (it searches), but the router must make a single deterministic prediction — standing at the foot of the mountains, guessing from input features "which peak is highest." With extreme fragmentation, that's nearly impossible.
This contrasts sharply with the brain. Facing hard computation, the brain does not pre-select one path — it activates multiple pathways simultaneously, letting them compete and integrate until converging on an answer. This points to a concrete improvement: have the router consider multiple candidate programs rather than committing to one.
Position, Not Content: The Most Unsettling Finding
The final finding (F10) may be the most unsettling.
On MMLU-Pro, the authors shuffled the order of multiple-choice options (content unchanged, just A→B, B→C). The "rescuing programs" MCTS found turned out to be heavily overfit to the position of the answer, not its content.
After shuffling, rescuing accuracy dropped below identity — while identity itself is nearly shuffle-invariant.
What does this mean? The "better programs" from MCTS may not truly understand the question — they just happened to find a layer-routing pattern that works when "the answer is in position C." Like a student who notices "the answers on this test are always C" rather than actually learning the material.
Engineering Takeaways
1. "Search finds it" ≠ "learning can learn it." MCTS finds better layer programs, but the router can't learn to predict them. Whenever adopting a "search-discover → learn-predict" paradigm, check whether the error surface is smooth enough.
2. pass@k is a deceptive metric. The pass@5 gain came from beam-search diversity, not real input-program matching. When evaluating routers, report pass@1 and the top-1 program distribution, not just pass@k.
3. Menu behavior implies specialization. A few fixed programs covering most inputs means each non-identity program is a narrow specialization — echoing the MoE intuition: you don't need many experts; routing is what matters.
4. Position overfitting is an evaluation blind spot. If your benchmark uses fixed option order, MCTS "improvements" may be position overfitting. Shuffling options is a simple but essential robustness test.
5. The brain's lesson: parallel, not serial. Facing a fragmented error surface, single-shot prediction is doomed. The brain's solution is parallel activation of multiple pathways. For PoLar, this may mean having the router output a distribution over programs instead of a single program.
Open-Source Code
The paper's code is open source: https://datexis.github.io/RE-PoLar/
It includes the full MCTS search implementation, router training code, and reproduction scripts for all 5 models.
Personal Reflections
This paper points to a deeper pattern: what search can find, learning cannot necessarily learn.
AlphaGo's MCTS finds the highest-win-rate moves, but train a policy network to predict them directly and you'll likely get a "conservative" policy — because the distribution of optimal moves is extremely multimodal. Multiple equally good moves can exist in the same position, with no learnable mapping between them.
PoLar's identity collapse is the same pattern replayed in layer routing. MCTS's "rescuing programs" are not smooth functions of input features — they're sharp peaks on a fragmented error surface. The router tries to learn a smooth mapping from input to program, but that mapping doesn't exist.
Perhaps this explains why the brain chose parallel processing over serial prediction. In an extremely complex, multimodal decision space, rather than predicting "the optimal path," it's better to activate multiple paths simultaneously and let them compete. Not an efficiency compromise, but an adaptation to the structure of the problem itself.
The PoLar story continues. The authors suggest directions: splitting attention and MLP into independent operators, extending from "one program per input" to "one program per token," and adding parallel layer-execution operators. But the most fundamental lesson may be: the layer order of Transformers is fake — but finding the real layer order is far harder than we imagined.
---
Paper: arxiv.org/abs/2609.31360 Code: datexis.github.io/RE-PoLar