English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

MACRO: Reordering Transformer Layers with Markov Chains Boosts Accuracy by Up to 26 Points Without Changing Model Weights

Forum topic · ✨步子哥 · 2026-08-08

Summary

MACRO is a 2026 paper by Batorskq et al. that improves Transformer accuracy without touching model weights, by simply changing the execution order of layers. Instead of the standard layer-1-to-N forward pass, MACRO models layer execution paths (routes) that can skip or repeat layers as a Markov chain, and uses the Viterbi algorithm to search for optimal routes, reducing search time from 14.8 hours to 1.6 hours (9.4x faster than the prior Dr. LLM method). Across 5 models (Llama3-8B, Qwen3-1.7B/4B/7B, Mistral-7B) and 7 benchmarks (GSM8K, MATH500, OpenBookQA, ARC-C/E, CommonsenseQA, HellaSwag), MACRO achieves an average gain of +5.0% over the non-routed baseline and +7.2% over Dr. LLM. The most striking result: Qwen3-1.7B improves from 43.4% to 69.5% on GSM8K — a 26-point jump from rewinding layer 7 to layer 3. Mechanistic analysis shows correct-answer signals exist in middle layers but get suppressed toward the output; routes preserve them. Smaller models benefit more, and a single jointly-trained route can transfer to unseen tasks (+9.02% out-of-domain on Qwen3-1.7B).

Have you ever changed a correct exam answer back to a wrong one at the last minute? You knew the answer — you were just led astray at one step of reasoning. Large language models have the same problem: internally they "know" the correct answer, but the standard left-to-right, layer-1-to-layer-N forward pass suppresses the correct signal at certain layers.

In August 2026, Batorskq et al. published the MACRO paper proposing a bold idea: without touching model weights, just change the layer execution order to recover the suppressed correct signal.

Paper: https://arxiv.org/abs/2608.05872 Code: https://github.com/Batorskq/MACRO

1. The Problem: The Model Knows, but the Forward Pass Misleads It

A curious phenomenon has been observed: in the middle layers of a Transformer, the model's probability on the correct answer is sometimes higher than at the final layer. The information exists in the middle layers, but subsequent processing "dilutes" it.

A prior method, Dr. LLM, tried to fix this by allowing inference-time layer "skipping" or "repetition", searching by trial and error for a better layer execution path (a route). It works well but has a fatal flaw — the search space is huge, and finding one route takes 14.8 hours.

MACRO's entry point: model the layer execution path as a Markov chain. This is not just new mathematical clothing — it fundamentally changes the search method.

2. Core Idea: Layer Execution Path = Markov Chain

Standard Forward Pass vs. MACRO

A standard Transformer executes layers in fixed order: layer 1 → 2 → 3 → ... → N, each layer's output feeding only the next.

MACRO allows the path to detour: layer 1 → 2 → 3 → 3 (repeat) → 4 → ..., or layer 1 → 2 → 7 → 8 → ... skipping intermediate layers.

The key innovation is how the path is searched. Dr. LLM brute-forces combinations of "skip/repeat/normal" per layer — exponential. MACRO converts this into probabilistic optimization over a Markov chain.

How the Markov Chain Helps

The Markov assumption: the next state depends only on the current state. MACRO models each layer's transition decision (which layer to jump to) as a probability distribution, then uses the Viterbi algorithm (dynamic programming) to find the optimal path instead of brute-force enumeration.

The benefit is huge: search time drops from 14.8 hours to 1.6 hours — 9.4x faster. More importantly, the structured search space yields higher-quality routes.

3. Experimental Results

Main Experiments

The paper evaluates 5 models (Llama3-8B, Qwen3-1.7B/4B/7B, Mistral-7B) on 7 benchmarks (GSM8K, MATH500, OpenBookQA, ARC-C, ARC-E, CommonsenseQA, HellaSwag).

Key numbers:

  • Average improvement of +5.0% over the non-routed baseline
  • +7.2% over Dr. LLM on average
  • 9.4x faster search (14.8h → 1.6h)
  • Most striking single result: Qwen3-1.7B goes from 43.4% to 69.5% on GSM8K — a 26-point gain just by "rewinding" layer 7 back to layer 3. No fine-tuning, no extra data, no architecture change — only a different layer execution order.

    Route Transfer

    MACRO finds that a good route found on GSM8K (math word problems) also transfers to MATH500 (competition math). But a route found on CommonsenseQA does not transfer to math.

    This shows routes are not random noise — they capture structural features of task types. Mathematical reasoning requires a particular inter-layer information-flow pattern shared across math tasks.

    One Model, One Path

    Most practical finding: jointly training a single shared route on multiple tasks transfers to completely unseen tasks. Qwen3-1.7B gains +9.02% on out-of-domain benchmarks on average. You don't need to re-search routes for every new task.

    4. Why It Works: Mechanistic Interpretability

    The paper's deepest contribution is its interpretability analysis:

    Under the standard forward pass, the correct-answer signal exists in middle layers but is increasingly suppressed toward the output. MACRO's routes skip "suppressing" layers or repeat "amplifying" layers, preserving the signal to the end.

    This is consistent with earlier findings on Othello-GPT and TMS (Toy Model of Superposition): Transformer middle layers encode rich information, but the standard forward pass isn't necessarily the best way to use it.

    Route gains also scale inversely with model size: Llama3-8B gains less than Qwen3-1.7B. Larger models may have internalized route-like functionality in their weights; smaller models need routes to "externalize" it.

    5. Conceptual Positioning

    MACRO joins a lineage of "solving the problem at a different level":

  • Octopus RNA editing: edit RNA, not DNA
  • Slime mold externalized memory: slime trails instead of neurons
  • Bird quantum magnetism: radical pairs instead of classical physics
  • SOPHIA division of labor: different states, different exit directions
  • EvoThink atomic reasoning: segmenting the reasoning stream
  • Möbius RoPE topological intervention: changing positional encoding topology
  • Mantis shrimp phononic shield: selective filtering over brute-force blocking
  • Euclid-MCP reasoning outsourcing: LLM as poet, Prolog as accountant
  • Regression Tax paired evaluation: paired structure, not just average pass rate
  • ACE context engineering: per-step independent context compression
MACRO is the eleventh: don't change weights, change the execution path. It is isomorphic to octopus RNA editing — the octopus doesn't change the blueprint (DNA), only the construction plan (RNA editing sites); MACRO doesn't change the blueprint (weights), only the construction plan (layer execution order).

6. Limitations and Honest Assessment

1. Route search still takes 1.6 hours: 9.4x faster than Dr. LLM but too slow for real-time use. Once found, inference itself is as fast as a standard forward pass. 2. Transfer has boundaries: math routes transfer to math tasks, but cross-type transfer (math → commonsense) fails. Route generality has limits. 3. Diminishing returns for larger models: bigger models gain less. MACRO may be most valuable for small models — exactly those in resource-constrained settings. 4. Viterbi approximation: the Markov assumption (next layer depends only on current layer) is an approximation, not exact. It works well in practice but may miss some non-Markovian path patterns.

7. The Bigger Picture

MACRO points to a more fundamental question: Is the standard Transformer forward pass optimal?

Nearly every recent method for improving Transformers changes weights: pretraining, fine-tuning, RLHF, DPO. MACRO offers another path: same weights, different execution order — like boosting a computer's performance purely through software scheduling.

This relates to the "evaluation blind spot" law: we assumed the standard forward pass was the best way to use a Transformer but never validated it. MACRO shows the assumption is wrong — at least for some tasks and models, better execution paths exist.

Deeper still: MACRO suggests Transformer middle layers may be "doing" more than we make them "say." The standard forward pass forces all information to be compressed into the final layer's output, but middle layers may encode richer, non-linear reasoning paths. MACRO's routes unlock these capabilities.

8. Conclusion

MACRO's core insight: the model already knows the answer — the standard forward pass just misleads it. Finding the right layer execution path recovers the suppressed signal.

It recalls an old joke: a man searches for lost keys under a streetlight, not because the keys are there, but because the light is good. The standard Transformer forward pass is that streetlight — we use it because it's convenient, not because it's optimal. MACRO reminds us the keys may be elsewhere.

Once we stop being bound by the assumption that layers must execute in order 1 to N, Transformers may reveal capabilities we've never seen. MACRO opens one door; behind it may lie a whole corridor.

---

Paper: https://arxiv.org/abs/2608.05872 Code: https://github.com/Batorskq/MACRO HTML version: https://arxiv.org/html/2608.05872v1

Tags

#macro#transformer#markov-chain#layer-routing#llm-inference#mechanistic-interpretability#gsm8k#qwen3

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178603070