An Overlooked Inefficiency Zone
If you've used a diffusion language model (DLM), you may have noticed something odd: when generating tokens for a block, most of the time is spent on the "last few tokens."
Not slightly slower — 40% of the denoising steps produce only 17% of the tokens.
That figure comes from a newly released paper, *Blurring the Block Boundaries: Overlapping Block Denoising for Efficient Diffusion Language Models* (arXiv: 2611.14387, Technion + Amazon, November 10, 2026). The authors call this phenomenon block boundary inefficiency.
More importantly, they propose a method called B3 that requires no retraining and no hyperparameter tuning, yet delivers an 11.5–14.5% speedup on existing diffusion language models with virtually no quality loss.
Background: How Diffusion Language Models Work
If you're familiar with image diffusion models (Stable Diffusion, etc.), the logic of DLMs is similar — just swap "pixels" for "tokens":
1. Initialization: replace the tokens of an entire text segment with noise (mask). 2. Denoising: the model gradually turns noise into real tokens, a few at a time. 3. Block diffusion: for efficiency, instead of denoising the whole sequence at once, text is split into blocks (e.g., 256 tokens each) denoised sequentially.
A recent improvement is self-editing: after a block is denoised, the model revisits it and replaces weakly-supported tokens. This improves quality but makes each block's processing time less predictable.
That's where the problem lies.
The Bottleneck: A Bucket Effect
A block has 256 tokens. Denoising is gradual — at each step, the model accepts some tokens and rejects others.
Ideally, tokens would be accepted uniformly: roughly equal numbers per step across the block.
Reality is very different. The paper's Figure 1 shows data from DiffusionGemma 26B on HumanEval+:
- Block start: high acceptance rate; the model is confident and finishes quickly.
- Block middle: acceptance rate is stable.
- Block end (last ~20% of positions): acceptance rate falls off a cliff. The model hesitates, revises repeatedly, and many steps produce few tokens.
- LLaDA 2.1 (mini 16B and large 64B)
- DiffusionGemma (26B)
- FlowBlock: a flow-matching-based multi-block method.
- MBD-LMs (Multi-Block Decoding Language Models): another parallel decoding method.
- Throughput up 20% (Base 127 → B3 133 tokens/s)
- Compared to FlowBlock (+4%) and MBD-LMs (+12%), B3's +15% is the highest.
- Quality preserved: B3's MT-Bench score is 68.4 vs. Base's 68.6 — essentially no difference.
- CPU pipelining: overlap stages of different instructions rather than making each instruction faster.
- GPU parallelism: run thousands of threads simultaneously rather than speeding up one thread.
- B3: overlap stages of two blocks rather than making one block faster.
Hence the 40% / 17% figure: the last 40% of the block consumes 40% of the denoising steps but yields only 17% of the tokens.
Worse: the next block cannot begin until the current block fully finishes. So this "tail" not only wastes time on the current block but also blocks every subsequent block.
It's like an assembly line where the last 20% of each station's work takes 40% of the time, while the next station idles.
B3's Idea: Overlap Two Blocks
B3 stands for Blurring the Block Boundaries.
The core idea in one sentence: when the current block enters its inefficient "tail" region, start the next block early and share the same forward pass.
Specifically:
1. While denoising the current block, the model also tracks how many tokens remain undetermined. 2. When the remaining fraction drops below a threshold, the model, in a single forward pass, continues denoising the current block's tail while beginning initial denoising of the next block. 3. The threshold is not hand-tuned — B3 self-calibrates, automatically deciding when to start the next block based on the model's state.
The result: the current block's tail and the next block's head are processed in overlap. That wasted 40% of time is put to useful work.
A More Precise Analogy
Imagine writing a paper chapter by chapter.
Normal approach: you finish Chapter 1 (including that hard-to-write conclusion) before starting Chapter 2. If you're stuck on the conclusion for three days, Chapter 2 doesn't move.
B3 approach: when you reach Chapter 1's conclusion (which you know is slow), you simultaneously start drafting Chapter 2's opening (which you know is fast), in the same work sessions.
The key: this isn't deliberate scheduling — you have an "automatic sensor" that detects Chapter 1 entering a hard region and pulls in Chapter 2's opening. When and how much to pull in are all adaptive.
This is what "training-free, hyperparameter-free" means: no retraining, no manual tuning, plug and play.
Experimental Results
The authors tested B3 on two SOTA diffusion language models:
across 10 datasets including HumanEval+ (code generation), GSM8K (math reasoning), and MT-Bench (multi-turn dialogue).
| Metric | Improvement | |--------|-------------| | Speedup | 11.5 – 14.5% | | Quality | On par with baseline (minor fluctuations, statistically insignificant) | | Extra training | None | | Hyperparameters | None |
How good is an 11–14% speedup in inference optimization? For reference, FlashAttention originally delivered roughly 2–4x (but that was a fundamental attention-layer optimization). B3 is a decoding-strategy-level optimization that doesn't touch model architecture or training — 11–14% is a solid number.
More importantly, B3 pushes the Pareto frontier: on the latency–quality trade-off curve, B3's points strictly dominate the baseline. Not "faster but worse" — "faster and equally good."
Comparison with Other Methods
The paper also compares two concurrent multi-block methods:
B3 on LLaDA 2.1 mini 16B:
Why This Matters
Diffusion language models are emerging as an alternative to autoregressive (AR) models. Their core advantage is controllable latency — they can generate multiple tokens in parallel rather than one at a time.
But DLMs have their own bottleneck: block boundary inefficiency. This is not a capability problem but a structural problem of the decoding strategy. B3's contribution: rather than making the model stronger, it makes the decoding pipeline smarter.
This echoes many classical computer-architecture ideas:
Limitations and Outlook
B3 isn't universal. It applies only to self-editing block diffusion models — models with the ability to revisit and revise tokens. For pure block diffusion models without self-editing, B3 doesn't apply.
Also, the 11–14% speedup ceiling is bounded by the tail's length. If future models solve block boundary inefficiency at the architecture level (e.g., a decoding scheme with no "tail"), B3's value would diminish. But for now, B3 is a zero-cost, plug-and-play optimization: no model changes, no training changes, no hyperparameters — just 11–14% faster.
Such "free lunches" are rare in LLM inference optimization in 2026.
---
Paper link: https://arxiv.org/abs/2611.14387
Authors: Amir Mann, Jonathan Fhima, Shaked Brody, Or Avitan, Asif Zamir, Shahar Tsiper, Niv Nayman (Technion / Amazon Web Services)