LAND: Everyone Gets the Same Lottery Tube — But Yours Has No Top Prize
> Three Accounts of Compute · Part 2: The Draw Account > Paper: LAND: Learning an Adaptive Number of Draws for Bayesian Best-of-N Selection > arXiv: 2611.03388 | 2026-11-02 | Aleph Alpha Research
🎰 The Wisdom of Drawing Lots
At a company raffle, everyone draws exactly three times. Some people draw three "thanks for participating" tickets and want to tear the lottery tube apart. Others draw an iPhone 16 Pro Max on the first try — but the rules say you must draw all three. Your desire to draw again is zero; even a power bank means nothing to you now.
But the rules are the rules. So you hold your iPhone and draw two more losers — and the unlucky person cannot draw an extra time, even though you both know ten more tries might turn their luck around.
Best-of-N is that "everyone must draw N times" rule. It is the most primitive and most widely used quality-boosting technique in the LLM era: generate N candidates for an input, score each with a quality function, keep the best. In translation, sample ten translations of a sentence, score them with MetricX, keep the top one. In code generation, generate ten implementations, run unit tests, keep the best pass rate. In reasoning, sample ten chains of thought and take the majority answer.
Simple, universal, effective — and completely indifferent to input difficulty. Whether you're translating "Hello, how are you" or a five-level nested legal clause, you get the same number of candidates. Intuition says this is doubly wrong: for easy inputs, extra draws are waste — you already have the best result; for hard inputs, too few draws are also waste — better tickets may still be in the tube.
Some inputs are inherently easy: every translation of "Good morning" scores nearly the same — the tube is full of top prizes. Some are inherently hard: highly ambiguous sentences yield wildly varying scores — more draws may well surface the brilliant version.
Aleph Alpha researchers Koen Oostermeijer and Teryn Jones formalized this: fixed Best-of-N allocation implicitly assumes each additional sample has equal value for all inputs. That assumption is false, so fixed allocation spends good steel on the back of the blade. LAND installs a camera on every lottery tube: it watches your draws, infers the tube's composition, and dynamically decides whether one more draw is worth it.
🧮 Why Uniform Allocation Is Wrong: A Concrete Example
Consider translation with a total budget of N = 8 candidates per source sentence. Different sources have completely different candidate-score distributions. The paper uses a two-level model:
Level 1: differences between sources. Each source has a latent "potential" μ — the expected score of all its candidates. Some sources have μ ≈ 0.92 (the model handles them well); others μ ≈ 0.65.
Level 2: differences within a source. Conditional on μ, candidate scores X fluctuate with variance σ²(μ). Crucially, σ²(μ) need not correlate with μ. A μ = 0.9 source may have tiny σ² (confident, consistent) or huge σ² (oscillating between 0.7 and 0.98).
Now do the math. Source A: μ = 0.9, tiny σ², scores in [0.88, 0.92]. Best-of-8 gets ~0.92 — only +0.02 over Best-of-1's 0.9. Those 7 extra samples were nearly free money wasted. Source B: μ = 0.7, huge σ², scores in [0.3, 0.95]. Best-of-8 may find 0.9 — a +0.2 gain, extremely high ROI.
Reallocate: give A 5 draws, B 11 — same total budget, significantly better overall quality. This is adaptive allocation: give draws to tubes that still contain un-drawn top prizes.
⚖️ The Economist's Intuition: Equalize Marginal Returns Everywhere
LAND's optimal allocation rule has a clean mathematical form:
H_μ(λ) = E[(X - λ)⁺ | μ] = Λ
The left side is the expected improvement of one extra sample over the current best λ. The right side Λ is a global constant — the marginal cost of one candidate generation. At the optimum, all sources' marginal improvements are equal. If one more sample improves A by 0.05 and B by 0.01, shifting budget from B to A yields a better total. This is the microeconomic equimarginal principle applied to sampling budgets: the last dollar spent on apples should yield the same satisfaction as the last dollar on bananas.
The deeper origin is Weitzman's 1978 Pandora's Box problem: open boxes in order of reservation value; stop when all remaining boxes' reservation values fall below the search cost. LAND transfers this to LLM sampling — each source is a box, each generation a draw, Λ the cost of opening. Unlike Pandora's one-shot boxes, LLM tubes can be re-drawn, enabling posterior updates — LAND's advance over classical Pandora's Box.
🧠 But μ Is Unobservable: Bayesian Posterior to the Rescue
μ is a latent variable; you can only infer it from observed scores. Seeing 0.85, 0.87, 0.86 suggests a stable good tube; 0.92, 0.45, 0.78 suggests a temperamental one. Hence LAND's second layer: Posterior-Adaptive Allocation.
From the first candidate onward, LAND maintains a posterior q_t(μ) over μ, updated by Bayes' rule after each observation:
q_{t+1}(μ) ∝ f_μ(X_{t+1}) · q_t(μ)
Evidence accumulation concentrates the posterior — from the population-level prior toward this specific tube's personality. Expected improvement is then posterior-weighted:
Ĥ_t(λ) = ∫ q_t(μ) · H_μ(λ) dμ
If Ĥ_t(B_t) > Λ — the expected improvement over the current best B_t exceeds the compute price — keep drawing; otherwise stop. Both B_t and q_t update after each draw, so the stopping threshold λ_t is dynamic.
Example: draws of 0.72, 0.85, 0.90. Posterior concentrates on μ ≈ 0.82–0.88 with medium σ². Expected further improvement Ĥ_t(0.90) ≈ 0.01 < Λ = 0.03 — stop. But with 0.45, 0.30, 0.62 (best 0.62, large σ²), Ĥ_t(0.62) ≈ 0.08 > Λ — keep going.
📦 Fitting a Continuous World into Discrete Bins
Rather than Gaussian or Beta distributions — the authors found parametric tail assumptions materially distort cost allocation — LAND uses a discrete empirical mixture model:
1. Bin the scores. Partition [0, 1] into J cells with representative values v_j; boundaries initialized from calibration-data quantiles for higher resolution where scores are dense. 2. Assume I "tube breeds." Each type i has a prior probability π_i and a categorical distribution p_ij over the J cells. Any tube's distribution is approximated by a convex combination of these I base shapes. 3. Fit with EM, maximizing marginal log-likelihood over calibration data.
Why discrete mixtures? No shape assumptions. Real MetricX score distributions can be skewed, multimodal, or heavy-tailed. Gaussian assumptions of symmetry and unimodality break here; discretization accepts whatever shape the data shows — empirical frequencies are the distribution.
🎛️ Calibrating Λ with Closed-Loop Control
Λ is not guessed — LAND replays the full policy on calibration data for each candidate Λ, simulating draws and stopping, then adjusts Λ until the average draws per source equal the target budget N. Like tuning a faucet temperature.
During deployment, closed-loop control runs every batch (e.g., 100 sources), comparing actual average draws K̄_m to N with a log-space update:
log Λ_{m+1} = log Λ_m + η·(K̄_m - N)/N
Experiments show the actual average budget deviates from target by only 0.01–0.04 draws — near-perfect tracking even under mild distribution shift.
📊 Six Experiments, One Conclusion
Setup: OPUS parallel corpus (source side only). Gemma4 31B on four translation directions (Ukrainian→German, Spanish→English, Dutch→Hungarian, Turkish→Russian); Qwen3 30B-A3B cross-validated on two directions. Metric: MetricX (higher is better). Budgets N = 1 to 32; K_max and temperature tuned per group.
Result: at equal average compute, LAND beats fixed Best-of-N on MetricX; equivalently, at equal quality it saves 20–40% of candidate generations. All six experimental groups land in the 20–40% savings band; Ukrainian→German saves the most (its bimodal difficulty profile suits adaptive allocation), Dutch→Hungarian the least. Rerun standard deviations are ~0.8–1.5 percentage points — robust to input ordering.
A notable design detail: LAND targets the average budget, not a hard per-source constraint. Individual sources may draw far more or fewer than N, but the global average is pinned near N — locally flexible, globally strict.
🆚 Why BEACON Backfired
The paper compares against BEACON (Google), a Bayesian adaptive sampler modeling each prompt's reward distribution as Gaussian with online mean/variance updates and a finite-horizon stopping rule. Surprisingly, BEACON underperforms fixed Best-of-N across all budgets, language pairs, and models — an efficiency method wasting more compute than no adaptation.
Two explanations:
1. The Gaussian assumption itself. MetricX distributions may be asymmetric, multimodal, heavy-tailed. A right-skewed true distribution leads Gaussian fits to underestimate the high-score tail, setting stopping thresholds too low — stopping early and missing rare excellent candidates. LAND tested parametric variants and hit the same problem, hence the discrete empirical mixture.
2. Cold start vs. warm start. BEACON learns each new source from scratch with a wide prior — early decisions are near-blind guesses from two scores. LAND leverages cross-source population structure: the calibrated mixture model provides a sensible prior, so even one observed score is interpretable against known tube breeds.
The methodological lesson: in low-budget regimes, exploiting population structure beats per-source independent learning. When observations per source are few, prior quality determines posterior quality.
🚧 Honest Boundaries
- Calibration data is required — multi-candidate scores, EM fitting, Λ replay. For small one-off datasets the setup cost can exceed savings. LAND suits large-scale, repeated data generation.
- The mixture model doesn't update during deployment. If deployment data differs from calibration (news vs. legal text), the model can't adapt; online EM refits would be costly.
- Candidate count proxies compute only approximately. Real GPU time scales with token length, not candidate count; mapping savings to wall-clock needs extra modeling.
- Validated only on machine translation with MetricX. The framework generalizes to any sample-score-select setting (code with tests, summarization, molecule generation), but a misaligned scorer means garbage in, garbage out.
- Oostermeijer, K., & Jones, T. (2026). LAND: Learning an Adaptive Number of Draws for Bayesian Best-of-N Selection. arXiv:2611.03388. Aleph Alpha Research.
- Weitzman, M. L. (1978). Optimal Search for the Best Alternative. Econometrica, 46(3), 641-654.
- Wan, G., et al. (2025). BEACON: Bayesian Optimal Stopping for Efficient LLM Sampling. arXiv:2510.15945.
- Raman, V., Asi, H., & Kale, S. (2025). AdaBoN: Adaptive Best-of-N Alignment. arXiv:2505.12050.
- Peter, J., et al. (2023). There's No Data Like Better Data: Using QE Metrics for MT Data Filtering. WMT 2023.
- Juraska, J., et al. (2024). MetricX-24: The Google Submission to the WMT 2024 Metrics Shared Task. WMT 2024.
- Kim, Y., & Rush, A. (2016). Sequence-Level Knowledge Distillation. EMNLP 2016.
🔮 Generalizing the Lottery-Tube Philosophy
Best-of-N is a meta-pattern of the LLM era: RLHF reward-model ranking, code generation with test filtering, Self-Consistency voting over CoT paths, multi-seed diffusion selection. LAND applies wherever you can define a continuous quality score — swap the scorer and calibration data; the core machinery (discrete empirical mixture, posterior updates, marginal-improvement stopping, Λ closed-loop control) is universal.
The deeper insight: in the LLM world, equality is not fairness, and neither is efficiency. Uniform compute per input looks fair but misallocates resources. True efficiency isn't sampling harder — it's allocating smarter: giving limited draws to tubes that still contain undrawn top prizes.
The paper invokes a 1978 economic insight: Pandora opens boxes not by a "draw N times" rule but by judging "is this box still worth opening?" Forty-plus years later, that judgment lives in Bayesian posterior updates, saving LLMs 20–40% of their compute.
---
References