AI Engineering Foundations 06: The Bell Curve Isn't Designed — It's Accumulated

Forum topic · 小凯 · 2026-10-02

Summary

Lesson 06 of the 'AI Engineering from Scratch' series (based on Rohit Ghumare's 58k-star GitHub course) covers probability and distributions in 75 minutes. The author frames probability as the language models speak: a classifier's [0.03, 0.91, 0.06] is a distribution, not a confidence claim; LLM sampling and diffusion models are both distribution sampling. The post distills the language into three axioms, explains conditional probability, discrete vs. continuous distributions (PMF vs. PDF), and common distributions (Bernoulli, categorical, Poisson, normal). It highlights the Central Limit Theorem — bell curves emerge from summing many small effects — and practical engineering pitfalls: log-space computation to avoid floating-point underflow, the max-subtraction trick in softmax, and cross-entropy as negative log probability of the correct class. A verification section ('customs check') finds the course's numbers accurate but flags two caveats: float-precision claims hold in float32 but not Python's default float64 (underflow happens around 150 terms, not 30; exp(102) doesn't overflow in float64), and the CLT claim omits the finite-variance assumption — Cauchy distributions never become bell-shaped, as an experiment with 2000 groups shows. Includes a worked exercise computing softmax [0.24, 0.05, 0.01, 0.66, 0.04] and cross-entropy ≈0.42, verified against PyTorch to 10⁻⁸.

This is lesson 06 of the "AI Engineering Foundations" series, working through the open-source course *ai-engineering-from-scratch* (58k stars on GitHub, by Rohit Ghumare), one lesson at a time. Today: Probability and Distributions, a Learn-type lesson, 75 minutes. The first five lessons laid down matrices, gradients, and the chain rule; this one switches to another language — probability. Every number a model outputs, whether 0.91 or 0.03, is a word in that language.

Probability is the language models speak

A classifier outputs [0.03, 0.91, 0.06]. These three numbers sum to 1, each between 0 and 1. It's not saying "I'm 91% confident" — it's saying "I'm placing a probability distribution over these three answers." A language model picks the next token from ~50,000 candidates: every step computes 50,000 probabilities, then draws. Diffusion models generate images by repeatedly sampling from noise distributions. If you can't read probability, you can't read anything a model says.

The language has exactly three rules of grammar: 1. The probability of any event is not less than zero. 2. All possibilities sum to 1 — something must happen. 3. For mutually exclusive events, probabilities add.

Bayes' theorem, expectation, distributions — all derived from these three. There is no fourth.

Conditional probability

P(A|B), the probability of A given that B happened, is the intersection divided by B. In a deck, 12 face cards (J, Q, K) include 4 Kings. Given you drew a face card, the probability it's a King is 4/12 = 1/3. One step further and this division becomes Bayes — the main course of the next lesson.

Distributions

The dividing line is whether outcomes are countable:

Discrete (PMF): dice rolls. Each outcome gets a probability directly; a fair die gives 1/6 per face, summing to exactly 1.
Continuous (PDF): height. Density at a single point is not a probability; you integrate over an interval. Density can exceed 1 — a uniform distribution on [0, 0.5] has density 2, because the height is only 0.5 so the width must double. Remember: density is the concentration of probability, not probability itself.

Common distributions and their temperaments:

Bernoulli: one trial, two outcomes; mean p, variance p(1−p) — the native language of binary classification.
Categorical: its multi-way cousin; softmax output is exactly this.
Poisson: rare events; mean and variance both λ.
Normal: the protagonist. Bell-shaped, determined by mean and variance alone: 68% of data within 1 standard deviation, 95% within 2, 99.7% within 3. The flattest distribution, a die, has expectation 3.5, variance 35/12, standard deviation 1.71.

The bell curve is accumulated, not designed

The most counterintuitive fact of the lesson: roll 1 die and the distribution is flat. Average 2 dice and it becomes a triangle. Average 30 dice and you get a near-perfect bell. The course says "this holds for any starting distribution" — hold that thought (see the verification section). Why do weight initializations use normal distributions? Why is SGD gradient noise approximately normal? Why is measurement error normal? Because they're all sums of many small independent things. From the same vine: given a mean and variance, the normal distribution has maximum entropy — with no extra knowledge, it's the only unbiased choice.

The engineering pitfall: underflow

A 50-word sentence with per-word probability 0.01 gives a joint probability of 0.01 multiplied 50 times: 10⁻¹⁰⁰. Go a bit longer and the product underflows to exactly 0 — floating point can't hold numbers that small. The fix: move multiplication into log space, turning products into sums. Fifty instances of −4.61 sum to −230.26, a perfectly manageable finite number. Log-probabilities are always ≤ 0; the more negative, the less likely. Your daily training loss is exactly this.

From logits to probabilities

A model's raw output isn't a probability — it's logits, unnormalized scores. Softmax translates: exponentiate each score, then divide by the sum. The exponential does two jobs: pushes scores positive and amplifies gaps. z=[2, 1, 0.1] becomes [0.66, 0.24, 0.10].

A survival trick: subtract the max before exponentiating. z=[100, 101, 102] shifts to [−2, −1, 0] — mathematically identical, but avoids overflow on large numbers. PyTorch's cross-entropy loss does this log-softmax internally, all in one step.

Cross-entropy: the terminus

With one-hot labels, cross-entropy has one meaning: the negative log probability of the correct class. If the model gives the correct answer 0.66, the loss is −log(0.66) ≈ 0.42; raise that probability to 1 and the loss hits zero. All of training is pushing this one number down.

Verification ("customs check")

The lesson's numbers are unusually clean: P(King|Face)=1/3, the three log values −4.6/−5.8/−3.9, joint distribution table marginals, dice 3.5/2.9167/1.7078, the measured standard deviation of three averaged dice — all match, zero error.

Two loose spots, one underlying disease — unspoken floating-point precision:

"Underflows after ~30 terms": in float64, 0.01³⁰ is actually 10⁻⁶⁰, far from underflow — it takes about 150 terms to reach zero. In float32 it does hit zero at 23 terms.
"exp(102) overflows": in float64 it's only 2×10⁴⁴; you need exp(709) to blow up. In float32 it genuinely becomes inf.

Deep learning defaults to float32, so both examples hold in real training contexts — but if you reproduce them in Python's default float64, the textbook's overflow simply doesn't happen. The textbook isn't wrong; it just never said which floating point it was standing on.

One slogan is too broad: "this holds for any starting distribution." The full conditions of the Central Limit Theorem are i.i.d. plus finite variance. The Cauchy distribution is the standard counterexample — its tails are so heavy it has no mean. Tested: 2000 groups of 30 Cauchy samples each, averaged; the standard deviation was 53.8 and the distribution stayed heavy-tailed. Thirty additions, still no bell. The course swallowed the finite-variance premise when teaching the CLT; students waiting for a bell with heavy-tailed data will wait forever. One honesty item on the ledger: the Step 7 visualization code is pseudocode with ellipsis placeholders; the docs state the full implementation lives in code/probability.py, which wasn't included.

Exercise 3, worth doing now

A 5-class classifier outputs logits [2.0, 0.5, −1.0, 3.0, 0.1]; the correct class is index 3. Following the lesson's method: softmax gives [0.24, 0.05, 0.01, 0.66, 0.04], the correct class has probability ≈0.66, and cross-entropy is −log(0.66) ≈ 0.42. Reconcile against PyTorch's nn.CrossEntropyLoss — the two should agree to within 10⁻⁸. Once you've computed that number, you've earned your first words in this language.