Central Limit Theorem

5 min read#probability

The sum or average of many independent random quantities tends to a normal distribution, no matter what shape the individual quantities have.

Contents

Central Limit Theorem

The central limit theorem (CLT) is the reason the bell curve rules the world. It makes a claim that sounds too strong to be true: add up many independent random quantities — whatever their individual shapes — and their sum is approximately normal. The pieces can be lopsided, flat, spiky, or weird; pile up enough of them and the lumps wash out into the same smooth, symmetric bell. The shape of the parts does not survive; only the bell does.

This is why heights, measurement errors, test scores, and the means of almost any survey come out Gaussian. Each is a sum of countless small independent influences, and the CLT erases the fingerprints of the individual causes.

The statement

Let X_1, X_2, \ldots, X_n be independent draws from any distribution with mean \mu and finite variance \sigma^2. Form their average \bar X_n. Then as n grows, the standardized average

\frac{\bar X_n - \mu}{\sigma / \sqrt{n}} \;\xrightarrow{\;n\to\infty\;}\; \mathcal{N}(0,1),
(1)

a standard normal distribution. Two facts hide inside this. First, the average converges to \mu — that is the Law of Large Numbers. Second, the fluctuations around \mu shrink like 1/\sqrt{n} and, rescaled, take the universal bell shape. The CLT is the law of large numbers' more refined companion: it describes not just where the average lands, but the exact statistics of how it gets there.

Watch a bell emerge

Below, the machine repeatedly draws k independent random numbers (each uniform on 0 to 1), averages them, and drops the result into a histogram. With k=1 the histogram is flat — a single uniform draw is equally likely anywhere. Nudge k up and the histogram sharpens into a bell; the more numbers you average, the narrower and more Gaussian it gets. Move your mouse left and right to set k (from 1 to about 12) and watch a flat slab fold itself into a bell.

The CLT in action. Each sample is the average of k independent uniform random numbers; the histogram counts millions of such averages. Mouse x sets k: at k=1 it is flat, and it sharpens to a bell as k rises. A reference normal curve is overlaid.

It is not magic — there are rules

The CLT needs independence (or near enough) and a finite variance. Quantities with wild, heavy tails — where rare giant values dominate — can break it, converging instead to other "stable" laws. And the parts must each be small relative to the sum; one overwhelming term keeps its own shape. Within those bounds, though, the theorem is astonishingly forgiving, which is exactly why the normal distribution shows up so relentlessly. The Galton Board makes the same convergence physical, and the Probability Distribution page collects the cast of characters.

See also