Law of Large Numbers
As you collect more independent samples, their running average converges to the true expected value.
Law of Large Numbers
The law of large numbers (LLN) is the promise that makes statistics possible: average enough independent samples and the average converges to the truth. Any single measurement is noisy and might mislead you. But the running mean — the average of everything you have seen so far — settles down, drifting ever closer to the underlying expected value as the sample count grows. It is the formal version of the intuition that a fair coin flipped many times lands heads "about half" the time.
This is why casinos always win, why polls of more people are sharper, and why a long experiment beats a short one. Randomness does not cancel on any single trial — it cancels in the aggregate.
The statement
For independent samples X_1, X_2, \ldots each with mean \mu, the sample average
The convergence is real but slow and shaky. Early on, a few unlucky draws can swing the average wildly. As n grows the wobble shrinks — at the same 1/\sqrt{n} rate that governs a Random Walk — and the curve flattens onto \mu. The Central Limit Theorem is the fine print here: it says precisely how big the remaining wobble is at each n.
A noisy mean settling onto the truth
Below, a fair process (think a coin paying +1 for heads, -1 for tails, true mean 0) is sampled forever. The jagged line is the running average after each new draw. Watch it: violent at the left edge where only a handful of samples exist, then steadier and steadier, hugging the dashed line at the true value as the sample count climbs into the thousands.
Why it matters for computing
The LLN is the engine under the Monte Carlo Method: to estimate a hard quantity, sample it at random many times and average — the LLN guarantees the estimate converges to the right answer, and the Central Limit Theorem tells you the error bars. Every simulation that "runs it a million times and takes the mean" is cashing in this theorem.