Law of large numbers

In probability theory, the weak law of large numbers states that if X₁, X₂, X₃, ... is an infinite sequence of random variables, all of which have the same expected value μ and the same finite variance σ², and they are uncorrelated (i.e., the correlation between any two of them is zero), then the sample average

<math>\overline{X}-n=(X-1+\cdots+X-n)/n</math>

converges in probability to μ. Somewhat less tersely: For any positive number ε, no matter how small, we have

<math>\lim-{n\rightarrow\infty}P\left(\left|\overline{X}-n-\mu\right|<\varepsilon\right)=1.</math>

Chebyshev's inequality is used to prove this result.

A consequence of the weak law of large numbers is the Asymptotic Equipartition Property.

The strong law of large numbers states that if X₁, X₂, X₃, ... is an infinite sequence of random variables that are independent and identically distributed, and have a common expected value μ then

<math>P\left(\lim-{n\rightarrow\infty}\overline{X}-n=\mu\right)=1,</math>

i.e., the sample average converges almost surely to μ.

This law justifies the intuitive interpretation of the expected value of a random variable as the "long-term average when sampling repeatedly".