Alongside the Central Limit Theorem, the Law of Large Numbers is equally important. The Weak Law of Large Numbers essentially states that the sample average converges in probability toward the expected value. There is also a Strong version which I won’t discuss for now. But, both versions are of great importance in science as they imply that large sample sizes are better for estimating population averages.

Mathematically, the Weak Law states that if we let be a sequence of iid random variables each having finite mean , for any

as

In order to demonstrate this, we shall first go through two Russian inequalities.

Markov’s inequality:

If is a random variable that takes only non-negative values then for any ,

Proof: For , let

and note that since ,

Taking expectations of he previous inequality yields

so we have

Chebyshev’s inequality:

If is a random variable with finite mean and finite variance

then for any value ,

Proof: Since , we can apply Markov’s inequality with to obtain

Since iff , we have

With the above ingredients we are now ready to provide a proof of the Weak Law of Large Numbers:

Assuming that all random variables are iid with finite mean and finite variance,

and

and it follows from Chebyshev’s inequality that

Now, if we take the limit as we obtain

the desired result.