Here I present a very useful version of the strong law of large numbers
where we assume that a random variable has finite
variance. Briefly, the Strong Law of Large Numbers states that for large
sample sizes the sample average converges almost surely to the expectation
of the random variable as . In some cases, where we don’t have a strict upper-bounds for this may not be a good assumption and this is the case for many distributions encountered in Economics or Finance such as the Pareto distribution. But, I dare say that for most scientists this assumption holds for most of the distributions that they deal with.
Lemma: If , and then almost surely.
By the monotone convergence theorem, which implies that is finite with probability 1. Therefore, almost surely which also implies that almost surely.
Now we’re ready to proceed with the proof. But, first we must rigorously state
the version of the Strong Law of Large Numbers to be proven.
Theorem: Let be i.i.d. random variables and assume that . Let , then converges almost surely to .
Assuming that we have .
If we only consider values of n that are perfect squares, we obtain
which implies that converges to with probability
Let’s suppose the variables are non-negative. Consider some such that . We then have . It follows that
As , and since we have
Note: If doesn’t always hold, you can apply the above method to the
positive and negative parts of where and show
that the Strong Law of Large Numbers holds for this variable as well due to the linearity of expectation.