03 - Weak Law of Large Numbers

Weak Law of Large Numbers

Let $X_1,X_2,\cdots$ be i.i.d random variables with finite mean $\mu$ and

$$S_n=X_1+X_2+\cdots +X_n$$

Then

$$\frac{S_n}{n}\stackrel{P}{\to }\mu \text{ as }n\to \infty$$

So for all $ϵ>0$ then

$$\mathbb{P}\left(\left|\frac{S_n}{n}-\mu \right|<ϵ\right)\to 1\text{ as }n\to \infty$$

Proof

Let $X_i$ be i.i.d with mean $\mu$ and variance ${\sigma }^2$ then

$$\mathbb{E}\left[\frac{S_n}{n}\right]=\mu$$

and

$$\begin{array}{rl}\text{var}\left(\frac{S_n}{n}\right)& =\frac{\text{var}(S_n)}{n^2}\\ & =\frac{\text{var}(X_1)+\text{var}(X_2)+\cdots +\text{var}(X_n)}{n^2}\\ & =\frac{n{\sigma }^2}{n^2}\\ & =\frac{{\sigma }^2}{n}\end{array}$$

Fix any $ϵ>0$ then by Chebyshev’s Inequality then

$$\begin{array}{rl}\mathbb{P}\left(\left|\frac{S_n}{n}-\mu \right|\ge ϵ\right)& \le \frac{\text{var}\left(\frac{S_n}{n}\right)}{{ϵ}^2}\\ & =\frac{{\sigma }^2}{n{ϵ}^2}\\ & \to 0\text{ as }n\to \infty \end{array}$$