01 - Implications Between Modes of Convergence

Implications Between Modes of Convergence

Let $X_1,X_2,\cdots$ and $X$ be random variables then

$$X_n\to X\text{ almost surely }⟹X_n\to X\text{ in probability }⟹X_n\to X\text{ in distribution}$$

Proof

In Probability $⟹$ In Distribution

Let $F_n$ be the distribution function of $X_n$
Let $F$ be the distribution function of $X$

For any $x$ such that $F$ is continuous at $x$ then fix any $ϵ>0$ then
If $X_n\le x$ then either

$$X\le x+ϵ\text{ or }|X_n-X|>ϵ$$

Hence

$$\begin{array}{rl}{F}_n(x)& =\mathbb{P}(X_n\le x)\\ & \le \mathbb{P}(X\le x+ϵ\text{ or }|X_n-X|>ϵ)\\ & \le \mathbb{P}(X\le x+ϵ)+\mathbb{P}(|X_n-X|>ϵ)\\ & \to F(x+ϵ)\text{ as }n\to \infty \end{array}$$

So

$$F_n(x)<F(x+ϵ)+ϵ\text{ for all large enough }n$$

Similarly as $1-F_n(x)=\mathbb{P}(X_n>x)$ then

$$F_n(x)>F(x-ϵ)-ϵ\text{ for all large enough }n$$

As $ϵ>0$ and $F$ is continuous at $x$ then

$$F_n(x)\to F(x)\text{ as }n\to \infty$$

In Distribution $\text{centernot}⟹$ In Probability

Note that random variables do not need to be defined on the same probability space

Suppose $X$ and $Y$ are random variables with same distribution but

$$P(X=Y)<1$$

Then

$$X,X,X,\cdots \text{ converge to }Y\text{ in distribution}$$

However not in probability

Almost Surely $⟹$ In Probability

For $N\in \mathbb{N}$ and fix $ϵ>0$ then

Define event $A_n$ by

$$A_N=\{|X_n-X|<ϵ\text{ for all }n\ge N\}$$

Suppose $X_n\to X$ almost surely

If event $\{X_n\to X\}$ occurs then event $A_N$ occurs for some $N$ hence

$$\mathbb{P}\left(\bigcup A_N\right)=1$$

As $A_N$ is an increasing sequence of events
By Continuity of Probability from Below then

$$\underset{N\to \infty }{\lim }\mathbb{P}(A_N)=1$$

However

$$A_N⟹|X_N-X|<ϵ$$

So

$$\mathbb{P}(|X_N-X|<ϵ)\to 1$$

Since $ϵ$ is arbitrary then

$$X_n\to X\text{ in probability}$$

In Probability $\text{centernot}⟹$ Almost Surely

Consider sequence of independent random variables $X_n$ where

$$\mathbb{P}(X_n=1)=\frac{1}{n}\text{ and }\mathbb{P}(X_n=0)=\frac{n-1}{n}$$

with

$$X_n\to 0\text{ in probability}$$

as for any $ϵ>0$

$$\mathbb{P}(|X_n-0|<ϵ)\ge \mathbb{P}(X=0)\to 1$$

As $X_n$ only takes values $0$ and $1$ then event

$$\{X_n\to 0\}\text{ is equivalent to }\{X_n=0\text{eventually}\}$$

This is

$$\bigcup _{N\ge 1}{B}_N\text{ where }{B}_N=\{X_n=0\text{ for all }n\ge N\}$$

But for any $N$ and $K$ then

$$\begin{array}{rl}\mathbb{P}(B_N)& \le \mathbb{P}(X_n=0\text{ for all }n=N,\cdots ,N+K)\\ & =\frac{N-1}{N}\frac{N}{N+1}\frac{N+1}{N+2}\cdots \frac{N+K-1}{N+K}\\ & =\frac{N-1}{N+K}\end{array}$$

As $K\ge 1$ then

$$\mathbb{P}(B_N)=0$$

Hence by Continuity of Probability from Below then

$$\mathbb{P}\left(\bigcup _{N\ge 1}{B}_N\right)=0$$

Thus

$$\mathbb{P}(X_n\to 0)=0$$

Hence $X_n$ does not converge to $0$ almost surely