19 - Characterisation of Recurrence via Return Probabilities

Characterisation of Recurrence via Return Probabilities

Let $I$ be a state space then

State $i\in I$ is recurrent if and only if

$$\sum _{n=0}^{\infty }{p}_{ii}^{(n)}=\infty$$

Proof

Total number of visits to $i$ is

$$\sum _{n=0}^{\infty }1(X_n=i)$$

which as expectation

$$\sum _{n=0}^{\infty }\mathbb{E}[1(X_n=i)]=\sum _{n=0}^{\infty }\mathbb{P}(X_n=i)=\sum _{n=0}^{\infty }{p}_{ii}^{(n)}$$

If $i$ is transient then total number of visits to $i$ is geometric with parameter $1-p$ hence

$$\text{mean }=\frac{1}{1-p}<\infty$$

Otherwise if $i$ is recurrent then number of visits to $i$ is infinite with probability $1$ hence

$$\text{mean }=\infty$$