24 - Ergodic Theorem

Ergodic Theorem

Let $P$ be a irreducible transition matrix
Let $V_i(n)$ be the number of visits to state $i$ before time $n$ so

$$V_i(n)=\sum _{r=0}^{n-1}1(X_r=i)$$

Then for any initial distribution

$$\frac{V_i(n)}{n}\to \frac{1}{m_i}\text{ almost surely as }n\to \infty \text{ for all }i\in I$$

So

$$\mathbb{P}\left(\frac{V_i(n)}{n}\to \frac{1}{m_i}\text{ as }n\to \infty \right)=1$$