I know which are the recurrent states {2,4,5} and It's clear to see it drawing a diagram but how can I prove that {2,4,5} are the recurrent states?
In a formal way?
Also the transient states are {0,1,3} but I saw it intuitively, how can I prove it formally?
$\endgroup$ 61 Answer
$\begingroup$Use First Entrance Theorem, which states that for any states $i$ and $j$ and integer $n\geq 1$, \begin{equation*} p_{ij}^{(n)}=\sum_{m=1}^{n}f_{ij}^{(m)}p_{jj}^{(n-m)} \end{equation*} where the zero-transition probabilities are defined by \begin{equation*} p_{ij}^{(0)}=\begin{cases} 1,& \text{ if } i=j\\ 0,& \text{ if } i\neq j\\ \end{cases} \end{equation*} If state space is moderately large, then the following recursive formula will be useful: \begin{equation*} F^{(n+1)}=P\left( F^{(n)}-F_{d}^{(n)}\right) \end{equation*} where $P$ is the transition probability matrix of a Markov chain, $F^{(n)}=\left(f_{ij}^{(n)}\right)$ and $F_{d}^{(n)}$ denotes a diagonal matrix containing the diagonal elements of $F^{(d)}$.
$\endgroup$ 6
