John Mount has some thoughts for us:

I want to collect some “great things to know about linear Markov chains.”

For this note we are working with a Markov chain on states that are the integers

`0`

through`k`

(`k > 0`

). A Markov chain is an iterative random process with time tracked as an increasing integer`t`

, and the next state of the chain depending only on the current (soon to be previous) state. For our linear Markov chain the only possible next states from state`i`

are:`i`

(called a “self loop” when present),`i+1`

(called up or right), and`i-1`

(called down or left). In no case does the chain progress below`0`

or above`k`

.

Click through for notes on two variants of this sort of linear Markov chain, as well as a pair of appendices containing derivation notes and Python code.