24 Sep 2003
CS Module 7
9/28
Markov chains
A Markov chain
consists of n _____, plus an n´n __________________
P.
lAt each step, we are in exactly one of the
states.
lFor
1 £ i,k
£ n, the
matrix entry Pik tells us the probability of k being
the next state, given we are currently in state i.
i
k
Pik
Pik > 0
is OK.