Summary MATH 220 The Transition Matrix

MATH 220 The Transition Matrix 2 Markov Chains 2.1 Basic Principles. 2.1.1 The Transition Matrix. A stochastic process is a mathematical model of a situation in the real world that evolves in time in a probabilistic fashion, i.e. we don't seem to be able to completely predict the future. The situation in the real world is often called a system. For the time being we model the system in discrete time as opposed to continuous time. For example, this might be because we observe the system at discrete time intervals. We let n = 0 be the initial time that we observe the system, n = 1 the next time and so on. We are interested in calculating probabilities that the system is in various states at various times. Mathematically, we describe the system by a sequence of random variables X0, X1, ..., Xn where X0 = the state of the system at time n = 0, X1 = the state of the system at time n = 1, . . .Xn = the state of the system at time n At any particular time the system is in one of s possible states that we number 1, 2, …, s. Example 1. An office copier is either in 1. good condition (G or 1), 2. poor condition (P or 2), or 3. broken (B or 3). So the possible states of the system are G = 1, P = 2 and B = 3. At the start of each day we check the condition of the copier. If we do this for a sequence of four days, we might observe that today it is G, tomorrow it is G, the next day it is P and the fourth day it is B. In this case we would have X0 = 1, X1 = 1, X2 = 2 and X3 = 3. Unfortunately, we can't predict the state of the copier in the future. One of the most basic probabilities that one might want to know is the probability of a sequence of observations x0, x1, ..., xn. We would denote this probability by Pr{X0 = x0, X1 = x1, ..., Xn = xn}