Solutions Manual for Stochastic Processes with R An Introduction 1st Edition By Olga Korosteleva

ALL 9 CHAPTER COVERED
u u u




SOLUTIONS MANUAL u

, TABLEOFCONTENTS
u u




CHAPTER 1 ……………………………………………………………………………………. 3
u u u




CHAPTER 2 ……………………………………………………………………………………. 31
u u u




CHAPTER 3 ……………………………………………………………………………………. 41
u u u




CHAPTER 4 ……………………………………………………………………………………. 48
u u u




CHAPTER 5 ……………………………………………………………………………………. 60
u u u




CHAPTER 6 ……………………………………………………………………………………. 67
u u u




CHAPTER 7 ……………………………………………………………………………………. 74
u u u




CHAPTER 8 ……………………………………………………………………………………. 81
u u u




CHAPTER 9 ……………………………………………………………………………………. 87
u u u




2

, CHAPTER 1 u




0.3 0.4 0.3 u u u u




EXERCISE1.1. Fora Markovchainwith aone-steptransitionprobabilitymatrix �0.2 0.3 0.5� u u u u u u u u u u u u u u u u u u




0.8 0.1 0.1 u u u u



we compute: u




(a) 𝑃𝑃(𝑋𝑋3 =2|𝑋𝑋0 =1, 𝑋𝑋1 =2, 𝑋𝑋2 =3)=𝑃𝑃(𝑋𝑋3 =2|𝑋𝑋2 =3)
u
u
u u
u
u u
u
u u
u
u u u
u
u u u
u
u (by the Markov property)
u u u




= 𝑃𝑃32 = 0.1. u
u
u




(b)𝑃𝑃(𝑋𝑋4 =3|𝑋𝑋0 =2, 𝑋𝑋3 =1)=𝑃𝑃(𝑋𝑋4 =3|𝑋𝑋3 =1)
u
u
u u
u
u u
u
u u u
u
u u u
u
u (by the Markov property)
u u u




= 𝑃𝑃13 = 0.3. u
u
u




(c) 𝑃𝑃(𝑋𝑋0 = 1,𝑋𝑋1 = 2,𝑋𝑋2 = 3,𝑋𝑋3 = 1) = 𝑃𝑃(𝑋𝑋3 = 1 | 𝑋𝑋0 = 1,𝑋𝑋1 = 2,𝑋𝑋2 = 3)𝑃𝑃(𝑋𝑋2 = 3 |𝑋𝑋0 = 1,
u
u
u u
u
u u
u
u u
u
u u u
u
u u u
u
u u
u
u u
u
u u
u
u u
u
u




𝑋𝑋1 = 2) 𝑃𝑃(𝑋𝑋1 = 2 | 𝑋𝑋0 = 1)𝑃𝑃(𝑋𝑋0 = 1) (byconditioning)
u
u u
u
u u u
u
u u
u
u u u




= 𝑃𝑃(𝑋𝑋3 = 1 | 𝑋𝑋2 = 3) 𝑃𝑃(𝑋𝑋2 = 3 | 𝑋𝑋1 = 2) 𝑃𝑃(𝑋𝑋1 = 2 | 𝑋𝑋0 = 1) 𝑃𝑃(𝑋𝑋0 = 1) (by the Markov property)
u
u
u u u
u
u u
u
u u u
u
u u
u
u u u
u
u u
u u
u u u u u




=𝑃𝑃31 𝑃𝑃23 𝑃𝑃12 𝑃𝑃(𝑋𝑋0 =1)=(0.8)(0.5)(0.4)(1)=0.16.
u
u u u u
u u u u u




(d) We first compute the two-step transition probability matrix. We obtain
u u u u u u u u u u




0.3 0.4 0.3 u u u u 0.3 0.4 0.3 u u u u 0.41 0.27 0.32
𝐏𝐏(2) = �0.2 0.3 0.5��0.2 0.3 0.5� =� u

u u u u u u u u u u u u 0.52 0.22 0.26�.
Now we write u u 0.8 0.1 0.1 u u u u 0.8 0.1 0.1 u u u u 0.34 0.36 0.30
𝑃𝑃(𝑋𝑋0 = 1,𝑋𝑋1 = 2,𝑋𝑋3 = 3,𝑋𝑋5 = 1) = 𝑃𝑃(𝑋𝑋5 = 1 | 𝑋𝑋0 = 1,𝑋𝑋1 = 2,𝑋𝑋3 = 3)𝑃𝑃(𝑋𝑋3 = 3 |𝑋𝑋0 = 1,
u
u u
u
u u
u
u u
u
u u u
u
u u u
u
u u
u
u u
u
u u
u
u u
u
u




𝑋𝑋1 = 2) 𝑃𝑃(𝑋𝑋1 = 2 | 𝑋𝑋0 = 1)𝑃𝑃(𝑋𝑋0 = 1) (byconditioning)
u
u u
u
u u u
u
u u
u
u u u




= 𝑃𝑃(𝑋𝑋5 = 1 | 𝑋𝑋3 = 3) 𝑃𝑃(𝑋𝑋3 = 3 | 𝑋𝑋1 = 2) 𝑃𝑃(𝑋𝑋1 = 2 | 𝑋𝑋0 = 1) 𝑃𝑃(𝑋𝑋0 = 1) (by the Markov property)
(0.34)(0.26)(0.4)(1) = 0.03536.
𝑃𝑃 𝑃𝑃(𝑋𝑋 = 1) =
u u u u u u u u u u u u u u u u u u u u u


(2) (2)
u u u u u u u u

uu u u u u u




= 𝑃𝑃31 u
𝑃𝑃23 12 0

EXERCISE 1.2. (a) We plot a diagram of the Markov chain. u u u u u u u u u u u




#specifying transition probability matrix u u u




tm<- matrix(c(1, 0, 0, 0, 0, 0.5, 0, 0, 0, 0.5, 0.2, 0, 0, 0, 0.8,
u u u u u u u u u u u u u u u




0, 0, 1, 0, 0, 0, 0, 0, 1, 0), nrow=5, ncol=5, byrow=TRUE)
u u u u u u u u u u u u




#transposing transition probability matrix tm.tr<- u u u u




t(tm)
u




#plotting diagram library(diagram) u u




plotmat(tm.tr, arr.length=0.25, arr.width=0.1, box.col="light blue", box.lwd=1, u u u u u




box.prop=0.5, box.size=0.12, box.type="circle", cex.txt=0.8, lwd=1,
u u u u u




self.cex=0.3,self.shiftx=0.01, self.shifty=0.09)
u u u




3