,Stochastic Processes: An Introduction
Solutions Manual
Third Edition
c Peter W Jones and Peter Smith
School of Computing and Mathematics, Keele University, UK
, Preface
The website includes answers and solutions of all the end-of-chapter problems in the textbook
Stochastic Processes: An Introduction, third edition. We hope that they will prove helpful to
lecturers in designing courses, and to students as a source of model examples. The original problems
as numbered in the text are also included.
There are obviously references to results and examples from the textbook, and the manual
should be viewed as a supplement to the book. To help identify the sections and chapters, the full
contents of Stochastic Processes follow this preface.
Every effort has been made to eliminate misprints or errors (or worse), and the authors, who
were responsible for the LaTeX code, apologise in advance for any which occur.
Peter W. Jones
Peter Smith Keele, 2017
1
, Contents of Stochastic Processes
Chapter 1: Some Background in Probability
1.1 Introduction
1.2 Probability
1.3 Conditional probability and independence
1.4 Discrete random variables
1.5 Continuous random variables
1.6 Mean and variance
1.7 Some standard discrete probability distributions
1.8 Some standard continuous probability distributions
1.9 Generating functions
1.10 Conditional expectation
Problems
Chapter 2: Some Gambling Problems
2.1 Gambler’s ruin
2.2 Probability of ruin
2.3 Some numerical simulations
2.4 Expected duration of the game
2.5 Some variations of gambler’s ruin
2.5.1 The infinitely rich opponent
2.5.2 The generous gambler
2.5.3 Changing the stakes
Problems
Chapter 3: Random Walks
3.1 Introduction
3.2 Unrestricted random walks
3.3 Probability distribution after n steps
3.4 First returns of the symmetric random walk
Problems
Chapter 4: Markov Chains
4.1 States and transitions
4.2 Transition probabilities
4.3 General two-state Markov chain
4.4 Powers of the transition matrix for the m-state chain
4.5 Gambler’s ruin as a Markov chain
4.6 Classification of states
4.7 Classification of chains
4.8 A wildlife Markov chain model
Problems
Chapter 5: Poisson Processes
5.1 Introduction
5.2 The Poisson process
5.3 Partition theorem approach
5.4 Iterative method
5.5 The generating function
5.6 Variance for the Poisson process
2
,5.7 Arrival times
5.8 Summary of the Poisson process
Problems
Chapter 6: Birth and Death Processes
6.1 Introduction
6.2 The birth process
6.3 Birth process: generating function equation
6.4 The death process
6.5 The combined birth and death process
6.6 General population processes
Problems
Chapter 7: Queues
7.1 Introduction
7.2 The single server queue
7.3 The stationary process
7.4 Queues with multiple servers
7.5 Queues with fixed service times
7.6 Classification of queues
Problems
Chapter 8: Reliability and Renewal
8.1 Introduction
8.2 The reliability function
8.3 The exponential distribution and reliability
8.4 Mean time to failure
8.5 Reliability of series and parallel systems
8.6 Renewal processes
8.7 Expected number of renewals
Problems
Chapter 9: Branching and Other Random Processes
9.1 Introduction
9.2 Generational growth
9.3 Mean and variance
9.4 Probability of extinction
9.5 Branching processes and martingales
9.6 Stopping rules
9.7 A continuous time epidemic
9.8 A discrete time epidemic model
9.9 Deterministic epidemic models
9.10 An iterative scheme for the simple epidemic
Problems
Chapter 10: Brownian Motion: Wiener Process
10.1 Introduction
10.2 Brownian motion
10.3 Wiener process as a limit of a random walk
10.4 Brownian motion with drift
10.5 Scaling
10.6 First visit times
3
,10.7 Other Brownian motions in one dimension
10.8 Brownian motion in more than one dimension
Problems
Chapter 11: Computer Simulations and Projects
4
, Contents of the Solutions Manual
Chapter 1: Some Background in Probability 6
Chapter 2: Some Gambling Problems 20
Chapter 3: Random Walks 34
Chapter 4: Markov Chains 49
Chapter 5: Poisson Processes 74
Chapter 6: Birth and Death Processes 80
Chapter 7: Queues 105
Chapter 8: Reliability and Renewal 121
Chapter 9: Branching and Other Random Processes 130
Chapter 10: Brownian Motion: Wiener Process 150
5
,Chapter 1
Some background in probability
1.1. The Venn diagram of three events is shown in Figure 1.5(in the text). Indicate on the diagram the
following events:
(a) A ∪ B; (b) A ∪ (B ∪ C); (c) A ∩ (B ∪ C); (d) (A ∩ C)c ; (e) (A ∩ B) ∪ C c .
S S
A B A B
C C
(a) (b)
S S
A B A B
C C
(c) (d)
S
A B
C
(e)
Figure 1.1:
The events are shaded in Figure 1.1.
1.2. In a random experiment, A, B, C are three events. In set notation write down expressions for the
events:
(a) only A occurs;
(b) all three events A, B, C occur;
(c) A and B occur but C does not;
(d) at least one of the events A, B, C occurs;
(e) exactly one of the events A, B, C occurs;
(f ) not more than two of the events occur.
(a) A ∩ (B ∪ C)c ; (b) A ∩ (B ∩ C) = A ∩ B ∩ C; (c) (A ∩ B) ∩ C c ; (d) A ∪ B ∪ C;
6
,(e) A ∩ (B ∪ C)c represents an event in A but not in either B nor C: therefore the answer is
(A ∩ (B ∪ C)c ) ∪ (B ∩ (A ∪ C)c ) ∪ (C ∩ (A ∪ B)c ).
(f) A ∩ B ∩ C
1.3. For two events A and B, P(A) = 0.4, P(B) = 0.5 and P(A ∩ B) = 0.3. Calculate
(a) P(A ∪ B); (b) P(A ∩ B c ); (c) P(Ac ∪ B c ).
(a) From (1.1) P(A ∪ B) = P(A) + P(B) − P(A ∩ B), it follows that
P(A ∪ B) = 0.4 + 0.5 − 0.3 = 0.6.
(b) Since A = (A ∩ B c ) ∪ (A ∩ B) and A ∩ B c , and A ∩ B are mutually exclusive, then,
P(A) = P[(A ∩ B c ) ∪ (A ∩ B)] = P(A ∩ B c ) + P(A ∩ B),
so that
P(A ∩ B c ) = P(A) − P(A ∩ B) = 0.4 − 0.3 = 0.1.
(c) Since Ac ∪ B c = (A ∩ B)c , then
P(Ac ∪ B c ) = P[(A ∩ B)c ] = 1 − P(A ∩ B) = 1 − 0.3 = 0.7.
1.4. Two distinguishable fair dice a and b are rolled. What are the elements of the sample space? What is
the probability that the sum of the face values of the two dice is 9? What is the probability that at least one
5 or at least one 3 appears?
The 36 elements of the sample space are listed in Example 1.1. The event A1 , that the sum is 9, is
given by
A1 = {(3, 6), (4, 5), (5, 4), (6, 3)}.
4
Hence P = 36 = 19 .
Let A2 be the event that at least one 5 or at least one 3 appears. Then by counting the elements in
20
the sample space in Example 1.1, P(A2 ) = 36 = 59 .
1.5. Two distinguishable fair dice a and b are rolled. What is the probability that the sum of the faces is
not more than 6?
Let the random variable X be the sum of the faces. By counting events in the sample space in Example
15 5
1.1, P(X) = 36 = 12 .
1.6. For the probability generating function
1
G(s) = (2 − s)− 2
find {pn } and its mean.
Note that G(1) = 1. Using the binomial theorem (see the Appendix)
∞
1 1 1 1 X − 21 n
s − 21
G(s) = √ (1 − s)− 2 = √ where = 1.
2 2 2 n=0 n 2 0
Hence
1 1 1 − 21
p0 = √ , pn = √ n , (n = 1, 2, . . .).
2 22 n
The mean is
3
µ = G′ (1) = 1
2
(2 − s)− 2 = 12 .
s=1
7
, 1.7. Find the probability generating function G(s) of the Poisson distribution (see Section 1.7) with param-
eter α given by
e−α αn
pn = , n = 0, 1, 2, . . . .
n!
Determine the mean and variance of {pn } from the generating function.
Given pn = e−α αn /n!, the generating function is given by
∞ ∞ ∞
X X e−α αn sn X (αs)n
G(s) = p n sn = = e−α = eα(s−1) .
n! n!
n=0 n=0 n=0
The mean and variance are given by
d α(s−1)
µ = G′ (1) = e = α,
ds s=1
σ 2 = G′′ (1) + G′ (1) − [G′ (1)]2 = [α2 eα(s−1) + αeα(s−1) − α2 e2α(s−1) ]s=1 = α.
1.8. A panel contains n warning lights. The times to failure of the lights are the independent random
variables T1 , T2 , . . . , Tn which have exponential distributions with parameters α1 , α2 , . . . , αn respectively.
Let T be the random variable of the time to first failure, that is
T = min{T1 , T2 , . . . , Tn }.
Pn
Show that T has an exponential distribution with parameter j=1
αj .
The probability that no warning light has failed by time t is
P(T ≥ t) = P(T1 ≥ t ∩ T2 ≥ t ∩ · · · ∩ Tn ≥ t)
= P(T1 ≥ t)P(T2 ≥ t) · · · P(Tn ≥ t)
= e−α1 t e−α2 t · · · e−αn t = e−(α1 +α2 +···+αn )t .
1.9. The geometric distribution with parameter p is given by
p(x) = q x−1 p, x = 1, 2, . . .
where q = 1−p (see Section 1.7). Find its probability generating function. Calculate the mean and variance
of the geometric distribution from its pgf.
The generating function is given by
∞ ∞
X pX p qs ps
G(s) = q x−1 psx = (qs)x = = ,
q q 1 − qs 1 − qs
x=1 x=1
using the formula for the sum of a geometric series.
The mean is given by
′ d ps p pqs 1
µ = G (1) = = + = .
ds 1 − qs s=1
1 − qs (1 − qs)2 s=1
p
For the variance,
′′ d p 2pq
G (s) = = .
ds (1 − qs)2 (1 − qs)3
is required. Hence
2q 1 1 q
σ 2 = G′′ (1) + G′ (1) − [G′ (1)]2 = + − 2 = 2.
p2 p p p
8
Solutions Manual
Third Edition
c Peter W Jones and Peter Smith
School of Computing and Mathematics, Keele University, UK
, Preface
The website includes answers and solutions of all the end-of-chapter problems in the textbook
Stochastic Processes: An Introduction, third edition. We hope that they will prove helpful to
lecturers in designing courses, and to students as a source of model examples. The original problems
as numbered in the text are also included.
There are obviously references to results and examples from the textbook, and the manual
should be viewed as a supplement to the book. To help identify the sections and chapters, the full
contents of Stochastic Processes follow this preface.
Every effort has been made to eliminate misprints or errors (or worse), and the authors, who
were responsible for the LaTeX code, apologise in advance for any which occur.
Peter W. Jones
Peter Smith Keele, 2017
1
, Contents of Stochastic Processes
Chapter 1: Some Background in Probability
1.1 Introduction
1.2 Probability
1.3 Conditional probability and independence
1.4 Discrete random variables
1.5 Continuous random variables
1.6 Mean and variance
1.7 Some standard discrete probability distributions
1.8 Some standard continuous probability distributions
1.9 Generating functions
1.10 Conditional expectation
Problems
Chapter 2: Some Gambling Problems
2.1 Gambler’s ruin
2.2 Probability of ruin
2.3 Some numerical simulations
2.4 Expected duration of the game
2.5 Some variations of gambler’s ruin
2.5.1 The infinitely rich opponent
2.5.2 The generous gambler
2.5.3 Changing the stakes
Problems
Chapter 3: Random Walks
3.1 Introduction
3.2 Unrestricted random walks
3.3 Probability distribution after n steps
3.4 First returns of the symmetric random walk
Problems
Chapter 4: Markov Chains
4.1 States and transitions
4.2 Transition probabilities
4.3 General two-state Markov chain
4.4 Powers of the transition matrix for the m-state chain
4.5 Gambler’s ruin as a Markov chain
4.6 Classification of states
4.7 Classification of chains
4.8 A wildlife Markov chain model
Problems
Chapter 5: Poisson Processes
5.1 Introduction
5.2 The Poisson process
5.3 Partition theorem approach
5.4 Iterative method
5.5 The generating function
5.6 Variance for the Poisson process
2
,5.7 Arrival times
5.8 Summary of the Poisson process
Problems
Chapter 6: Birth and Death Processes
6.1 Introduction
6.2 The birth process
6.3 Birth process: generating function equation
6.4 The death process
6.5 The combined birth and death process
6.6 General population processes
Problems
Chapter 7: Queues
7.1 Introduction
7.2 The single server queue
7.3 The stationary process
7.4 Queues with multiple servers
7.5 Queues with fixed service times
7.6 Classification of queues
Problems
Chapter 8: Reliability and Renewal
8.1 Introduction
8.2 The reliability function
8.3 The exponential distribution and reliability
8.4 Mean time to failure
8.5 Reliability of series and parallel systems
8.6 Renewal processes
8.7 Expected number of renewals
Problems
Chapter 9: Branching and Other Random Processes
9.1 Introduction
9.2 Generational growth
9.3 Mean and variance
9.4 Probability of extinction
9.5 Branching processes and martingales
9.6 Stopping rules
9.7 A continuous time epidemic
9.8 A discrete time epidemic model
9.9 Deterministic epidemic models
9.10 An iterative scheme for the simple epidemic
Problems
Chapter 10: Brownian Motion: Wiener Process
10.1 Introduction
10.2 Brownian motion
10.3 Wiener process as a limit of a random walk
10.4 Brownian motion with drift
10.5 Scaling
10.6 First visit times
3
,10.7 Other Brownian motions in one dimension
10.8 Brownian motion in more than one dimension
Problems
Chapter 11: Computer Simulations and Projects
4
, Contents of the Solutions Manual
Chapter 1: Some Background in Probability 6
Chapter 2: Some Gambling Problems 20
Chapter 3: Random Walks 34
Chapter 4: Markov Chains 49
Chapter 5: Poisson Processes 74
Chapter 6: Birth and Death Processes 80
Chapter 7: Queues 105
Chapter 8: Reliability and Renewal 121
Chapter 9: Branching and Other Random Processes 130
Chapter 10: Brownian Motion: Wiener Process 150
5
,Chapter 1
Some background in probability
1.1. The Venn diagram of three events is shown in Figure 1.5(in the text). Indicate on the diagram the
following events:
(a) A ∪ B; (b) A ∪ (B ∪ C); (c) A ∩ (B ∪ C); (d) (A ∩ C)c ; (e) (A ∩ B) ∪ C c .
S S
A B A B
C C
(a) (b)
S S
A B A B
C C
(c) (d)
S
A B
C
(e)
Figure 1.1:
The events are shaded in Figure 1.1.
1.2. In a random experiment, A, B, C are three events. In set notation write down expressions for the
events:
(a) only A occurs;
(b) all three events A, B, C occur;
(c) A and B occur but C does not;
(d) at least one of the events A, B, C occurs;
(e) exactly one of the events A, B, C occurs;
(f ) not more than two of the events occur.
(a) A ∩ (B ∪ C)c ; (b) A ∩ (B ∩ C) = A ∩ B ∩ C; (c) (A ∩ B) ∩ C c ; (d) A ∪ B ∪ C;
6
,(e) A ∩ (B ∪ C)c represents an event in A but not in either B nor C: therefore the answer is
(A ∩ (B ∪ C)c ) ∪ (B ∩ (A ∪ C)c ) ∪ (C ∩ (A ∪ B)c ).
(f) A ∩ B ∩ C
1.3. For two events A and B, P(A) = 0.4, P(B) = 0.5 and P(A ∩ B) = 0.3. Calculate
(a) P(A ∪ B); (b) P(A ∩ B c ); (c) P(Ac ∪ B c ).
(a) From (1.1) P(A ∪ B) = P(A) + P(B) − P(A ∩ B), it follows that
P(A ∪ B) = 0.4 + 0.5 − 0.3 = 0.6.
(b) Since A = (A ∩ B c ) ∪ (A ∩ B) and A ∩ B c , and A ∩ B are mutually exclusive, then,
P(A) = P[(A ∩ B c ) ∪ (A ∩ B)] = P(A ∩ B c ) + P(A ∩ B),
so that
P(A ∩ B c ) = P(A) − P(A ∩ B) = 0.4 − 0.3 = 0.1.
(c) Since Ac ∪ B c = (A ∩ B)c , then
P(Ac ∪ B c ) = P[(A ∩ B)c ] = 1 − P(A ∩ B) = 1 − 0.3 = 0.7.
1.4. Two distinguishable fair dice a and b are rolled. What are the elements of the sample space? What is
the probability that the sum of the face values of the two dice is 9? What is the probability that at least one
5 or at least one 3 appears?
The 36 elements of the sample space are listed in Example 1.1. The event A1 , that the sum is 9, is
given by
A1 = {(3, 6), (4, 5), (5, 4), (6, 3)}.
4
Hence P = 36 = 19 .
Let A2 be the event that at least one 5 or at least one 3 appears. Then by counting the elements in
20
the sample space in Example 1.1, P(A2 ) = 36 = 59 .
1.5. Two distinguishable fair dice a and b are rolled. What is the probability that the sum of the faces is
not more than 6?
Let the random variable X be the sum of the faces. By counting events in the sample space in Example
15 5
1.1, P(X) = 36 = 12 .
1.6. For the probability generating function
1
G(s) = (2 − s)− 2
find {pn } and its mean.
Note that G(1) = 1. Using the binomial theorem (see the Appendix)
∞
1 1 1 1 X − 21 n
s − 21
G(s) = √ (1 − s)− 2 = √ where = 1.
2 2 2 n=0 n 2 0
Hence
1 1 1 − 21
p0 = √ , pn = √ n , (n = 1, 2, . . .).
2 22 n
The mean is
3
µ = G′ (1) = 1
2
(2 − s)− 2 = 12 .
s=1
7
, 1.7. Find the probability generating function G(s) of the Poisson distribution (see Section 1.7) with param-
eter α given by
e−α αn
pn = , n = 0, 1, 2, . . . .
n!
Determine the mean and variance of {pn } from the generating function.
Given pn = e−α αn /n!, the generating function is given by
∞ ∞ ∞
X X e−α αn sn X (αs)n
G(s) = p n sn = = e−α = eα(s−1) .
n! n!
n=0 n=0 n=0
The mean and variance are given by
d α(s−1)
µ = G′ (1) = e = α,
ds s=1
σ 2 = G′′ (1) + G′ (1) − [G′ (1)]2 = [α2 eα(s−1) + αeα(s−1) − α2 e2α(s−1) ]s=1 = α.
1.8. A panel contains n warning lights. The times to failure of the lights are the independent random
variables T1 , T2 , . . . , Tn which have exponential distributions with parameters α1 , α2 , . . . , αn respectively.
Let T be the random variable of the time to first failure, that is
T = min{T1 , T2 , . . . , Tn }.
Pn
Show that T has an exponential distribution with parameter j=1
αj .
The probability that no warning light has failed by time t is
P(T ≥ t) = P(T1 ≥ t ∩ T2 ≥ t ∩ · · · ∩ Tn ≥ t)
= P(T1 ≥ t)P(T2 ≥ t) · · · P(Tn ≥ t)
= e−α1 t e−α2 t · · · e−αn t = e−(α1 +α2 +···+αn )t .
1.9. The geometric distribution with parameter p is given by
p(x) = q x−1 p, x = 1, 2, . . .
where q = 1−p (see Section 1.7). Find its probability generating function. Calculate the mean and variance
of the geometric distribution from its pgf.
The generating function is given by
∞ ∞
X pX p qs ps
G(s) = q x−1 psx = (qs)x = = ,
q q 1 − qs 1 − qs
x=1 x=1
using the formula for the sum of a geometric series.
The mean is given by
′ d ps p pqs 1
µ = G (1) = = + = .
ds 1 − qs s=1
1 − qs (1 − qs)2 s=1
p
For the variance,
′′ d p 2pq
G (s) = = .
ds (1 − qs)2 (1 − qs)3
is required. Hence
2q 1 1 q
σ 2 = G′′ (1) + G′ (1) − [G′ (1)]2 = + − 2 = 2.
p2 p p p
8
