Math 255 Introduction to Probability and Statistics

Bilkent University
Spring 2020-21
Math 255 Introduction to Probability and Statistics
Final Exam 28 May 2021
Solutions


1. [10 pts] Let X be a Bernoulli(θ) random variable where θ is modeled as a sample of a random
variable Θ uniform on [0, 1]. In other words, we use a Bayesian model for the pair (Θ, X)
so that (
1, 0 ≤ θ ≤ 1,
fΘ (θ) =
0, otherwise,
and (
θ, x = 1,
pX|Θ (x|θ) =
1 − θ, x = 0.

(a) (5 pts) Compute the Least Mean Squares (LMS) estimate of Θ, namely Θ̂(x) =
E[Θ|X = x], as a function of a single sample x ∈ {0, 1} of X.

2 
(b) (5 pts) Compute the resulting mean squared error E Θ − Θ̂(X) .

Solution. (a) We first compute the posterior probability. First note that
(R 1
θdθ, x = 1;
Z
pX (x) = fΘ (θ)pX|Θ (x|θ)dθ = R01
0 (1 − θ)dθ, x = 0
1 2 1
(
θ , x = 1;
= 21 0 2 1
− 2 (1 − θ) 0 , x = 0
(
1
, x = 1;
= 12
2 , x = 0.

Using this, we have

2θ, x = 1, 0 ≤ θ ≤ 1,
fΘ (θ)pX|Θ (x|θ) 
fΘ|X (θ|x) = = 2(1 − θ), x = 0, 0 ≤ θ ≤ 1,
pX (x) 
0, otherwise


The LMS estimate is given by
(R 1
2θ2 dθ, x=1
Z
E[Θ|X = x] = θfΘ|X (θ|x) = R01
0 2θ(1 − θ)dθ, x = 0
(
2/3, x = 1
=
1/3, x = 0

(b) The MSE is given by

 

2  
2 
E Θ − Θ̂(X) = E E Θ − Θ̂(X) | X
1 
2  1 
2 
= E Θ − Θ̂(X) | X = 0 + E Θ − Θ̂(X) | X = 1
2 2


This study source was downloaded by 100000899606396 from CourseHero.com on 09-25-2025 13:53:33 GMT -05:00


https://www.coursehero.com/file/125671807/Final2020-21Spring-Solutionspdf/