Bilkent University Fall 2024
MATH 255 - Probability and Statistics
Final Exam Solutions
5 January 2025
Problem 1. [10pt] The joint pdf of random variables X and Y is given by:
c if (x, y) ∈ S,
fX,Y (x, y) =
0 otherwise,
where c is a constant and S is the set shown in the plot.
(a) Find the least mean square (LMS) estimator g(X) of Y .
0.5 if X ∈ [0, 1] ∪ [2, 3] ,
E[Y |X] =
1 if X ∈ [1, 2].
Before we start solving the problem, let us first find the value of c. Since the joint distribu-
1
tion fX,Y (x, y) is uniform over the region S, we have c = area(S) = 14 .
The optimal LMS estimator is given by g(X) = E[Y |X]. For that, we will characterize
E[Y |X = x]. When 0 < x < 1 and 2 < x < 3, the conditional distribution of Y given X = x,
that is fY |X (y|x) is a uniform distribution in [0, 1]. Thus, when 0 < x < 1 and 2 < x < 3,
E[Y |X = x] = 0.5. On the other hand, when 1 < x < 2, fY |X (y|x) is a uniform distribution
in [0,2]. Thus, when 1 < x < 2, E[Y |X = x] = 1. As a result, E[Y |X] can be written as:
0.5 if X ∈ [0, 1] ∪ [2, 3] ,
E[Y |X] =
1 if X ∈ [1, 2].
1
, (b) Find the mean squared error of the estimator g(X), i.e., E[(Y − g(X))2 ].
5
E[(Y − g(X))2 ] = 24 .
Next, we will find the mean squared error of the estimator g(X). For that, we will use the
uniform distribution mean and variance. If U is a continuous r.v. in between [a, b], its mean
(b−a)2
and variance is given by E[U ] = a+b2 and Var(U ) = 12 .
We will also utilize the law of iterated expectation for this part in the following form:
E[(Y − g(X))2 ] = E[E[(Y − g(X))2 |X]].
We obtain the marginal distribution of X as
1 if x ∈ [0, 1] ∪ [2, 3] ,
fX (x) = 4
1 if x ∈ [1, 2].
2
Thus, we need to find E[(Y − g(X))2 |X = x]. When 0 < x < 1 and 2 < x < 3, g(x) =
E[Y |X = x] = 0.5. Since the conditional distribution of Y given X is a uniform distribution
in [0,1] and g(x) = E[Y |X = x] = 0.5 is its conditional mean, we have E[(Y − g(X))2 |X =
2
x] = V ar(Y |X = x) = (1−0)
12 = 121
. Similarly, when 1 < x < 2, we have E[(Y − g(X))2 |X =
2
x] = V ar(Y |X = x) = (2−0)
12 = 13 . As a result, we have
1
12 if X ∈ [0, 1] ∪ [2, 3] ,
E[(Y − g(X))2 |X = x] =
1 if X ∈ [1, 2].
3
Now, we are ready to find E[(Y − g(X))2 ].
Z 3
2 2
E[(Y − g(X)) ] = E[E[(Y − g(X)) |X]] = E[(Y − g(X))2 |X = x]fX (x)dx
0
Z 1 Z 3 Z 2
1 1 1 1 11
= dx + dx + dx
0 12 4 2 12 4 1 32
1 1 1 5
= + + = .
48 48 6 24
2
MATH 255 - Probability and Statistics
Final Exam Solutions
5 January 2025
Problem 1. [10pt] The joint pdf of random variables X and Y is given by:
c if (x, y) ∈ S,
fX,Y (x, y) =
0 otherwise,
where c is a constant and S is the set shown in the plot.
(a) Find the least mean square (LMS) estimator g(X) of Y .
0.5 if X ∈ [0, 1] ∪ [2, 3] ,
E[Y |X] =
1 if X ∈ [1, 2].
Before we start solving the problem, let us first find the value of c. Since the joint distribu-
1
tion fX,Y (x, y) is uniform over the region S, we have c = area(S) = 14 .
The optimal LMS estimator is given by g(X) = E[Y |X]. For that, we will characterize
E[Y |X = x]. When 0 < x < 1 and 2 < x < 3, the conditional distribution of Y given X = x,
that is fY |X (y|x) is a uniform distribution in [0, 1]. Thus, when 0 < x < 1 and 2 < x < 3,
E[Y |X = x] = 0.5. On the other hand, when 1 < x < 2, fY |X (y|x) is a uniform distribution
in [0,2]. Thus, when 1 < x < 2, E[Y |X = x] = 1. As a result, E[Y |X] can be written as:
0.5 if X ∈ [0, 1] ∪ [2, 3] ,
E[Y |X] =
1 if X ∈ [1, 2].
1
, (b) Find the mean squared error of the estimator g(X), i.e., E[(Y − g(X))2 ].
5
E[(Y − g(X))2 ] = 24 .
Next, we will find the mean squared error of the estimator g(X). For that, we will use the
uniform distribution mean and variance. If U is a continuous r.v. in between [a, b], its mean
(b−a)2
and variance is given by E[U ] = a+b2 and Var(U ) = 12 .
We will also utilize the law of iterated expectation for this part in the following form:
E[(Y − g(X))2 ] = E[E[(Y − g(X))2 |X]].
We obtain the marginal distribution of X as
1 if x ∈ [0, 1] ∪ [2, 3] ,
fX (x) = 4
1 if x ∈ [1, 2].
2
Thus, we need to find E[(Y − g(X))2 |X = x]. When 0 < x < 1 and 2 < x < 3, g(x) =
E[Y |X = x] = 0.5. Since the conditional distribution of Y given X is a uniform distribution
in [0,1] and g(x) = E[Y |X = x] = 0.5 is its conditional mean, we have E[(Y − g(X))2 |X =
2
x] = V ar(Y |X = x) = (1−0)
12 = 121
. Similarly, when 1 < x < 2, we have E[(Y − g(X))2 |X =
2
x] = V ar(Y |X = x) = (2−0)
12 = 13 . As a result, we have
1
12 if X ∈ [0, 1] ∪ [2, 3] ,
E[(Y − g(X))2 |X = x] =
1 if X ∈ [1, 2].
3
Now, we are ready to find E[(Y − g(X))2 ].
Z 3
2 2
E[(Y − g(X)) ] = E[E[(Y − g(X)) |X]] = E[(Y − g(X))2 |X = x]fX (x)dx
0
Z 1 Z 3 Z 2
1 1 1 1 11
= dx + dx + dx
0 12 4 2 12 4 1 32
1 1 1 5
= + + = .
48 48 6 24
2
