Probability Theory & Statistics 2
Complete Course Summary
Multivariate Distributions, Transformations & Sampling Distributions,
Estimation and Confidence Intervals, Hypothesis Testing
Lecture Notes, Step-by-Step Solution Guides & Exam-Style Practice Questions
Midterm: Chapters 5–6 — Final: Chapters 5–8
,Probability Theory & Statistics 2 Complete Course Summary
Contents
How to use this document 3
I Lecture Notes & Worked Examples 3
1 Multivariate Distributions (Chapter 5) 3
1.1 Joint, marginal and cumulative distributions . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Named multivariate discrete distributions . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Expected values of functions of several variables . . . . . . . . . . . . . . . . . . . . 4
1.4 Independence, covariance, correlation . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.5 Conditional distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.6 Joint moment generating functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.7 Bivariate and multivariate normal . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2 Functions of Multivariate Random Variables (Chapter 6) 6
2.1 Three methods for transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2 Sums of independent random variables . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3 The χ2 , t and F distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.4 Samples, sample mean and variance . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.5 Order statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3 Estimation and Confidence Intervals (Chapter 7) 8
3.1 Point estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.2 Confidence intervals: definition and interpretation . . . . . . . . . . . . . . . . . . 8
3.3 Confidence intervals for one population . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.4 Confidence intervals for two populations . . . . . . . . . . . . . . . . . . . . . . . . 8
4 Hypothesis Testing (Chapter 8) 9
4.1 Fundamental concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4.2 The p-value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4.3 Tests for one population . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4.4 Tests for two populations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
II Quick-Reference Step-by-Step Guides 10
5 Guide: Joint Distributions (Chapter 5) 11
6 Guide: Transformations and Sampling Distributions (Chapter 6) 12
7 Guide: Estimation (Chapter 7) 13
8 Guide: Hypothesis Testing (Chapter 8) 13
1
,Probability Theory & Statistics 2 Complete Course Summary
III Exam-Style Practice Questions 14
9 Midterm-Style Questions (Chapters 5–6) 14
10 Final-Style Questions (Chapters 7–8) 16
11 Mixed Conceptual Questions 18
2
,Probability Theory & Statistics 2 Complete Course Summary
How to use this document
Part I collects the theory chapter by chapter, with short worked examples. Part II holds one-page
“what do I do on the exam” guides: use them as checklists while solving. Part III has practice
questions (Midterm set on Chapters 5–6, Final set on Chapters 7–8) with complete solutions. Try
each question on paper first.
Conventions: Φ is the standard normal cdf, zα the point with upper tail probability α (z0.05 = 1.645,
z0.025 = 1.96, z0.005 = 2.576), and tν;α , χ2ν;α , fν1 ,ν2 ;α are upper-tail points.
Part I
Lecture Notes & Worked Examples
1 Multivariate Distributions (Chapter 5)
1.1 Joint, marginal and cumulative distributions
Definition
For random variables X, Y the joint cdf is FX,Y (x, y) = P[X ≤ x, Y ≤P y]. P
Discrete: joint pdf fX,Y (x, y) = P[X = x, Y = y], with f ≥ 0 and x y f (x, y) = 1.
Ry Rx
Continuous: FX,Y (x, y) = −∞ −∞ fX,Y (u, v) du dv, with f ≥ 0 and
RR
f = 1. Con-
∂ 2 FX,Y
versely fX,Y = .
∂x ∂y
Marginals: fX (x) = y f (x, y) resp. f (x, y) dy; similarly for Y .
P R
A function F is a bivariate cdf iff it tends to 0 when either argument → −∞, to 1 when both → ∞,
is right-continuous, and satisfies the rectangle inequality F (b, d) − F (b, c) − F (a, d) + F (a, c) ≥ 0
for a < b, c < d.
Multivariate versions. For X ⃗ = (X1 , . . . , Xk ): joint pdf f (⃗x) ≥ 0 integrating/summing to 1;
the marginal of Xj sums/integrates out all other variables.
1.2 Named multivariate discrete distributions
Distribution Situation pdf
n x
n−x
Binomial BIN(n, p) n independent trials, 2 outcomes; x p (1 − p)
X=#successes
n! xk+1 P
Multinomial n independent trials, k + 1 out- px1 1 · · · pk+1 , xj = n
x1 ! · · · xk+1 !
MULT(n, p1 ..pk ) comes; Xj =#times outcome j
M N −M
N
Hypergeometric n draws without replacement from x n−x n
HYP(n, M, N ) N items, M of type S
Mi N
Q
Multivariate hyper- as above with k + 1 categories of i xi n
geometric sizes Mi
Merging categories: in a multinomial, adding categories gives a multinomial again with probabil-
ities added (e.g. “5 or 6” becomes one category with p = 2/6). Each single count Xj ∼ BIN(n, pj ).
3
,Probability Theory & Statistics 2 Complete Course Summary
1.3 Expected values of functions of several variables
Theorem
P P RR
E[g(X, Y )] = x y g(x, y)f (x, y) (discrete) or g(x, y)f (x, y) dx dy (continuous). Lin-
P P
earity: E[aX + bY + c] = aEX + bEY + c always, and E ai Xi = ai EXi .
1.4 Independence, covariance, correlation
Definition
X, Y are independent iff FX,Y = FX FY , equivalently fX,Y = fX fY for all (x, y).
Cov(X, Y )
Cov(X, Y ) = E[(X − µX )(Y − µY )] = E[XY ] − EX EY . ρXY = ∈ [−1, 1].
σX σY
Theorem (Independence test for pdfs)
X, Y with joint pdf are independent iff (1) the support of (X, Y ) is a Cartesian product
A × B, and (2) fX,Y (x, y) = g(x)h(y) on it (the pdf factorises).
If X, Y independent: E[g(X)h(Y )] = Eg(X) Eh(Y ); E[XY ] = EXEY ; Cov = 0; MX+Y =
MX MY ; GX+Y = GX GY ; Var(X + Y ) = Var X + Var Y .
Watch out
Independent ⇒ uncorrelated, but not the converse (exception: bivariate normal). A trian-
gular support (x < y) always means dependent.
Rules. Cov(aX, bY ) = ab Cov(X, Y ); Cov(X + a, Y + b) = Cov(X, Y ); Cov(X, aX + b) = a Var X;
Cov(X + U, Y ) = Cov(X, Y ) + Cov(U, Y ).
Var(aX + bY + c) = a2 Var X + b2 Var Y + 2ab Cov(X, Y ),
X X X
Var ai Xi = a2i Var Xi + 2 ai aj Cov(Xi , Xj ).
i<j
|ρ| = 1 iff Y = aX + b a.s.; ρ(aX + c, bY + d) = sign(ab) ρ(X, Y ). Mutual vs. pairwise
independence: X1 , . . . , Xk are mutually independent iff the joint pdf/cdf is the product of all
marginals. Pairwise independence is weaker (e.g. two fair coins X, Y and Z = 1 if they differ:
every pair is independent, but Z is determined by (X, Y )).
Worked example
f (x, y) = c(2x + y), x ∈ {1, 2}, y ∈ {0, 1, 2}. Sum of 2x + y over all cells is 24, so c = 1/24.
Marginals: fX = (3/8, 5/8), fY = (1/4, 1/3, 5/12). EX = 13/8, EY = 7/6, E[XY ] = 15/8,
hence Cov = 15/8 − (13/8)(7/6) = −1/48 ̸= 0: dependent.
4
Complete Course Summary
Multivariate Distributions, Transformations & Sampling Distributions,
Estimation and Confidence Intervals, Hypothesis Testing
Lecture Notes, Step-by-Step Solution Guides & Exam-Style Practice Questions
Midterm: Chapters 5–6 — Final: Chapters 5–8
,Probability Theory & Statistics 2 Complete Course Summary
Contents
How to use this document 3
I Lecture Notes & Worked Examples 3
1 Multivariate Distributions (Chapter 5) 3
1.1 Joint, marginal and cumulative distributions . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Named multivariate discrete distributions . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Expected values of functions of several variables . . . . . . . . . . . . . . . . . . . . 4
1.4 Independence, covariance, correlation . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.5 Conditional distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.6 Joint moment generating functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.7 Bivariate and multivariate normal . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2 Functions of Multivariate Random Variables (Chapter 6) 6
2.1 Three methods for transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2 Sums of independent random variables . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3 The χ2 , t and F distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.4 Samples, sample mean and variance . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.5 Order statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3 Estimation and Confidence Intervals (Chapter 7) 8
3.1 Point estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.2 Confidence intervals: definition and interpretation . . . . . . . . . . . . . . . . . . 8
3.3 Confidence intervals for one population . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.4 Confidence intervals for two populations . . . . . . . . . . . . . . . . . . . . . . . . 8
4 Hypothesis Testing (Chapter 8) 9
4.1 Fundamental concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4.2 The p-value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4.3 Tests for one population . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4.4 Tests for two populations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
II Quick-Reference Step-by-Step Guides 10
5 Guide: Joint Distributions (Chapter 5) 11
6 Guide: Transformations and Sampling Distributions (Chapter 6) 12
7 Guide: Estimation (Chapter 7) 13
8 Guide: Hypothesis Testing (Chapter 8) 13
1
,Probability Theory & Statistics 2 Complete Course Summary
III Exam-Style Practice Questions 14
9 Midterm-Style Questions (Chapters 5–6) 14
10 Final-Style Questions (Chapters 7–8) 16
11 Mixed Conceptual Questions 18
2
,Probability Theory & Statistics 2 Complete Course Summary
How to use this document
Part I collects the theory chapter by chapter, with short worked examples. Part II holds one-page
“what do I do on the exam” guides: use them as checklists while solving. Part III has practice
questions (Midterm set on Chapters 5–6, Final set on Chapters 7–8) with complete solutions. Try
each question on paper first.
Conventions: Φ is the standard normal cdf, zα the point with upper tail probability α (z0.05 = 1.645,
z0.025 = 1.96, z0.005 = 2.576), and tν;α , χ2ν;α , fν1 ,ν2 ;α are upper-tail points.
Part I
Lecture Notes & Worked Examples
1 Multivariate Distributions (Chapter 5)
1.1 Joint, marginal and cumulative distributions
Definition
For random variables X, Y the joint cdf is FX,Y (x, y) = P[X ≤ x, Y ≤P y]. P
Discrete: joint pdf fX,Y (x, y) = P[X = x, Y = y], with f ≥ 0 and x y f (x, y) = 1.
Ry Rx
Continuous: FX,Y (x, y) = −∞ −∞ fX,Y (u, v) du dv, with f ≥ 0 and
RR
f = 1. Con-
∂ 2 FX,Y
versely fX,Y = .
∂x ∂y
Marginals: fX (x) = y f (x, y) resp. f (x, y) dy; similarly for Y .
P R
A function F is a bivariate cdf iff it tends to 0 when either argument → −∞, to 1 when both → ∞,
is right-continuous, and satisfies the rectangle inequality F (b, d) − F (b, c) − F (a, d) + F (a, c) ≥ 0
for a < b, c < d.
Multivariate versions. For X ⃗ = (X1 , . . . , Xk ): joint pdf f (⃗x) ≥ 0 integrating/summing to 1;
the marginal of Xj sums/integrates out all other variables.
1.2 Named multivariate discrete distributions
Distribution Situation pdf
n x
n−x
Binomial BIN(n, p) n independent trials, 2 outcomes; x p (1 − p)
X=#successes
n! xk+1 P
Multinomial n independent trials, k + 1 out- px1 1 · · · pk+1 , xj = n
x1 ! · · · xk+1 !
MULT(n, p1 ..pk ) comes; Xj =#times outcome j
M N −M
N
Hypergeometric n draws without replacement from x n−x n
HYP(n, M, N ) N items, M of type S
Mi N
Q
Multivariate hyper- as above with k + 1 categories of i xi n
geometric sizes Mi
Merging categories: in a multinomial, adding categories gives a multinomial again with probabil-
ities added (e.g. “5 or 6” becomes one category with p = 2/6). Each single count Xj ∼ BIN(n, pj ).
3
,Probability Theory & Statistics 2 Complete Course Summary
1.3 Expected values of functions of several variables
Theorem
P P RR
E[g(X, Y )] = x y g(x, y)f (x, y) (discrete) or g(x, y)f (x, y) dx dy (continuous). Lin-
P P
earity: E[aX + bY + c] = aEX + bEY + c always, and E ai Xi = ai EXi .
1.4 Independence, covariance, correlation
Definition
X, Y are independent iff FX,Y = FX FY , equivalently fX,Y = fX fY for all (x, y).
Cov(X, Y )
Cov(X, Y ) = E[(X − µX )(Y − µY )] = E[XY ] − EX EY . ρXY = ∈ [−1, 1].
σX σY
Theorem (Independence test for pdfs)
X, Y with joint pdf are independent iff (1) the support of (X, Y ) is a Cartesian product
A × B, and (2) fX,Y (x, y) = g(x)h(y) on it (the pdf factorises).
If X, Y independent: E[g(X)h(Y )] = Eg(X) Eh(Y ); E[XY ] = EXEY ; Cov = 0; MX+Y =
MX MY ; GX+Y = GX GY ; Var(X + Y ) = Var X + Var Y .
Watch out
Independent ⇒ uncorrelated, but not the converse (exception: bivariate normal). A trian-
gular support (x < y) always means dependent.
Rules. Cov(aX, bY ) = ab Cov(X, Y ); Cov(X + a, Y + b) = Cov(X, Y ); Cov(X, aX + b) = a Var X;
Cov(X + U, Y ) = Cov(X, Y ) + Cov(U, Y ).
Var(aX + bY + c) = a2 Var X + b2 Var Y + 2ab Cov(X, Y ),
X X X
Var ai Xi = a2i Var Xi + 2 ai aj Cov(Xi , Xj ).
i<j
|ρ| = 1 iff Y = aX + b a.s.; ρ(aX + c, bY + d) = sign(ab) ρ(X, Y ). Mutual vs. pairwise
independence: X1 , . . . , Xk are mutually independent iff the joint pdf/cdf is the product of all
marginals. Pairwise independence is weaker (e.g. two fair coins X, Y and Z = 1 if they differ:
every pair is independent, but Z is determined by (X, Y )).
Worked example
f (x, y) = c(2x + y), x ∈ {1, 2}, y ∈ {0, 1, 2}. Sum of 2x + y over all cells is 24, so c = 1/24.
Marginals: fX = (3/8, 5/8), fY = (1/4, 1/3, 5/12). EX = 13/8, EY = 7/6, E[XY ] = 15/8,
hence Cov = 15/8 − (13/8)(7/6) = −1/48 ̸= 0: dependent.
4
