Summary Probability and Statistics (Blok 2)| EOR Year 1 Tilburg University

Probability and Statistics (Statistics)

Max Batstra
April 2024




1

, Chapter 8 Statistics and Sampling
Distributions
Chapter 8.2

Definition 8.2.1 (Sample mean). Let X1 , ..., Xn be a random sample, then sample mean is defined:
n
1X
X̄ = Xi (8.2.1)
n i=1

Theorem 8.2.1. Let X1 , ..., Xn be a random sample from f (x) with E[X] = µ and Var(X) = σ 2 , then:
 
E X̄ = µ (8.2.2)
and
σ2
Var(X̄) = (8.2.3)
n
Definition 8.2.2 (Sample variance). Let X1 , ..., Xn be a random sample, then sample variance is defined:
n
2 1 X
2
S = Xi − X̄ (8.2.4)
n − 1 i=1

Theorem 8.2.2. Let X1 , ..., Xn be a random sample with E[Xi ] = µ and Var(Xi ) = σ 2 , then
E[S 2 ] = σ 2 (8.2.5)
Proof
" n # " n
# !
2 1 X
2 2 1 X
Xi2
 2
E[S ] = E Xi − nX̄ = E − nE X̄
n−1 i=1
n−1 i=1
" n
# !
1 X 2 2
X̄ − E[X̄ )2 + E[X̄]2
  

= E (Xi − E[Xi ]) + E[Xi ] −n E
n−1 i=1
σ2
  
1
n σ 2 + µ2 − n + µ2


=
n−1 n
= σ2


It’s important to note that X ∼ Exp(θ) ⇐⇒ X ∼ GAM(θ, 1) and since a gamma distributed variable can be
seen as the sum of n exponentially distributed random variables, we have that:
n
X n
X
nX̄ = Xi ∼ GAM(θ, n) ⇐⇒ 2nX̄ = 2 Xi ∼ GAM(2θ, n) = χ22·n
i=1 i=1




2

,Chapter 8.3 Sampling Distributions


Theorem 8.3.1. Let X1 , ..., Xn ∼ N µi , σi2 be a random sample and let α1 , ..., αn ∈ R. Then:
n n n
!
X X X
αi Xi ∼ N αi µi , αi2 σi2 (8.3.6)
i=1 i=1 i=1
 2

Corollary 8.3.1. If X1 , ..., Xn denotes a random sample from N µ, σ 2 then X̄ ∼ N µ, σn




Consider a special gamma distribution with θ = 2 and κ = v/2. The variable Y is said to follow a chi-square
distribution with v degrees of freedom if Y ∼ GAM(2, v/2). A special notation for this is Y ∼ χ2v

Theorem 8.3.2. (Chi-square distribution) If Y ∼ χ2v then:


MY (t) = (1 − 2t)−v/2 E(Y ) = v Var(Y ) = 2v

2X
Theorem 8.3.3 (Gamma to chi-square). If X ∼ GAM(θ, κ), then Y = ∼ χ22κ
θ
Theorem 8.3.4. If Yi ∼ χ2υi ; i = 1, ..., n are independent chi-square variables, then:
n
X
Yi ∼ χ2 Pn
(8.3.7)
i=1 vi
i=1

Theorem 8.3.5. If Z ∼ N (0, 1), then Z 2 ∼ χ21

Corollary 8.3.5. If X1 , ..., Xn are i.i.d. from N (µ, σ 2 ), then:

n 2
X (Xi − µ)
2
∼ χ2n
i=1
σ ∗

2
n X̄ − µ
∼ χ21
σ2 ∗∗

* Sum of χ21 distributed random variables.
** Root is N (0, 1) distributed (Corollary 8.3.1)

Theorem 8.3.6. If X1 , ..., Xn are i.i.d. from N (µ, σ 2 ), then:


1. X̄ and the terms Xi − X̄; i = 1, ..., n are independent
2. X̄ and S 2 are independent
(n − 1)S 2
3. ∼ χ2n−1
σ2


3

,Chapter 8.4 The t, F and BET A distributions

Theorem 8.4.1. If Z ∼ N (0, 1) and V ∼ χ2υ are independent, then the distribution of:
Z
T =p ∼ tυ (8.4.8)
V /υ
is referred to as the Student’s t distribution with υ degrees of freedom.
Theorem 8.4.3. If X1 , ..., Xn are i.i.d. from N (µ, σ 2 ), then:

X̄ − µ
√ ∼ tn−1 (8.4.9)
S/ n
Proof
√
√

√
n X̄ − µ n X̄ − µ
X̄ − µ n X̄ − µ
√ = √ = rσ = s σ
S/ n S2 S 2
(n − 1)S 2

/(n − 1)
σ2 σ2
According to Corollary 8.3.1 and Theorem 8.3.6 we have that:
√

√
n X̄ − µ
n X̄ − µ (n − 1)S 2 σ
∼ N (0, 1) , ∼ χ2n−1 according to theorem 8.4.1 : ∼ tn−1
σ2
s
σ (n − 1)S 2

/(n − 1)
σ2


Theorem 8.4.4 (Snedecor’s F distribution). If V1 ∼ χ2υ1 and V2 ∼ χ2υ2 are independent, then:
V1 /υ1 V1 υ2
= ∼ Fυ1 ,υ2 (8.4.10)
V2 /υ2 V2 υ1
Theorem 8.4.5. If X ∼ Fυ2 ,υ1 , then:
υ2
E[X] = if υ2 > 2 (8.4.11)
υ2 − 2
1
Theorem 8.4.6. If X ∼ Fυ1 ,υ2 =⇒ X ∼ Fυ2 ,υ1 and if T ∼ tυ =⇒ T 2 ∼ F1,υ
Proof : The first result is trivial, for the second result recall Z 2 ∼ χ21 and V ∼ χ2υ , we have:

Z2 Z 2 /1
T2 = = ∼ F1,υ
V /υ V /υ

Corollary 8.4.7. Consider following independent random samples: X1 , ..., Xn ∼ N (µ1 , σ12 ) and Y1 , ..., Yk ∼
N (µ2 , σ22 ), then:

(n − 1)S12 (k − 1)S22 S12 σ22
∼ χ2n−1 , ∼ χ2k−1 =⇒ ∼ Fn−1,k−1
σ12 σ22 S22 σ12


4

, Chapter 9 Point Estimation

Chapter 9.1

Notation note: We denote the class of probability distributions as P.
Notation note: We denote the parameter space as Θ .

Definition 9.1.1. [Estimator] A statistic, T = ℓ (X1 , X2 , . . . , Xn ), that is used to estimate the value of
τ (θ) is called an estimator of τ (θ), and an observed value of the statistic, t = ℓ (x1 , x2 , . . . , xn ), is called an
estimate of τ (θ).




Chapter 9.2 Some Methods of Estimation

Definition 9.2.2. [Sample Moments] If X1 , . . . , Xn is a random sample from f (x; θ1 , . . . , θk ), the first k
sample moments are given by Pn
Xj
M ′j = i=1 i j = 1, 2, . . . , k (9.2.12)
n

We will now discuss the Method of Moments. Suppose X1 , . . . , Xn is a random sample from Pθ , where
θ = (θ1 , . . . , θm ) ∈ Θ ⊂ Rm . Then we can estimate θ by developing a system of m equations that has a
unique solution, namely θ̂M M E .
MME
Definition
 9.2.3 (Method of Moments Estimator). The method of moments estimator (MME), θ̂ =
θ̂1 , . . . , θ̂m of θ solves the following m equations:
 
Mr′ = µ′r θ̂1 , . . . , θ̂m , r = 1, . . . , m

Here µ′r denotes the rth moment about the origin of Xi which is: E [X r ]. We have a system of m equations
so that the system is not overidentified and therefore we’ll have a solution.
Algorithm for finding MME
Let X1 , . . . , Xn be a random sample from Pθ , where θ = (θ1 , . . . , θm ) ∈ Θ ⊂ Rm . Then:
1. Determine the first m moments: µ′1 = Eθ [Xi ], µ′2 = Eθ [Xi2 ], . . . , µ′m = Eθ [Xim ]
Pn
2. Determine the first m sample moments: Mr′ = n1 i=1 Xir = X r , r = 1, . . . , m.
Note: if r ̸= 1, X r ̸= X̄ r
 
3. Find θ̂1 , . . . , θ̂m such that they solve: Mj′ = µ′j θ̂1 , . . . , θ̂m j = 1, 2, . . . , m



5