Summary probability and statistics

Contents
1 Chapter 4 2
1.1 Discrete case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Continuous case . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 more properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.4 Empirical distribution functions . . . . . . . . . . . . . . . . . . . 2

2 Chapter 5 3
2.1 Conditional expectation . . . . . . . . . . . . . . . . . . . . . . . 3
2.2 Bivariate Normal distribution . . . . . . . . . . . . . . . . . . . . 4

3 Chapter 6 4
3.1 CDF-approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3.2 pdf-approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3.3 mgf approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.4 Order statistics and their distribution . . . . . . . . . . . . . . . 5
3.5 Jensen inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

4 chapter 7 6
4.1 Convergence in terms of densities . . . . . . . . . . . . . . . . . . 6
4.2 Weak law of large numbers (WLLN) . . . . . . . . . . . . . . . . 6
4.3 Central limit theorem . . . . . . . . . . . . . . . . . . . . . . . . 6
4.4 Delta Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

5 Chapter 8 7
5.1 Chi-square distribution . . . . . . . . . . . . . . . . . . . . . . . . 7
5.2 F- distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

6 Chapter 9 8
6.1 Method of moments . . . . . . . . . . . . . . . . . . . . . . . . . 9
6.2 Method of Maximum likelihood . . . . . . . . . . . . . . . . . . . 9
6.3 quality of estimators . . . . . . . . . . . . . . . . . . . . . . . . . 9
6.4 Cramer-Rao inequality . . . . . . . . . . . . . . . . . . . . . . . . 10




1

, 1 Chapter 4
The k-dimensional discrete random vector (X1 , ...., Xk ) has a density function
defined by:
f (x1 , ......, xk ) = P (X1 = x1 , X2 = x2 , ..., Xk = xk )
We will discuss a few multivariate distributions:
The mutinomial distribution, the extension of the binomial distribution:
xk+1
n!
P (X1 = x1 , X2 = x2 , ..., Xk = xk ) = x1 !∗x2 !∗...x∗k!∗x∗k+1! px1 1 px2 2 ∗ ...... ∗ pk+1




1.1 Discrete case
P P
Theorem: x1 .... xk f (x1 , ..., xk ) = 1 P
Definition for marginal densities: fX2 (x2 ) = allx1 f (x1 , x2 )
Definition: F (x1 , ..., xk ) = P (X1 ≤ x1 , X2 ≤ x2 , ....., Xk ≤ xk
We also can have that:
P (x11 < X1 ≤ x12 , x21 < X2 ≤ x2 2)
= F (x12 , x22 ) − F (x12 , x21 ) − F (x11 , x22 + F (x11 , x21 ), is bigger or equal to zero,
obviously since it is a probability.


1.2 Continuous case
R xk R x1
F (x1 , ...., xk ) = −∞ ..... −∞ f (t1 , ..., tk )dt1 ....dtk
Hence it also holds that:
δk
f (x1 , ....., xk ) = δx .....δx1
F (x1 , ...., xk )
1
Additionally, taking the integral of all xi from −∞ to ∞ is equal to 1.




1.3 more properties
R R
P (X ∈ Region ) = Region f (x1 , x2 )dx1 dx2
For f (and F) we have that k random variables (X1 , ..., Xk ) are said to be inde-
pendent if f (x1 , ..., xk ) = fX1 (x1 ) ∗ ..... ∗ fXk (xk )
Conditional density is: f (x2 |x1 ) = ff(x 1 ,x2 )
X1 (x1 )

For f is a uni variate density function and (X1 , ..., Xn ) an n-dimensional random
vector, if g(x1 , ...., xn ) = f (x1 ) ∗ .... ∗ f (xn ) holds for the joint density g, then
X1 , ..., Xn is said to be a random sample


1.4 Empirical distribution functions
X1:n ≤ X2:nP≤ ... ≤ Xn:n The empirical distribution function is defined by:
n
Fn (x) = n1 i=1 1(−∞,x) (Xi )


2