we want to infor the parameters ! In this case ,
we are parametric :
models
Chapter 1 :
a) Likelihood are functions of an unknown parameter .
O
110(U) = Px(NIO) < for a
single case
IID obs , due to independence
(1014) = Px(ip)
,
roan
↳ can
use (101) =
log((10ln)) .
Why ? log transformation
log is an
increasing
is one-to-one
,
function
, >N
Why MLE ? Likelihood
says how likely a value of the parameter is
given the data
to inter from the data :
maximise the likelihood
↳ find of the the data
mostly likely value parameter given .
b) Score Statistic , VIX
vix)
0 ETu(x)]
=
l'10m)
0
=
vologPx(n10) - derivative of
log-likelihood !
Fisher information :↑210) ; distribution changes
·
quickly
=
when
>
-
② var(v(x)) = -
Ele"(Olns] =
210) O departs from 00 ; Oo
estimate well
&
210) =
nicd
c) Sufficiency :
Def :
A partition A of sample space - is called sufficient for O if for all AjzA ,
Px/MO , MEAj)
is independent of 0
.
sufficient stat of without .
0
>
knowing a we can find the probablity an event the need to know
-
,
>
-
E at least I sufficient partition ,
ie.
knowing all the n Idatal outcome
↳ minimal Sufficient
Def If sufficient partition A sufficient paration B set
:
is
a such that
given any other , any
element of B is contained in a set element of A then A is said to be minimal sufficient
.
,
>
If T isAncient
for O and Amplete then T is minimal sufficient (Bahadur Incorem (
-
.
,
, TO FIND SUFFICIENT STATISTIC !!
① Meyman's Factorisation Theorem to this !
- My show
A statistic T is sufficient for 0 Px(u10) =
g(0 , T(x)) n(X) ·
② Exponential Family
If X, Xn are IID from a dist of the
exp family
·
·
,
.
...
Px(n10) =
expLACOBIn) + <10) + D(u)} =
try to show this
lif you start with I obe,
Then T = ZBIXi) is sufficient for 0 to because the [BIxi)
state
.
need
clearly
comes from likelihood :
TTPx (n/0)
from def .
>
-
⑤ T is sufficient for O If
(i) for all n and a such that TIu) = a
,
Px(n10 , T(n) =
a) is independent of .
0
(ii) for all 2 and I',
T(u) E PxIMIO) is of O
=
T(u'l
independent
↑x (n'10)
AND T is minimal sufficient if(you show the other direction ,
is ,
E)
Px(n(0) => T() =
TIu'l
↑ x In '10)
d) completeness
family [PX (410) 083
&
Def :
A :
of distributions on - is called complete
if E[hix)] = 0 for all 00 >
- P(nIX) =
010) = 1 for all .
OE h(X) is a zero function
any statistic n(x)
for such that the above expectation makes sense .
>
- A statistic T is said to be
complete if its
family of distributions &P + CtIO) 08] :
is complete
① Exponential
family
Suppose (X , Xn)
X is IID sample from the probability model
=
..., an
Px(n(0) =
expCAIOBIn) + col + DinI] ,
Do
and let T = [B(Xi) denote the corresponding sufficient statistic.
If & contains an open interval ,
then T is .
complete
↓ (R 1 - 0 , 3)
Eg :
,
space of O
, *
completeness of X =
completeness of the
family of distributions of .
X
Chapter 2 :
Goal :
estimate gloy ,
a function of 0
.
Setting : Let (x) be our estimate of g(0) when we observe X =
.
a
An estimator is a function of the r v
.
.
XI, ...,
Xn
.
-
L 1) Estimator not
should take values outside the parameter space .
I
2) Unbiasedness
E[(x)) =
g(0) +
bg(0)
want bg10) = 0 g(x)] =
g(0) VOzO
desirable 3) small volatility
of Squared Error IMSE) > 0
properties mean -
estimators Def :
McE =
ESigIX)-glOT"] =...
=
var((x)) + (b(0))
4)
Consistency
g(x) +
g(0)asn + 0
,
it .
for every 320 , PlIIX)-gl01K) < 10) + 0 as n+
>
-
E(g(X)] + g(0) and var(g(x)) + 0 as n + & = g(X) is consistent for glo)
construct estimators :
a) method of moments
for the r-th moment, E(x] + x
>
n+ 0
-
,
Exr] = Xi [ law of
large numbers]
[theoretical] =
[data]
Exi
Eg
: Eix] =
N
we are parametric :
models
Chapter 1 :
a) Likelihood are functions of an unknown parameter .
O
110(U) = Px(NIO) < for a
single case
IID obs , due to independence
(1014) = Px(ip)
,
roan
↳ can
use (101) =
log((10ln)) .
Why ? log transformation
log is an
increasing
is one-to-one
,
function
, >N
Why MLE ? Likelihood
says how likely a value of the parameter is
given the data
to inter from the data :
maximise the likelihood
↳ find of the the data
mostly likely value parameter given .
b) Score Statistic , VIX
vix)
0 ETu(x)]
=
l'10m)
0
=
vologPx(n10) - derivative of
log-likelihood !
Fisher information :↑210) ; distribution changes
·
quickly
=
when
>
-
② var(v(x)) = -
Ele"(Olns] =
210) O departs from 00 ; Oo
estimate well
&
210) =
nicd
c) Sufficiency :
Def :
A partition A of sample space - is called sufficient for O if for all AjzA ,
Px/MO , MEAj)
is independent of 0
.
sufficient stat of without .
0
>
knowing a we can find the probablity an event the need to know
-
,
>
-
E at least I sufficient partition ,
ie.
knowing all the n Idatal outcome
↳ minimal Sufficient
Def If sufficient partition A sufficient paration B set
:
is
a such that
given any other , any
element of B is contained in a set element of A then A is said to be minimal sufficient
.
,
>
If T isAncient
for O and Amplete then T is minimal sufficient (Bahadur Incorem (
-
.
,
, TO FIND SUFFICIENT STATISTIC !!
① Meyman's Factorisation Theorem to this !
- My show
A statistic T is sufficient for 0 Px(u10) =
g(0 , T(x)) n(X) ·
② Exponential Family
If X, Xn are IID from a dist of the
exp family
·
·
,
.
...
Px(n10) =
expLACOBIn) + <10) + D(u)} =
try to show this
lif you start with I obe,
Then T = ZBIXi) is sufficient for 0 to because the [BIxi)
state
.
need
clearly
comes from likelihood :
TTPx (n/0)
from def .
>
-
⑤ T is sufficient for O If
(i) for all n and a such that TIu) = a
,
Px(n10 , T(n) =
a) is independent of .
0
(ii) for all 2 and I',
T(u) E PxIMIO) is of O
=
T(u'l
independent
↑x (n'10)
AND T is minimal sufficient if(you show the other direction ,
is ,
E)
Px(n(0) => T() =
TIu'l
↑ x In '10)
d) completeness
family [PX (410) 083
&
Def :
A :
of distributions on - is called complete
if E[hix)] = 0 for all 00 >
- P(nIX) =
010) = 1 for all .
OE h(X) is a zero function
any statistic n(x)
for such that the above expectation makes sense .
>
- A statistic T is said to be
complete if its
family of distributions &P + CtIO) 08] :
is complete
① Exponential
family
Suppose (X , Xn)
X is IID sample from the probability model
=
..., an
Px(n(0) =
expCAIOBIn) + col + DinI] ,
Do
and let T = [B(Xi) denote the corresponding sufficient statistic.
If & contains an open interval ,
then T is .
complete
↓ (R 1 - 0 , 3)
Eg :
,
space of O
, *
completeness of X =
completeness of the
family of distributions of .
X
Chapter 2 :
Goal :
estimate gloy ,
a function of 0
.
Setting : Let (x) be our estimate of g(0) when we observe X =
.
a
An estimator is a function of the r v
.
.
XI, ...,
Xn
.
-
L 1) Estimator not
should take values outside the parameter space .
I
2) Unbiasedness
E[(x)) =
g(0) +
bg(0)
want bg10) = 0 g(x)] =
g(0) VOzO
desirable 3) small volatility
of Squared Error IMSE) > 0
properties mean -
estimators Def :
McE =
ESigIX)-glOT"] =...
=
var((x)) + (b(0))
4)
Consistency
g(x) +
g(0)asn + 0
,
it .
for every 320 , PlIIX)-gl01K) < 10) + 0 as n+
>
-
E(g(X)] + g(0) and var(g(x)) + 0 as n + & = g(X) is consistent for glo)
construct estimators :
a) method of moments
for the r-th moment, E(x] + x
>
n+ 0
-
,
Exr] = Xi [ law of
large numbers]
[theoretical] =
[data]
Exi
Eg
: Eix] =
N
