Lecture 1
An estimator is a …
- calculation rule based on a sample intended to estimate a population
parameter
- nonrandom number but a random variable
Simple regression model (OLS) – single regressor
-
Why OLS?
- Optimal statistical properties under some theoretical assumptions
(unbiasedness, consistency, asymptotic normality, efficiency)
- Necessary conditions for unbiasedness, consistency and asymptotic
normality (approximate normal distribution)
1) is a random variable with E( |Xi) = 0
2) (Yi,Xi) are independently and identically distributed (i.i.d.)
3) Large outliers are unlikely (nonzero finite fourth moments)
- OLS-estimator is BLUE under Gauss-Markov assumptions
If these 3 assumptions hold, B0 and B1 have jointly normal sampling
distribution with ~ N(b1, var(B1)) and ~N(b0, var(B0))
Deriving the OLS estimators
->
-> ->
,Proving unbiasedness (E(B1 )=B1)
Consistency
- var(B1) -> 0 as n -> infinity & E(B1) -> B1 as n -> infinity
Efficiency of OLS -> Gauss-Markov assumptions (for BLUE
estimators)
1) is a random variable with E( |Xi) = 0
2) (Yi,Xi) are iid
3) Large outliers are unlikely (nonzero finite fourth moments)
4) Homoskedasticity: var( |Xi) = population variance
5) -> no serial correlation
If then the OLS estimators a and b are actually normally
distributed
R^2
- R^2 can only be compared if the Y is the same
- this only holds if the model contains a constant
SER – root MSE
- Biased estimator of the standard deviation of the regression error ui
- Biased because you take the square root
Statistical test procedure
1. Formulate the null and alternative hypothesis in terms of population
parameters
2. Construct a test statistic and calculate it for the available sample
-
3. Specify the distribution of the test statistic
- For small N, we can’t assume that the errors are normally
distributed -> stata does assume that, so for small n, use t-value rather
, than p-value
- Under the Gauss-Markov assumptions + normal distributed errors,
the t- statistic is t(n-2) distributed
4. Specify the decision criterion to reject H0 and draw a conclusion
Confidence interval
Confidence interval for a change in X
Homoskedasticity:
Heteroskedasticity: is not constant across observations
- Don’t use robust if n = small, otherwise ALWAYS use robust in stata
Interpretation of B1 is based on first derivative.
- t-ratio = value of B1 / SE (B1)
- Standard error is never negative!
H0
True False
Reject Type 1 error Correct conclusion
Test
Don’t reject Correct conclusion Type 2 error
- You can only answer this if you know the true value of the
population parameter
Monte-Carlo-data is not real data. You know the population values, which is
never the case with real data
Lecture 2
Problem with simple regression
- From economic theory there is almost always more than one causal
factor
- Omitting relevant explanatory variables will almost always cause
biased and inconsistent simple regression OLS-estimators
- We don’t know if the differences between not using a second
variable and using it are coincidental or structural. Sample is
randomly drawn, so coincidence might play a role
Theory: omitted variable
- Real model:
- Estimated model:
- Error term is then: -> error term includes everything
An estimator is a …
- calculation rule based on a sample intended to estimate a population
parameter
- nonrandom number but a random variable
Simple regression model (OLS) – single regressor
-
Why OLS?
- Optimal statistical properties under some theoretical assumptions
(unbiasedness, consistency, asymptotic normality, efficiency)
- Necessary conditions for unbiasedness, consistency and asymptotic
normality (approximate normal distribution)
1) is a random variable with E( |Xi) = 0
2) (Yi,Xi) are independently and identically distributed (i.i.d.)
3) Large outliers are unlikely (nonzero finite fourth moments)
- OLS-estimator is BLUE under Gauss-Markov assumptions
If these 3 assumptions hold, B0 and B1 have jointly normal sampling
distribution with ~ N(b1, var(B1)) and ~N(b0, var(B0))
Deriving the OLS estimators
->
-> ->
,Proving unbiasedness (E(B1 )=B1)
Consistency
- var(B1) -> 0 as n -> infinity & E(B1) -> B1 as n -> infinity
Efficiency of OLS -> Gauss-Markov assumptions (for BLUE
estimators)
1) is a random variable with E( |Xi) = 0
2) (Yi,Xi) are iid
3) Large outliers are unlikely (nonzero finite fourth moments)
4) Homoskedasticity: var( |Xi) = population variance
5) -> no serial correlation
If then the OLS estimators a and b are actually normally
distributed
R^2
- R^2 can only be compared if the Y is the same
- this only holds if the model contains a constant
SER – root MSE
- Biased estimator of the standard deviation of the regression error ui
- Biased because you take the square root
Statistical test procedure
1. Formulate the null and alternative hypothesis in terms of population
parameters
2. Construct a test statistic and calculate it for the available sample
-
3. Specify the distribution of the test statistic
- For small N, we can’t assume that the errors are normally
distributed -> stata does assume that, so for small n, use t-value rather
, than p-value
- Under the Gauss-Markov assumptions + normal distributed errors,
the t- statistic is t(n-2) distributed
4. Specify the decision criterion to reject H0 and draw a conclusion
Confidence interval
Confidence interval for a change in X
Homoskedasticity:
Heteroskedasticity: is not constant across observations
- Don’t use robust if n = small, otherwise ALWAYS use robust in stata
Interpretation of B1 is based on first derivative.
- t-ratio = value of B1 / SE (B1)
- Standard error is never negative!
H0
True False
Reject Type 1 error Correct conclusion
Test
Don’t reject Correct conclusion Type 2 error
- You can only answer this if you know the true value of the
population parameter
Monte-Carlo-data is not real data. You know the population values, which is
never the case with real data
Lecture 2
Problem with simple regression
- From economic theory there is almost always more than one causal
factor
- Omitting relevant explanatory variables will almost always cause
biased and inconsistent simple regression OLS-estimators
- We don’t know if the differences between not using a second
variable and using it are coincidental or structural. Sample is
randomly drawn, so coincidence might play a role
Theory: omitted variable
- Real model:
- Estimated model:
- Error term is then: -> error term includes everything