Empirical Economics Overview CHoogteijling
Empirical Economics Overview
Contents
1 Potential outcomes and causality 3
1.1 Correlation and causation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Potential outcome model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Treatment effect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Linear regression model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.5 Heterogeneous treatment effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.6 Selection bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2 Instrumental Variables 5
2.1 Omitted variable bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 Instrumental Variables (IV) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3 Two Stage Least Squares (2SLS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.4 Wald Estimator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.5 Testing instrument properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.6 Testing for endogeneity of regressor (Hausman test) . . . . . . . . . . . . . . . . . . . . . 9
2.7 Testing for validity of the instruments (Sargan-test/J-test) . . . . . . . . . . . . . . . . . . 10
2.8 Testing for instrument relevance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.9 Weak instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.10 Local average treatment effect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2
2.11 Partial R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3 Randomized Controlled Trial (RCT) 12
3.1 Power analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.2 Internal validity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.3 External validity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4 Regression Discontinuity 17
4.1 Fuzzy RD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
4.2 RD diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
5 Binary outcomes 18
5.1 Binary choice models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
5.2 Linear probability model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
5.3 Probit and Logit models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
6 Panel data models 21
6.1 Types of datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
6.2 Pooled regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
6.3 Fixed effects model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
, page 1 of 34
,Empirical Economics Overview CHoogteijling
6.4 Within-estimator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
6.5 Random effects model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
6.6 Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
6.7 Dynamic panel data model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
7 Difference-in-difference 24
7.1 DiD in a regression framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
8 Machine Learning and Econometrics 26
8.1 The Regression Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
8.2 Bias-Variance Trade-off . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
8.3 Model selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
8.4 Cross validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
8.5 Machine learning models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
9 Application videos 30
9.1 Bonus Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
9.2 Tournament experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
9.3 Admission Lotteries to Medical School . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
9.4 The only child . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
9.5 Job search assistance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
9.6 The (un)importance of school assignment . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
9.7 Long term effects of relative age in school . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
9.8 Discontinuities in the age-victimization profile . . . . . . . . . . . . . . . . . . . . . . . . . 33
9.9 High achievers? Cannabis Access and Academic Performance . . . . . . . . . . . . . . . . 33
9.10 Parental Birth Order: Insights on Intergenerational Mobility of Human Capital . . . . . . 34
, page 2 of 34
,Empirical Economics Overview CHoogteijling
1 Potential outcomes and causality
1.1 Correlation and causation
Correlation is a statistical measure that expresses the extent to which two variables are linearly related,
meaning they can change together at a constant rate.
Causation is the presence of a demonstrated relationship between two events, often expressed through
statistical changes in one variable due to another.
• Ex post evaluation. What would have happened.
• Ex ante prediction. What would happen.
• We often observe only one potential outcome.
The formula for correlation is a measure of how Y and D are related. The correlation is bounded between
(−1, 1). Pn
Cov(Y, D) (Yi − Y )(Di − D)
corr(Y, D) = p = qP i=1
Var(Y )Var(D) n
(Yi − Y )2
Pn
(Di − D)2
i=1 i=1
Correlation between D and Y can be caused by:
• A causal effect D on Y .
• A causal effect Y on D.
• Omitted variables: Z affects both D and Y .
1.2 Potential outcome model
The general model to think about causal effects is:
1. Assume a treatment or choice variable that can take two values.
2. Each individual i has two potential outcomes: Y1i∗ with treatment, and Y0i∗ without treatment.
3. For an individual the effect of participating in the treatment is ∆i = Y1i∗ − Y0i∗ . ∆i is always an
unobserved random variable, because only one of the random variables Y1i∗ , Y0i∗ is observed.
• Factual. Potential outcome is observed.
• Counterfactual. Unobserved outcome.
• Power. The probability that the null hypothesis is rejected if the alternative is true.
• Covariate. An independent variable that can influence the outcome of a given statistical trial, but
which is not of direct interest.
1.3 Treatment effect
The average treatment effect (ATE) is the average benefit from receiving the treatment in the popu-
lation.
ATE = E(∆) = E(Y1∗ − Y0∗ ) = E(Y1∗ ) − E(Y0∗ )
• The ATE is often not most interesting.
• It also considers individuals who did not receive treatment.
• Not all people will have the same treatment effect, some could get a lot of effect and some none.
, page 3 of 34
, Empirical Economics Overview CHoogteijling
The average treatment effect on the treated (ATET) shows the benefit of the treatment for indi-
viduals that receive the treatment.
ATET = E(∆|D = 1) = E(Y1∗ − Y0∗ |D = 1) = E(Y1∗ |D = 1) − E(Y0∗ |D = 1)
• Would be misleading if the not treated group would already have a better effect without treatment.
• Random assignment solves the self-selection problem.
E(Y1∗ ) = E(Y1∗ |D = 1) = E(Y1∗ |D = 1)
E(Y0∗ ) = E(Y0∗ |D = 1) = E(Y0∗ |D = 1)
ATE = ATET
The difference in means estimator is a standard statistic that measures the absolute difference
between the mean value in two groups in research. It estimates the amount by which the experimental
intervention changes the outcome on average compared with the control.
Let Yi = (1−Di )Y0∗ +(1−Di )Y1∗ be the observed outcome. We can estimate the conditional expectations,
the resulting estimator is the difference-in-means estimator ATE.ˆ The estimator does not impose any
structure on the model.
Pn
ˆ Di Yi
E(Y1 |D = 1) = Pi=1
∗
n
Di
Pni=1
ˆ = 0) = P i=1 (1 − Di )Yi
E(Y0∗ |D n
1 − Di
Pn i=1 Pn
ˆ = ATET ˆ = Pi=1 Di Yi i=1 (1 − Di )Yi
ATE n − P n
D
i=1 i i=1 1 − Di
We improve the difference-in-means estimator, by writing the potential outcome model as a regression
model. An advantage of OLS is that it provides standard errors.We have the regression model for the
potential outcome model, where Y0∗ = α + Ui , and Y1∗ = α + δi + Ui . Randomization ensures that the
zero conditional mean assumption E(Ui |Di ) = 0 holds.
ATE = E(Yi∗ |D = 1) − E(Yi∗ |D = 0)
= α + δ + E(Ui |D = 1) − α − E(Ui |D = 0)
=δ
1.4 Linear regression model
Individual characteristics Xi can be included in the regression equation.
Yi = α + δDi + βXi + Ui
• To provide consistent estimators δ and β, we need the zero conditional mean assumption E(Ui |Di , Xi ) =
0
• To estimate δ consistently, we only need E(Ui |Di , Xi ) = E(Ui |Xi ). This holds with random assign-
ment or conditional random assignment.
• Do not include intermediate outcomes or other confounding variables in Xi .
• If the estimated treatment effect differs with covariates, ’worry’ about the additional covariates of
the random assignment.
• The advantage of including controls can increase precision of the estimated treatment effects.
• Conditional random assignment. Treatment assignment depends on (some of) the individual char-
acteristics Xi .
• Intermediate outcome. Other factors that have an effect on the outcome during the treatment.
, page 4 of 34
Empirical Economics Overview
Contents
1 Potential outcomes and causality 3
1.1 Correlation and causation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Potential outcome model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Treatment effect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Linear regression model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.5 Heterogeneous treatment effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.6 Selection bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2 Instrumental Variables 5
2.1 Omitted variable bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 Instrumental Variables (IV) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3 Two Stage Least Squares (2SLS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.4 Wald Estimator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.5 Testing instrument properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.6 Testing for endogeneity of regressor (Hausman test) . . . . . . . . . . . . . . . . . . . . . 9
2.7 Testing for validity of the instruments (Sargan-test/J-test) . . . . . . . . . . . . . . . . . . 10
2.8 Testing for instrument relevance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.9 Weak instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.10 Local average treatment effect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2
2.11 Partial R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3 Randomized Controlled Trial (RCT) 12
3.1 Power analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.2 Internal validity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.3 External validity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4 Regression Discontinuity 17
4.1 Fuzzy RD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
4.2 RD diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
5 Binary outcomes 18
5.1 Binary choice models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
5.2 Linear probability model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
5.3 Probit and Logit models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
6 Panel data models 21
6.1 Types of datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
6.2 Pooled regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
6.3 Fixed effects model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
, page 1 of 34
,Empirical Economics Overview CHoogteijling
6.4 Within-estimator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
6.5 Random effects model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
6.6 Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
6.7 Dynamic panel data model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
7 Difference-in-difference 24
7.1 DiD in a regression framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
8 Machine Learning and Econometrics 26
8.1 The Regression Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
8.2 Bias-Variance Trade-off . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
8.3 Model selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
8.4 Cross validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
8.5 Machine learning models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
9 Application videos 30
9.1 Bonus Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
9.2 Tournament experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
9.3 Admission Lotteries to Medical School . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
9.4 The only child . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
9.5 Job search assistance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
9.6 The (un)importance of school assignment . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
9.7 Long term effects of relative age in school . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
9.8 Discontinuities in the age-victimization profile . . . . . . . . . . . . . . . . . . . . . . . . . 33
9.9 High achievers? Cannabis Access and Academic Performance . . . . . . . . . . . . . . . . 33
9.10 Parental Birth Order: Insights on Intergenerational Mobility of Human Capital . . . . . . 34
, page 2 of 34
,Empirical Economics Overview CHoogteijling
1 Potential outcomes and causality
1.1 Correlation and causation
Correlation is a statistical measure that expresses the extent to which two variables are linearly related,
meaning they can change together at a constant rate.
Causation is the presence of a demonstrated relationship between two events, often expressed through
statistical changes in one variable due to another.
• Ex post evaluation. What would have happened.
• Ex ante prediction. What would happen.
• We often observe only one potential outcome.
The formula for correlation is a measure of how Y and D are related. The correlation is bounded between
(−1, 1). Pn
Cov(Y, D) (Yi − Y )(Di − D)
corr(Y, D) = p = qP i=1
Var(Y )Var(D) n
(Yi − Y )2
Pn
(Di − D)2
i=1 i=1
Correlation between D and Y can be caused by:
• A causal effect D on Y .
• A causal effect Y on D.
• Omitted variables: Z affects both D and Y .
1.2 Potential outcome model
The general model to think about causal effects is:
1. Assume a treatment or choice variable that can take two values.
2. Each individual i has two potential outcomes: Y1i∗ with treatment, and Y0i∗ without treatment.
3. For an individual the effect of participating in the treatment is ∆i = Y1i∗ − Y0i∗ . ∆i is always an
unobserved random variable, because only one of the random variables Y1i∗ , Y0i∗ is observed.
• Factual. Potential outcome is observed.
• Counterfactual. Unobserved outcome.
• Power. The probability that the null hypothesis is rejected if the alternative is true.
• Covariate. An independent variable that can influence the outcome of a given statistical trial, but
which is not of direct interest.
1.3 Treatment effect
The average treatment effect (ATE) is the average benefit from receiving the treatment in the popu-
lation.
ATE = E(∆) = E(Y1∗ − Y0∗ ) = E(Y1∗ ) − E(Y0∗ )
• The ATE is often not most interesting.
• It also considers individuals who did not receive treatment.
• Not all people will have the same treatment effect, some could get a lot of effect and some none.
, page 3 of 34
, Empirical Economics Overview CHoogteijling
The average treatment effect on the treated (ATET) shows the benefit of the treatment for indi-
viduals that receive the treatment.
ATET = E(∆|D = 1) = E(Y1∗ − Y0∗ |D = 1) = E(Y1∗ |D = 1) − E(Y0∗ |D = 1)
• Would be misleading if the not treated group would already have a better effect without treatment.
• Random assignment solves the self-selection problem.
E(Y1∗ ) = E(Y1∗ |D = 1) = E(Y1∗ |D = 1)
E(Y0∗ ) = E(Y0∗ |D = 1) = E(Y0∗ |D = 1)
ATE = ATET
The difference in means estimator is a standard statistic that measures the absolute difference
between the mean value in two groups in research. It estimates the amount by which the experimental
intervention changes the outcome on average compared with the control.
Let Yi = (1−Di )Y0∗ +(1−Di )Y1∗ be the observed outcome. We can estimate the conditional expectations,
the resulting estimator is the difference-in-means estimator ATE.ˆ The estimator does not impose any
structure on the model.
Pn
ˆ Di Yi
E(Y1 |D = 1) = Pi=1
∗
n
Di
Pni=1
ˆ = 0) = P i=1 (1 − Di )Yi
E(Y0∗ |D n
1 − Di
Pn i=1 Pn
ˆ = ATET ˆ = Pi=1 Di Yi i=1 (1 − Di )Yi
ATE n − P n
D
i=1 i i=1 1 − Di
We improve the difference-in-means estimator, by writing the potential outcome model as a regression
model. An advantage of OLS is that it provides standard errors.We have the regression model for the
potential outcome model, where Y0∗ = α + Ui , and Y1∗ = α + δi + Ui . Randomization ensures that the
zero conditional mean assumption E(Ui |Di ) = 0 holds.
ATE = E(Yi∗ |D = 1) − E(Yi∗ |D = 0)
= α + δ + E(Ui |D = 1) − α − E(Ui |D = 0)
=δ
1.4 Linear regression model
Individual characteristics Xi can be included in the regression equation.
Yi = α + δDi + βXi + Ui
• To provide consistent estimators δ and β, we need the zero conditional mean assumption E(Ui |Di , Xi ) =
0
• To estimate δ consistently, we only need E(Ui |Di , Xi ) = E(Ui |Xi ). This holds with random assign-
ment or conditional random assignment.
• Do not include intermediate outcomes or other confounding variables in Xi .
• If the estimated treatment effect differs with covariates, ’worry’ about the additional covariates of
the random assignment.
• The advantage of including controls can increase precision of the estimated treatment effects.
• Conditional random assignment. Treatment assignment depends on (some of) the individual char-
acteristics Xi .
• Intermediate outcome. Other factors that have an effect on the outcome during the treatment.
, page 4 of 34
