Samenvatting Econometrics
H1: Introduction
1. Why econometrics?
• Many decisions in economics, business & government rely on understanding
relationships between variables
à These decisions require quantitative answers to quantitative questions
2. What is econometrics?
• Economic theory provides only a quantitative answer
• Econometrics quantifies these relationships using observed data
à Econometrics = measurement in economics
à Econometrics = the application of statistical and mathematical methods to
the analysis of economic data, with the aim of giving empirical content to
economic theories verifying or refuting them
• We are interested in measuring causal relationships: an action is said to
cause a result if the result is a direct consequence of that action
à Correlation ≠ causality: our ideas of causation must come from outside
statistics (economic theory)
à Different methods to uncover these causal relationships:
- Randomized Controlled Trials (RCT): using a treatment group and a
control group and then compare the results
à Often not feasible (ethical concerns, high costs, time constraints)
- Regression analysis
- Instrumental variables
3. Roadmap econometric analysis
• Econometrics: focuses on estimating causal effects
ó data science: aims to extract useful insights from data
4. Types of data
• Cross-sectional data: yi with i = 1,…, N
à Observations on a variable for different individuals at a single point in time
• Time series data: yt with t = 1,…, T
à Observations on a variable for an individual over multiple time periods
• Panel data: yit with i = 1,…, N and t = 1,…, T
à Observations for multiple individuals over multiple time periods
5. Objectives
The goal is to:
• Formulate an econometric specification (model / hypotheses)
• Estimate the proposed econometric model using statistical techniques
• Assess the statistical properties of the estimator (propose corrections)
• Interpret the economic implications (test the proposed hypotheses)
Jelle De Pelsmacker Econometrics 1
,H2: Basic concepts
1. Purpose of regression analysis
= modelling and estimating the relationship between:
- dependent variable (= outcome = response)
- independent variables (= explanatory variables = predictors = covariates)
à estimated model is useful for prediction & inferring causal relationships
• Conditional mean E(Y | Xi) of dependent variable is assumed to follow a
functional form
2. The Population Regression Function (PRF)
• Population regression curve: represents locus of conditional
expectations of the dependent variable Y given fixed values of
the independent variable X
• Linearity has 2 different meanings:
- Linear in the variables: linear function of Xi
à ex.: NOT E(Y|Xi) = b1 + b2Xi2
- Linear in the parameters: linear function of b
à ex.: NOT E(Y|Xi) = b1 + b22 Xi
à Always assumed in this course
• Stochastic error term: represents deviations
à 𝜇! = 𝑌! − 𝐸(𝑌|𝑋! ) with: 𝐸(𝑌|𝑋! ): systemic/deterministic component
è 𝑌! = 𝐸(𝑌|𝑋! ) + 𝜇! = 𝛽" + 𝛽# 𝑋! + 𝜇! 𝜇! : non-systemic/stochastic component
à The error term represents the collection of all variables & factors that
influence Yi, but are not included in the model:
à Possible explanations:
- Vague theory: model is based on an incomplete theoretical framework
- Lack of data: some relevant variables can’t be included (not available)
- Model simplicity: certain variables have only a marginal influence
- Measurement errors: imperfect/noisy data collection
- Misspecified functional form: true relationship is not correctly captured
- Human behaviour: decisions contain an inherent stochastic element
3. The Sample Regression Function (SRF)
• We typically observe a (randomly drawn) sample with size n from the
population instead of the whole population
à Goal: to reconstruct (= estimate) the population regression function based
on the available sample
• Sample Regression Function (SRF): 𝑌-! = 𝛽." + 𝛽.# 𝑋!
à With 𝑌-! an estimate of 𝐸(𝑌|𝑋! ) and 𝛽. an estimate of 𝛽
à Is only an approximation of the population regression function
à An estimator is stochastic: it varies across different samples
• Purpose of regression analysis: to estimate the parameters of the PRF using
the SRF and ensure that the estimates approximate the parameters as
closely as possible
à Use an estimator (this course: least squares method)
Jelle De Pelsmacker Econometrics 2
,H3: Estimating the sample regression function
1. The Ordinary Least Squares (OLS) method
• Goal: to find a sample regression line that minimizes the distance between
predicted values and the observed data point
• OLS: min ∑ 𝜇̂ !#
%! ,$
$ %"
à 2 first order conditions:
1 ' 1 '
6 𝜇̂ ! = 0 6 𝑋! 𝜇̂ ! = 0
𝑛 !(" 𝑛 !("
à formulas for 𝛽.# and 𝛽."
∑(𝑋! − 𝑋8)(𝑌! − 𝑌8) ∑ 𝑥! 𝑦! 𝑐𝑜𝑣(𝑋! 𝑌! ) ∑ 𝑋!# ∑ 𝑌! − ∑ 𝑋! ∑ 𝑋! 𝑌!
𝛽.# = = = 𝛽." = = 𝑌8 − 𝛽.# 𝑋8
∑(𝑋! − 𝑋8)# 𝑥!# 𝑣𝑎𝑟(𝑋! ) 𝑛 ∑ 𝑋!# − (∑ 𝑋! )#
• Numerical properties of the OLS estimator:
- The sample regression line passes through the sample means of Y and X
- The mean of 𝑌-! = mean of 𝑌! è 𝑌8- = 𝑌8
- Estimated error terms 𝜇̂ ! are zero on average
- Estimated error terms 𝜇̂ ! are not correlated with 𝑋!
@)
- Estimated error terms 𝜇̂ ! are not correlated with 𝑌
• Evaluation of OLS estimator: is based on analyzing its behavior over repeated
samples
• The precision is measured by the standard error (se) of 𝛽." and 𝛽.#
à BUT: these standard errors must be estimated: 𝑠𝑒(𝛽." ) è 𝑠𝑒C(𝛽." )
2. The Gauss-Markov assumptions
1) Linearity in the parameters: 𝑌! = 𝛽" + 𝛽# 𝑋! + 𝜇!
2) Fixed regressor model: 𝑋! is a deterministic variable (X-values are fixed)
3) The expected value of the error terms 𝜇! is zero: 𝐸(𝜇! |𝑋! ) = 0
à = variables not included in the model do not systematically affect Y
à 𝑋! and 𝜇! are not correlated
4) Homoskedasticity: 𝑣𝑎𝑟(𝜇! |𝑋! ) = 𝐸(𝜇! − 𝐸(𝜇! )|𝑋! )# = 𝜎 #
à = variance of the error terms 𝜇! are constant
à If violated: heteroskedasticity
5) No autocorrelation: 𝑐𝑜𝑣E𝜇! , 𝜇* G𝑋! , 𝑋* H = 0
à = no correlation in the error terms (no systematic pattern)
à If violated: autocorrelation
6) Number of observations (n) is larger than number of parameters to be
estimated
7) Non-zero variation in the X-values
8) No perfect multicollinearity
• When all Gauss-Markov assumptions hold
è Classical Linear Regression Model (CLRM)
Jelle De Pelsmacker Econometrics 3
, 3. Finite sample statistical properties of the OLS estimator
• OLS estimator is the Best Linear Unbiased Estimator (BLUE)
- Linear: it is a linear function of the stochastic variable 𝑌!
- Unbiased: the expected value equals true population variable: 𝐸E𝛽. H = 𝛽
- Efficient (best): it has the smallest variance withing the class of unbiased
estimators
4. Precision of the OLS estimator
• Variance formulas for the OLS estimator:
𝜎# ∑ 𝑋!# #
𝜎$%# = 𝑣𝑎𝑟E𝛽.# H = 𝜎$%# = 𝑣𝑎𝑟E𝛽." H = 𝜎
" ∑ 𝑥!# ! 𝑛 ∑ 𝑥!#
-"
∑,
• But: 𝜎 # is usually unknown è we have to estimate it: 𝜎I # = '.##
à 𝜎I # is unbiased: 𝐸(𝜎I # ) = 𝜎 #
• Estimator of the variance:
𝜎I # ∑ 𝑋!# #
𝜎I$%# = 𝜎
I%
#
= 𝜎I
" ∑ 𝑥!# $! 𝑛 ∑ 𝑥!#
à They are stochastic: they vary across repeated samples
à Determinants of the variance of 𝛽.# :
- Increases with 𝜎 # (larger variance è more noise è lower precision)
- Decreases with variance of X (more spread in X è more information è
higher precision)
- Decreases with sample size n (larger samples è more information è
higher precision)
5. Consistency of the OLS estimator
• Definition: an estimator is consistent if it converges in probability to the true
parameter as n approaches infinity
à 2 criteria:
- Unbiasedness: 𝐸E𝛽.# H = 𝛽#
- Variance shrinkage: 𝑣𝑎𝑟E𝛽.# H → 0 𝑎𝑠 𝑛 → ∞
• Rate of convergence: √𝑛E𝛽.# − 𝛽# H = 𝑂/ (1)
à = the distribution of 𝛽.# shrinks around 𝛽# at a rate proportional to 1/√𝑛
6. The coefficient of determination (R2)
• R2 measures the proportion of total variation in Yi that is explained by Xi
à if R2 = 1: model perfectly fits the data
à if R2 = 0: model explains nothing (X has no relationship with Y)
à higher R2 suggests a better fit (not necessarily causation!)
• 𝑌! = 𝑌-! + 𝜇̂ ! ó ∑(𝑌! − 𝑌8)# = ∑(𝑌-! − 𝑌8)# + ∑ 𝜇̂ !# ó ∑ 𝑦!# = ∑ 𝑦I!# + ∑ 𝜇̂ !# ó T = E + R
à 3 key components:
- Total Sum of Squares (TSS): ∑ 𝑦!# : total variation in Y
- Explained Sum of Squares (ESS): ∑ 𝑦I!# : variation explained by X
- Residual Sum of Squares (RSS): ∑ 𝜇̂ !# : unexplained variation (errors)
01/23!'45 738!39!:' 0<< =<<
• Formula for R2: 𝑅# = ;:932 738!39!:' = ;<< = 1 − ;<< (0 ≤ R2 ≤ 1)
Jelle De Pelsmacker Econometrics 4
H1: Introduction
1. Why econometrics?
• Many decisions in economics, business & government rely on understanding
relationships between variables
à These decisions require quantitative answers to quantitative questions
2. What is econometrics?
• Economic theory provides only a quantitative answer
• Econometrics quantifies these relationships using observed data
à Econometrics = measurement in economics
à Econometrics = the application of statistical and mathematical methods to
the analysis of economic data, with the aim of giving empirical content to
economic theories verifying or refuting them
• We are interested in measuring causal relationships: an action is said to
cause a result if the result is a direct consequence of that action
à Correlation ≠ causality: our ideas of causation must come from outside
statistics (economic theory)
à Different methods to uncover these causal relationships:
- Randomized Controlled Trials (RCT): using a treatment group and a
control group and then compare the results
à Often not feasible (ethical concerns, high costs, time constraints)
- Regression analysis
- Instrumental variables
3. Roadmap econometric analysis
• Econometrics: focuses on estimating causal effects
ó data science: aims to extract useful insights from data
4. Types of data
• Cross-sectional data: yi with i = 1,…, N
à Observations on a variable for different individuals at a single point in time
• Time series data: yt with t = 1,…, T
à Observations on a variable for an individual over multiple time periods
• Panel data: yit with i = 1,…, N and t = 1,…, T
à Observations for multiple individuals over multiple time periods
5. Objectives
The goal is to:
• Formulate an econometric specification (model / hypotheses)
• Estimate the proposed econometric model using statistical techniques
• Assess the statistical properties of the estimator (propose corrections)
• Interpret the economic implications (test the proposed hypotheses)
Jelle De Pelsmacker Econometrics 1
,H2: Basic concepts
1. Purpose of regression analysis
= modelling and estimating the relationship between:
- dependent variable (= outcome = response)
- independent variables (= explanatory variables = predictors = covariates)
à estimated model is useful for prediction & inferring causal relationships
• Conditional mean E(Y | Xi) of dependent variable is assumed to follow a
functional form
2. The Population Regression Function (PRF)
• Population regression curve: represents locus of conditional
expectations of the dependent variable Y given fixed values of
the independent variable X
• Linearity has 2 different meanings:
- Linear in the variables: linear function of Xi
à ex.: NOT E(Y|Xi) = b1 + b2Xi2
- Linear in the parameters: linear function of b
à ex.: NOT E(Y|Xi) = b1 + b22 Xi
à Always assumed in this course
• Stochastic error term: represents deviations
à 𝜇! = 𝑌! − 𝐸(𝑌|𝑋! ) with: 𝐸(𝑌|𝑋! ): systemic/deterministic component
è 𝑌! = 𝐸(𝑌|𝑋! ) + 𝜇! = 𝛽" + 𝛽# 𝑋! + 𝜇! 𝜇! : non-systemic/stochastic component
à The error term represents the collection of all variables & factors that
influence Yi, but are not included in the model:
à Possible explanations:
- Vague theory: model is based on an incomplete theoretical framework
- Lack of data: some relevant variables can’t be included (not available)
- Model simplicity: certain variables have only a marginal influence
- Measurement errors: imperfect/noisy data collection
- Misspecified functional form: true relationship is not correctly captured
- Human behaviour: decisions contain an inherent stochastic element
3. The Sample Regression Function (SRF)
• We typically observe a (randomly drawn) sample with size n from the
population instead of the whole population
à Goal: to reconstruct (= estimate) the population regression function based
on the available sample
• Sample Regression Function (SRF): 𝑌-! = 𝛽." + 𝛽.# 𝑋!
à With 𝑌-! an estimate of 𝐸(𝑌|𝑋! ) and 𝛽. an estimate of 𝛽
à Is only an approximation of the population regression function
à An estimator is stochastic: it varies across different samples
• Purpose of regression analysis: to estimate the parameters of the PRF using
the SRF and ensure that the estimates approximate the parameters as
closely as possible
à Use an estimator (this course: least squares method)
Jelle De Pelsmacker Econometrics 2
,H3: Estimating the sample regression function
1. The Ordinary Least Squares (OLS) method
• Goal: to find a sample regression line that minimizes the distance between
predicted values and the observed data point
• OLS: min ∑ 𝜇̂ !#
%! ,$
$ %"
à 2 first order conditions:
1 ' 1 '
6 𝜇̂ ! = 0 6 𝑋! 𝜇̂ ! = 0
𝑛 !(" 𝑛 !("
à formulas for 𝛽.# and 𝛽."
∑(𝑋! − 𝑋8)(𝑌! − 𝑌8) ∑ 𝑥! 𝑦! 𝑐𝑜𝑣(𝑋! 𝑌! ) ∑ 𝑋!# ∑ 𝑌! − ∑ 𝑋! ∑ 𝑋! 𝑌!
𝛽.# = = = 𝛽." = = 𝑌8 − 𝛽.# 𝑋8
∑(𝑋! − 𝑋8)# 𝑥!# 𝑣𝑎𝑟(𝑋! ) 𝑛 ∑ 𝑋!# − (∑ 𝑋! )#
• Numerical properties of the OLS estimator:
- The sample regression line passes through the sample means of Y and X
- The mean of 𝑌-! = mean of 𝑌! è 𝑌8- = 𝑌8
- Estimated error terms 𝜇̂ ! are zero on average
- Estimated error terms 𝜇̂ ! are not correlated with 𝑋!
@)
- Estimated error terms 𝜇̂ ! are not correlated with 𝑌
• Evaluation of OLS estimator: is based on analyzing its behavior over repeated
samples
• The precision is measured by the standard error (se) of 𝛽." and 𝛽.#
à BUT: these standard errors must be estimated: 𝑠𝑒(𝛽." ) è 𝑠𝑒C(𝛽." )
2. The Gauss-Markov assumptions
1) Linearity in the parameters: 𝑌! = 𝛽" + 𝛽# 𝑋! + 𝜇!
2) Fixed regressor model: 𝑋! is a deterministic variable (X-values are fixed)
3) The expected value of the error terms 𝜇! is zero: 𝐸(𝜇! |𝑋! ) = 0
à = variables not included in the model do not systematically affect Y
à 𝑋! and 𝜇! are not correlated
4) Homoskedasticity: 𝑣𝑎𝑟(𝜇! |𝑋! ) = 𝐸(𝜇! − 𝐸(𝜇! )|𝑋! )# = 𝜎 #
à = variance of the error terms 𝜇! are constant
à If violated: heteroskedasticity
5) No autocorrelation: 𝑐𝑜𝑣E𝜇! , 𝜇* G𝑋! , 𝑋* H = 0
à = no correlation in the error terms (no systematic pattern)
à If violated: autocorrelation
6) Number of observations (n) is larger than number of parameters to be
estimated
7) Non-zero variation in the X-values
8) No perfect multicollinearity
• When all Gauss-Markov assumptions hold
è Classical Linear Regression Model (CLRM)
Jelle De Pelsmacker Econometrics 3
, 3. Finite sample statistical properties of the OLS estimator
• OLS estimator is the Best Linear Unbiased Estimator (BLUE)
- Linear: it is a linear function of the stochastic variable 𝑌!
- Unbiased: the expected value equals true population variable: 𝐸E𝛽. H = 𝛽
- Efficient (best): it has the smallest variance withing the class of unbiased
estimators
4. Precision of the OLS estimator
• Variance formulas for the OLS estimator:
𝜎# ∑ 𝑋!# #
𝜎$%# = 𝑣𝑎𝑟E𝛽.# H = 𝜎$%# = 𝑣𝑎𝑟E𝛽." H = 𝜎
" ∑ 𝑥!# ! 𝑛 ∑ 𝑥!#
-"
∑,
• But: 𝜎 # is usually unknown è we have to estimate it: 𝜎I # = '.##
à 𝜎I # is unbiased: 𝐸(𝜎I # ) = 𝜎 #
• Estimator of the variance:
𝜎I # ∑ 𝑋!# #
𝜎I$%# = 𝜎
I%
#
= 𝜎I
" ∑ 𝑥!# $! 𝑛 ∑ 𝑥!#
à They are stochastic: they vary across repeated samples
à Determinants of the variance of 𝛽.# :
- Increases with 𝜎 # (larger variance è more noise è lower precision)
- Decreases with variance of X (more spread in X è more information è
higher precision)
- Decreases with sample size n (larger samples è more information è
higher precision)
5. Consistency of the OLS estimator
• Definition: an estimator is consistent if it converges in probability to the true
parameter as n approaches infinity
à 2 criteria:
- Unbiasedness: 𝐸E𝛽.# H = 𝛽#
- Variance shrinkage: 𝑣𝑎𝑟E𝛽.# H → 0 𝑎𝑠 𝑛 → ∞
• Rate of convergence: √𝑛E𝛽.# − 𝛽# H = 𝑂/ (1)
à = the distribution of 𝛽.# shrinks around 𝛽# at a rate proportional to 1/√𝑛
6. The coefficient of determination (R2)
• R2 measures the proportion of total variation in Yi that is explained by Xi
à if R2 = 1: model perfectly fits the data
à if R2 = 0: model explains nothing (X has no relationship with Y)
à higher R2 suggests a better fit (not necessarily causation!)
• 𝑌! = 𝑌-! + 𝜇̂ ! ó ∑(𝑌! − 𝑌8)# = ∑(𝑌-! − 𝑌8)# + ∑ 𝜇̂ !# ó ∑ 𝑦!# = ∑ 𝑦I!# + ∑ 𝜇̂ !# ó T = E + R
à 3 key components:
- Total Sum of Squares (TSS): ∑ 𝑦!# : total variation in Y
- Explained Sum of Squares (ESS): ∑ 𝑦I!# : variation explained by X
- Residual Sum of Squares (RSS): ∑ 𝜇̂ !# : unexplained variation (errors)
01/23!'45 738!39!:' 0<< =<<
• Formula for R2: 𝑅# = ;:932 738!39!:' = ;<< = 1 − ;<< (0 ≤ R2 ≤ 1)
Jelle De Pelsmacker Econometrics 4
