Econometrics III Overview CHoogteijling
Econometrics III Overview
Contents
1 Multiple equations in regression 3
1.1 Multiple equations in regressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Seemingly unrelated regression equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Generalized least squares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Seemingly Unrelated Regressions, same X . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.5 Feasible generalized least squares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.6 Pooled regressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2 Panel Data Models 6
2.1 Panel Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2 Feasible GLS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3 Between estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.4 Within estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.5 Fixed Effects model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.6 Hausman test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.7 Group-means . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.8 Two-Way Fixed Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.9 Difference-in-Difference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3 Vector Autoregressive Models 13
3.1 VAR(1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.1.1 Stationarity and stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.1.2 Dynamic properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.1.3 Mean, variance and autocovariance . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.2 VAR(1) Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.2.1 One-step ahead forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.2.2 Multi-step ahead forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.2.3 Granger causality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.2.4 Impulse response functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.2.5 Orthogonal IRF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.3 Parameter estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.3.1 Least squares method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.3.2 Maximum likelihood method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.3.3 Hypothesis testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.3.4 Testing for Granger-causality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.3.5 Bootstrap method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.4 VAR(p) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.4.1 Mean, variance and autocovariance . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
, page 1 of 33
,Econometrics III Overview CHoogteijling
3.4.2 Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.4.3 Impulse response function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.4.4 Maximum likelihood estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.5 Determine order in VAR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.5.1 Likelihood-ratio test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.5.2 Akaike information criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4 Non-stationary, Cointegration and Vector Error Correction Models 26
4.1 Cointegration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.2 Vector Error Correction Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.3 General VAR to VECM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.3.1 VAR representation of VECM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
4.4 Johansen Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
4.4.1 Trace test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.4.2 Maximum eigenvalue test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.4.3 Johansen Estimation Method for VECM . . . . . . . . . . . . . . . . . . . . . . . . 29
5 Prior knowledge 30
5.1 Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
5.2 Ordinary Least Squares (OLS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
5.3 Gauss-Markov assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
5.4 vec operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
5.5 Kronecker product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5.6 Coefficient of determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5.7 Autoregressive model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5.8 Eigenvalue analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5.9 Cointegration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
, page 2 of 33
,Econometrics III Overview CHoogteijling
1 Multiple equations in regression
1.1 Multiple equations in regressions
We can compose a multiple set of regression equations. Where i = 1, . . . , N , with N as the number of
different variables.
yi = Xi βi + εi
If the variables can be related to each other, we may wish to analyze the multiple set of regression
equations simultaneously, as if they are a part of a system of equations. Otherwise, we can analyze each
regression equation separately and apply OLS.
1.2 Seemingly unrelated regression equations
We have the multiple set of regression equations yi = Xi βi + εi , where the variables i can be related to
each other. The vector εi contains n noise terms.
The noise vectors may show dependence across different equations. When we want to allow for this
dependence, we use a Seemingly Unrelated Regression (SUR).
Here we assume the noise terms to be normally, independent and identically distributed (NID). Where
In is the n × n identity fmatrix and σi2 is the variance (or scale). The variance can be different for a
different equation i. Hence, the Gauss-Markov assumptions apply to each equation i.
εi ∼ NID(0, σi2 In )
We can express all equations in one system, or y = Xβ + ε, where ε ∼ N (0, Σ). The N n × N n matrix Σ
is the variance-covariance matrix for ε.
y1 X1 0 ... 0 β1 ε1
y2 0 X2 ... 0 β2 ε 2
.. = .. . + .
.. ..
. . . . 0 .. ..
yN 0 0 . . . XN βN εN
′ ′ ′
E(ε1 ε1 ) E(ε1 ε2 ) . . . E(ε1 εN )
E(ε2 ε′1 ) E(ε2 ε′2 ) . . . E(ε2 ε′N )
′
Σ = E(εε ) =
.. .. .. ..
. . . .
′ ′ ′
E(εN ε1 ) E(εN ε2 ) . . . E(εN εN )
We can assume across equations that E(εi ε′j ) = σij In , for i, j = 1, . . . , N with i ̸= j. This means that we
assume nonzero correlation between corresponding disturbances in the ith and jth equations and zero
correlation between non-corresponding disturbances.
We can also assume homoskedastic disturbances for each equation, then E(ε1 ε′i ) = σi2 In , for i = 1, . . . , N .
1.3 Generalized least squares
Given the SUR y = Xβ + ε, and ε ∼ NID(0, Σ)), where Σ = Ω ⊗ IN is clearly non-diagonal, we should
apply Generalized Least Squares (GLS). The GLS estimator has optimal properties, certainly under
Gauss-Markov (or similar) assumptions.
β̂GLS = (X ′ Σ−1 X)−1 X ′ Σ−1 y
, page 3 of 33
, Econometrics III Overview CHoogteijling
1.4 Seemingly Unrelated Regressions, same X
We consider the system of equations, but now with the same set of explanatory variables Xi = X, for all
equations i = 1, . . . , N .
yi = Xβi + εi
The SUR with same X is defined as follows, where ε ∼ NID(0, Σ), and Σ = Ω ⊗ In .
y1 X 0 ... 0 β1 ε1
y2 0 X . . . 0 β2 ε2
.. = .. . + .
.. . .
. . . . 0 .. ..
yN 0 0 ... X βN εN
β1
..
y = [IN ⊗ X] . + ε
βN
y = [IN ⊗ X]β + ε
Then the GLS estimate is given by the following equation and by applying some Kronecker properties
we simplify the equation.
β̂GLS = ([IN ⊗ X]′ Σ−1 [IN ⊗ X]−1 [IN ⊗ X]′ Σ−1 y
[IN ⊗ X]′ Σ−1 [IN ⊗ X] = [IN ⊗ X]′ [Ω ⊗ In ][IN ⊗ X]
′
= [IN ⊗ X ][Ω−1 ⊗ In ][IN ⊗ X]
′
= Ω−1 ⊗ X X
′ ′
β̂GLS = (Ω−1 ⊗ X X)−1 [Ω−1 ⊗ X ]y
′ ′
= (Ω ⊗ (X X)−1 )[Ω−1 ⊗ X ]y
′ ′
= (IN ⊗ (X X)−1 X y
This implies that for a SUR with same Xs, you can obtain the GLS estimate by applying OLS to each
equation. No joint estimation is needed.
1.5 Feasible generalized least squares
Given the least squares estimate β̂LS , we compute the least squares residuals ei . The estimates for the
elements in the variance-covariance matrix Ω are σ̂i2 and σ̂ij , for i, j = 1, . . . , N , with i ̸= j, leading to
estimate Ω̂.
ei = yi − Xi β̂i,LS
e = y − X β̂LS
e′i ei
σ̂i2 =
n−K
e′ ej
σ̂ij = i
n−K
Feasible GLS (FGLS) is the case of GLS where Ω is unknown. In the unfeasible case Ω is assumed to
be known.
, page 4 of 33
Econometrics III Overview
Contents
1 Multiple equations in regression 3
1.1 Multiple equations in regressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Seemingly unrelated regression equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Generalized least squares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Seemingly Unrelated Regressions, same X . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.5 Feasible generalized least squares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.6 Pooled regressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2 Panel Data Models 6
2.1 Panel Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2 Feasible GLS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3 Between estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.4 Within estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.5 Fixed Effects model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.6 Hausman test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.7 Group-means . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.8 Two-Way Fixed Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.9 Difference-in-Difference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3 Vector Autoregressive Models 13
3.1 VAR(1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.1.1 Stationarity and stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.1.2 Dynamic properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.1.3 Mean, variance and autocovariance . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.2 VAR(1) Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.2.1 One-step ahead forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.2.2 Multi-step ahead forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.2.3 Granger causality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.2.4 Impulse response functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.2.5 Orthogonal IRF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.3 Parameter estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.3.1 Least squares method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.3.2 Maximum likelihood method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.3.3 Hypothesis testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.3.4 Testing for Granger-causality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.3.5 Bootstrap method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.4 VAR(p) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.4.1 Mean, variance and autocovariance . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
, page 1 of 33
,Econometrics III Overview CHoogteijling
3.4.2 Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.4.3 Impulse response function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.4.4 Maximum likelihood estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.5 Determine order in VAR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.5.1 Likelihood-ratio test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.5.2 Akaike information criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4 Non-stationary, Cointegration and Vector Error Correction Models 26
4.1 Cointegration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.2 Vector Error Correction Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.3 General VAR to VECM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.3.1 VAR representation of VECM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
4.4 Johansen Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
4.4.1 Trace test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.4.2 Maximum eigenvalue test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.4.3 Johansen Estimation Method for VECM . . . . . . . . . . . . . . . . . . . . . . . . 29
5 Prior knowledge 30
5.1 Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
5.2 Ordinary Least Squares (OLS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
5.3 Gauss-Markov assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
5.4 vec operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
5.5 Kronecker product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5.6 Coefficient of determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5.7 Autoregressive model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5.8 Eigenvalue analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5.9 Cointegration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
, page 2 of 33
,Econometrics III Overview CHoogteijling
1 Multiple equations in regression
1.1 Multiple equations in regressions
We can compose a multiple set of regression equations. Where i = 1, . . . , N , with N as the number of
different variables.
yi = Xi βi + εi
If the variables can be related to each other, we may wish to analyze the multiple set of regression
equations simultaneously, as if they are a part of a system of equations. Otherwise, we can analyze each
regression equation separately and apply OLS.
1.2 Seemingly unrelated regression equations
We have the multiple set of regression equations yi = Xi βi + εi , where the variables i can be related to
each other. The vector εi contains n noise terms.
The noise vectors may show dependence across different equations. When we want to allow for this
dependence, we use a Seemingly Unrelated Regression (SUR).
Here we assume the noise terms to be normally, independent and identically distributed (NID). Where
In is the n × n identity fmatrix and σi2 is the variance (or scale). The variance can be different for a
different equation i. Hence, the Gauss-Markov assumptions apply to each equation i.
εi ∼ NID(0, σi2 In )
We can express all equations in one system, or y = Xβ + ε, where ε ∼ N (0, Σ). The N n × N n matrix Σ
is the variance-covariance matrix for ε.
y1 X1 0 ... 0 β1 ε1
y2 0 X2 ... 0 β2 ε 2
.. = .. . + .
.. ..
. . . . 0 .. ..
yN 0 0 . . . XN βN εN
′ ′ ′
E(ε1 ε1 ) E(ε1 ε2 ) . . . E(ε1 εN )
E(ε2 ε′1 ) E(ε2 ε′2 ) . . . E(ε2 ε′N )
′
Σ = E(εε ) =
.. .. .. ..
. . . .
′ ′ ′
E(εN ε1 ) E(εN ε2 ) . . . E(εN εN )
We can assume across equations that E(εi ε′j ) = σij In , for i, j = 1, . . . , N with i ̸= j. This means that we
assume nonzero correlation between corresponding disturbances in the ith and jth equations and zero
correlation between non-corresponding disturbances.
We can also assume homoskedastic disturbances for each equation, then E(ε1 ε′i ) = σi2 In , for i = 1, . . . , N .
1.3 Generalized least squares
Given the SUR y = Xβ + ε, and ε ∼ NID(0, Σ)), where Σ = Ω ⊗ IN is clearly non-diagonal, we should
apply Generalized Least Squares (GLS). The GLS estimator has optimal properties, certainly under
Gauss-Markov (or similar) assumptions.
β̂GLS = (X ′ Σ−1 X)−1 X ′ Σ−1 y
, page 3 of 33
, Econometrics III Overview CHoogteijling
1.4 Seemingly Unrelated Regressions, same X
We consider the system of equations, but now with the same set of explanatory variables Xi = X, for all
equations i = 1, . . . , N .
yi = Xβi + εi
The SUR with same X is defined as follows, where ε ∼ NID(0, Σ), and Σ = Ω ⊗ In .
y1 X 0 ... 0 β1 ε1
y2 0 X . . . 0 β2 ε2
.. = .. . + .
.. . .
. . . . 0 .. ..
yN 0 0 ... X βN εN
β1
..
y = [IN ⊗ X] . + ε
βN
y = [IN ⊗ X]β + ε
Then the GLS estimate is given by the following equation and by applying some Kronecker properties
we simplify the equation.
β̂GLS = ([IN ⊗ X]′ Σ−1 [IN ⊗ X]−1 [IN ⊗ X]′ Σ−1 y
[IN ⊗ X]′ Σ−1 [IN ⊗ X] = [IN ⊗ X]′ [Ω ⊗ In ][IN ⊗ X]
′
= [IN ⊗ X ][Ω−1 ⊗ In ][IN ⊗ X]
′
= Ω−1 ⊗ X X
′ ′
β̂GLS = (Ω−1 ⊗ X X)−1 [Ω−1 ⊗ X ]y
′ ′
= (Ω ⊗ (X X)−1 )[Ω−1 ⊗ X ]y
′ ′
= (IN ⊗ (X X)−1 X y
This implies that for a SUR with same Xs, you can obtain the GLS estimate by applying OLS to each
equation. No joint estimation is needed.
1.5 Feasible generalized least squares
Given the least squares estimate β̂LS , we compute the least squares residuals ei . The estimates for the
elements in the variance-covariance matrix Ω are σ̂i2 and σ̂ij , for i, j = 1, . . . , N , with i ̸= j, leading to
estimate Ω̂.
ei = yi − Xi β̂i,LS
e = y − X β̂LS
e′i ei
σ̂i2 =
n−K
e′ ej
σ̂ij = i
n−K
Feasible GLS (FGLS) is the case of GLS where Ω is unknown. In the unfeasible case Ω is assumed to
be known.
, page 4 of 33
