Week 1
Heteroskedasticity
In the form 𝑦 = 𝑋𝛽 + 𝜀 with 𝐸(𝜀𝜀 ! ) = Ω, where 𝜎"# is the 𝑖 $% diagonal element and all other
elements are 0.
Properties OLS estimator
The OLS under heteroskedasticity is still unbiased and consistent, but inefficient. In addition,
the usual OLS standard errors are incorrect, because
𝑣𝑎𝑟(𝑏&'( ) = (𝑋 ! 𝑋))* 𝑋 ! Ω𝑋(𝑋 ! 𝑋))*
White standard errors
From 𝑋 ! 𝑋 = ∑+",* 𝑥" 𝑥"! and 𝑋 ! Ω𝑋 = ∑+",* 𝜎"# 𝑥" 𝑥"! , and by using 𝑒"# as an estimator of the
unknown variance 𝜎"# , we get 𝑣𝑎𝑟 4 (𝑏&'( ) = (∑+",* 𝑥" 𝑥"! ))* (∑+",* 𝑒"# 𝑥" 𝑥"! )(∑+",* 𝑥" 𝑥"! ))* . This is
a consistent estimator of 𝑣𝑎𝑟(𝑏&'( )
Properties White standard errors
A common solution is to use OLS estimates with White standard errors. The advantage is that
heteroskedasticity does not need to be modelled, however, if you do know a model, OLS is
inefficient and not BLUE
Weighted least squares (WLS)
Suppose we know that 𝜎"# = 𝜎 # 𝑣" , with 𝑣" known. If we have 𝑦" = 𝑥"! 𝛽 + 𝜀" , 𝐸(𝜀"# ) = 𝜎 # 𝑣" ,
* * *
we can “standardize” this: . 𝑦" = . 𝑥"! 𝛽 + . 𝜀" ⟺ 𝑦"∗ = 𝑥"∗ ! 𝛽 + 𝜀"∗ . Then 𝐸6𝜀"∗ # 7 = 𝜎 # .
- ! - ! - !
Properties WLS
OLS on this equation gives a BLUE for 𝛽. Here, a larger variance gives a smaller weight. The
WLS estimator is 𝑏0'( = (𝑋∗! 𝑋∗ ))* 𝑋∗! 𝑦∗ , it is unbiased, its variance is
𝑣𝑎𝑟(𝑏0'( ) = 𝜎 # (𝑋∗! 𝑋∗ ))* and the 𝑏0'( is BLUE. In particular, WLS is more efficient than OLS.
Use the unweighted statistics
Weighted least squares in practice
The challenge with WLS in practice is that 𝑣" is unknown or unobserved. To apply WLS we
need an estimate of the variances. In 𝑦" = 𝑥"! 𝛽 + 𝜀" , with 𝜀" ~ 𝑁(0, 𝜎"# ), we need a model for
"
𝜎"# . Two important cases are 𝜎"# = 𝑧"! 𝛾 (linear form) and 𝜎"# = 𝑒 1! 2 (multiplicative form), with
𝑧" a vector of observables
Feasible WLS
1. Estimate the variance parameters. Apply OLS in 𝑦 = 𝑋𝛽 + 𝜀, then 𝑏&'( is consistent and
𝑒"# is an asymptotically unbiased estimate of 𝜎"# . Then run the regression 𝑒"# = 𝑧"! 𝛾 + 𝜂"
(or log 𝑒"# = 𝑧"! 𝛾 + 𝜂" )
2. Apply WLS with the estimated variances to estimate 𝛽. Use the estimated 𝛾 parameters
to do WLS (with e.g., 𝜎C"# = 𝑧"! 𝛾C)
Properties feasible WLS
The estimator 𝑏30'( is consistent, under the condition that 𝛾 is estimated consistently, and it
is asymptotically efficient (and equal to WLS). Estimator of 𝛾 is for the linear form always
consistent, and for the multiplicative form consistent when correction is included. The
correction is for 𝛾* , since it needs to be estimated as 𝛾C* + 1.27
Maximum likelihood
𝑓(𝑦; 𝜃) = ∏" 𝑓(𝑦" ; 𝜃), likelihood is 𝐿(𝜃) = 𝑓(𝑦; 𝜃). Find the value of 𝜃 that maximizes
)*
log 𝐿(𝜃) = ℓ(𝜃). Then 𝜃N4' ≈ 𝑁 P𝜃5 , 6𝐼R+ 7 S. ℓ(𝜃) = ∑+",* ℓ" (𝜃), where
* * *
ℓ" (𝜃) = − # log(2𝜋) − # log(𝜎 # ) − #6# (𝑦" − 𝑥"! 𝛽)# . Here 𝑥"! 𝛽 = 𝜇 and 𝜎 # = ℎ(𝑧"! 𝛾)
Tests for heteroskedasticity
To test whether there is heteroskedasticity, we test the null that there is homoskedasticity,
the more restricted form
Goldfeld-Quandt
, 1. Split the data in 3 (or 2) groups with 𝑛* , 𝑛# and 𝑛7 observations. The groups are based on
the suspicion of heteroskedasticity
2. Apply OLS in groups 𝑛* (obtain 𝑒* ) and in 𝑛# (obtain 𝑒# )
3. Compare estimated variances
8!" 8! 8!" 8! 8$" 8$
Under 𝐻5 of constant variance, we have 6#
~ 𝜒 # (𝑛" − 𝑘) and 6#
is independent of 6#
,𝑖 ≠
%"# %#
/(+# );) > #
&#
𝑗. So under 𝐻5 , "
%' %'
= P># S ~ 𝐹(𝑛# − 𝑘, 𝑛* − 𝑘). The largest expected variance should
/(+' );) '
&#
be in group 2
White and Breusch-Pagan LM tests
The White and Breusch-Pagan offer Lagrange Multiplier (LM) tests. The idea is to test whether
the gradient (score) is sufficiently close to zero at the restricted estimate. The steps are:
1. Estimate the restricted model: OLS on 𝑦" = 𝑥"! 𝛽 + 𝜀" , this gives 𝑒"
2. Auxiliary regression of residuals from step 1 on a specific set of regressors: 𝑒"# = 𝑧"! 𝛾 + 𝜂"
3. Calculate 𝐿𝑀 = 𝑛𝑅# ~ 𝜒 # (𝑝 − 1) using 𝑅# of the auxiliary regression from step 2
White test
White test is an LM test on 𝛾# = ⋯ = 𝛾? = 0 in 𝜎"# = ℎ(𝑧"! 𝛾) = ℎ6𝛾* + 𝛾# 𝑧#" + ⋯ + 𝛾? 𝑧?" 7.
We can have White with or without cross-terms
Breusch-Pagan test
The Breusch-Pagan test is very similar to the White test. The difference is that in the Breusch-
Pagan test some known variables are suspected of driving the variance. So now 𝑧" is a set of
selected/known variables. White is more pragmatic, in sense that simply use variables that
we have at hand anyway, with all kind of combinations of these
Likelihood Ratio test
In the LR test the significance of 𝛾 in 𝜎"# = ℎ(𝑧"! 𝛾) is directly examined. Here the function ℎ is
very important. The procedure is to estimate the model under 𝐻5 and 𝐻* and compare the
restricted and unrestricted log-likelihood values: 2(log 𝐿@ − log 𝐿A ) ~ 𝜒 # (𝑘) with 𝑘 the
number of restrictions
Summary
1. Apply a test for the possible presence of heteroskedasticity. If one has an idea what are
the possible causes of heteroskedasticity, it is helpful to formulate a corresponding model
for the variance of the error terms and to apply the Breusch–Pagan test. If one has no such
ideas, one can apply the White test or the Goldfeld–Quandt test
2. If tests indicate the presence of significant heteroskedasticity, then OLS should not be
routinely applied, as it is not efficient and the usual formulas for the standard errors (as
computed by software packages) do not apply. If one sees no possibility of formulating a
meaningful model for the heteroskedasticity, then one can apply GMM — that is, OLS with
White standard errors
3. If one can formulate a model for the heteroskedasticity (for instance, an additive or a
multiplicative model), then the model parameters can be estimated by weighted least
squares if the variances are known up to a scale factor. Otherwise one can use feasible
weighted least squares or maximum likelihood, with the usual approximate distributions
of the estimators
4. Let 𝑒" = 𝑦" − 𝑥"! 𝛽 be the 𝑖 $% residual and let 𝜎C"# be the estimated variance of the 𝑖 $%
disturbance. Then the model for heteroskedasticity may be evaluated by checking
whether the scaled residuals 𝑒" /𝜎C" are homoskedastic. If this is not the case, one can try
to improve the model for the variances, or otherwise apply OLS with White standard
errors
Heteroskedasticity
In the form 𝑦 = 𝑋𝛽 + 𝜀 with 𝐸(𝜀𝜀 ! ) = Ω, where 𝜎"# is the 𝑖 $% diagonal element and all other
elements are 0.
Properties OLS estimator
The OLS under heteroskedasticity is still unbiased and consistent, but inefficient. In addition,
the usual OLS standard errors are incorrect, because
𝑣𝑎𝑟(𝑏&'( ) = (𝑋 ! 𝑋))* 𝑋 ! Ω𝑋(𝑋 ! 𝑋))*
White standard errors
From 𝑋 ! 𝑋 = ∑+",* 𝑥" 𝑥"! and 𝑋 ! Ω𝑋 = ∑+",* 𝜎"# 𝑥" 𝑥"! , and by using 𝑒"# as an estimator of the
unknown variance 𝜎"# , we get 𝑣𝑎𝑟 4 (𝑏&'( ) = (∑+",* 𝑥" 𝑥"! ))* (∑+",* 𝑒"# 𝑥" 𝑥"! )(∑+",* 𝑥" 𝑥"! ))* . This is
a consistent estimator of 𝑣𝑎𝑟(𝑏&'( )
Properties White standard errors
A common solution is to use OLS estimates with White standard errors. The advantage is that
heteroskedasticity does not need to be modelled, however, if you do know a model, OLS is
inefficient and not BLUE
Weighted least squares (WLS)
Suppose we know that 𝜎"# = 𝜎 # 𝑣" , with 𝑣" known. If we have 𝑦" = 𝑥"! 𝛽 + 𝜀" , 𝐸(𝜀"# ) = 𝜎 # 𝑣" ,
* * *
we can “standardize” this: . 𝑦" = . 𝑥"! 𝛽 + . 𝜀" ⟺ 𝑦"∗ = 𝑥"∗ ! 𝛽 + 𝜀"∗ . Then 𝐸6𝜀"∗ # 7 = 𝜎 # .
- ! - ! - !
Properties WLS
OLS on this equation gives a BLUE for 𝛽. Here, a larger variance gives a smaller weight. The
WLS estimator is 𝑏0'( = (𝑋∗! 𝑋∗ ))* 𝑋∗! 𝑦∗ , it is unbiased, its variance is
𝑣𝑎𝑟(𝑏0'( ) = 𝜎 # (𝑋∗! 𝑋∗ ))* and the 𝑏0'( is BLUE. In particular, WLS is more efficient than OLS.
Use the unweighted statistics
Weighted least squares in practice
The challenge with WLS in practice is that 𝑣" is unknown or unobserved. To apply WLS we
need an estimate of the variances. In 𝑦" = 𝑥"! 𝛽 + 𝜀" , with 𝜀" ~ 𝑁(0, 𝜎"# ), we need a model for
"
𝜎"# . Two important cases are 𝜎"# = 𝑧"! 𝛾 (linear form) and 𝜎"# = 𝑒 1! 2 (multiplicative form), with
𝑧" a vector of observables
Feasible WLS
1. Estimate the variance parameters. Apply OLS in 𝑦 = 𝑋𝛽 + 𝜀, then 𝑏&'( is consistent and
𝑒"# is an asymptotically unbiased estimate of 𝜎"# . Then run the regression 𝑒"# = 𝑧"! 𝛾 + 𝜂"
(or log 𝑒"# = 𝑧"! 𝛾 + 𝜂" )
2. Apply WLS with the estimated variances to estimate 𝛽. Use the estimated 𝛾 parameters
to do WLS (with e.g., 𝜎C"# = 𝑧"! 𝛾C)
Properties feasible WLS
The estimator 𝑏30'( is consistent, under the condition that 𝛾 is estimated consistently, and it
is asymptotically efficient (and equal to WLS). Estimator of 𝛾 is for the linear form always
consistent, and for the multiplicative form consistent when correction is included. The
correction is for 𝛾* , since it needs to be estimated as 𝛾C* + 1.27
Maximum likelihood
𝑓(𝑦; 𝜃) = ∏" 𝑓(𝑦" ; 𝜃), likelihood is 𝐿(𝜃) = 𝑓(𝑦; 𝜃). Find the value of 𝜃 that maximizes
)*
log 𝐿(𝜃) = ℓ(𝜃). Then 𝜃N4' ≈ 𝑁 P𝜃5 , 6𝐼R+ 7 S. ℓ(𝜃) = ∑+",* ℓ" (𝜃), where
* * *
ℓ" (𝜃) = − # log(2𝜋) − # log(𝜎 # ) − #6# (𝑦" − 𝑥"! 𝛽)# . Here 𝑥"! 𝛽 = 𝜇 and 𝜎 # = ℎ(𝑧"! 𝛾)
Tests for heteroskedasticity
To test whether there is heteroskedasticity, we test the null that there is homoskedasticity,
the more restricted form
Goldfeld-Quandt
, 1. Split the data in 3 (or 2) groups with 𝑛* , 𝑛# and 𝑛7 observations. The groups are based on
the suspicion of heteroskedasticity
2. Apply OLS in groups 𝑛* (obtain 𝑒* ) and in 𝑛# (obtain 𝑒# )
3. Compare estimated variances
8!" 8! 8!" 8! 8$" 8$
Under 𝐻5 of constant variance, we have 6#
~ 𝜒 # (𝑛" − 𝑘) and 6#
is independent of 6#
,𝑖 ≠
%"# %#
/(+# );) > #
&#
𝑗. So under 𝐻5 , "
%' %'
= P># S ~ 𝐹(𝑛# − 𝑘, 𝑛* − 𝑘). The largest expected variance should
/(+' );) '
&#
be in group 2
White and Breusch-Pagan LM tests
The White and Breusch-Pagan offer Lagrange Multiplier (LM) tests. The idea is to test whether
the gradient (score) is sufficiently close to zero at the restricted estimate. The steps are:
1. Estimate the restricted model: OLS on 𝑦" = 𝑥"! 𝛽 + 𝜀" , this gives 𝑒"
2. Auxiliary regression of residuals from step 1 on a specific set of regressors: 𝑒"# = 𝑧"! 𝛾 + 𝜂"
3. Calculate 𝐿𝑀 = 𝑛𝑅# ~ 𝜒 # (𝑝 − 1) using 𝑅# of the auxiliary regression from step 2
White test
White test is an LM test on 𝛾# = ⋯ = 𝛾? = 0 in 𝜎"# = ℎ(𝑧"! 𝛾) = ℎ6𝛾* + 𝛾# 𝑧#" + ⋯ + 𝛾? 𝑧?" 7.
We can have White with or without cross-terms
Breusch-Pagan test
The Breusch-Pagan test is very similar to the White test. The difference is that in the Breusch-
Pagan test some known variables are suspected of driving the variance. So now 𝑧" is a set of
selected/known variables. White is more pragmatic, in sense that simply use variables that
we have at hand anyway, with all kind of combinations of these
Likelihood Ratio test
In the LR test the significance of 𝛾 in 𝜎"# = ℎ(𝑧"! 𝛾) is directly examined. Here the function ℎ is
very important. The procedure is to estimate the model under 𝐻5 and 𝐻* and compare the
restricted and unrestricted log-likelihood values: 2(log 𝐿@ − log 𝐿A ) ~ 𝜒 # (𝑘) with 𝑘 the
number of restrictions
Summary
1. Apply a test for the possible presence of heteroskedasticity. If one has an idea what are
the possible causes of heteroskedasticity, it is helpful to formulate a corresponding model
for the variance of the error terms and to apply the Breusch–Pagan test. If one has no such
ideas, one can apply the White test or the Goldfeld–Quandt test
2. If tests indicate the presence of significant heteroskedasticity, then OLS should not be
routinely applied, as it is not efficient and the usual formulas for the standard errors (as
computed by software packages) do not apply. If one sees no possibility of formulating a
meaningful model for the heteroskedasticity, then one can apply GMM — that is, OLS with
White standard errors
3. If one can formulate a model for the heteroskedasticity (for instance, an additive or a
multiplicative model), then the model parameters can be estimated by weighted least
squares if the variances are known up to a scale factor. Otherwise one can use feasible
weighted least squares or maximum likelihood, with the usual approximate distributions
of the estimators
4. Let 𝑒" = 𝑦" − 𝑥"! 𝛽 be the 𝑖 $% residual and let 𝜎C"# be the estimated variance of the 𝑖 $%
disturbance. Then the model for heteroskedasticity may be evaluated by checking
whether the scaled residuals 𝑒" /𝜎C" are homoskedastic. If this is not the case, one can try
to improve the model for the variances, or otherwise apply OLS with White standard
errors
