Stats 330: Advanced Statistical
Modelling Exam-Graded A
Technically correct data - ANSWER-Data is stored in a data frame with suitable column
names. Each column of the data frame is of adequate class
Consistent Data - ANSWER-Missing values, special values, unusual observations and
obvious errors have been dealt with in a suitable manner.
Dealing with missing values, special values and obvious errors - ANSWER-(1)
Removing (seldom)
(2) Imputing a reasonable alternative - the mean of the data, prediction, randomly from
a range, or the nearest neighbour
Potential problems with data (5) - ANSWER-(1) Data are non-planar
(2) There are outliers in the data
(3) There is non-constant scatter
(4) Errors are not independent
(5) Errors are not normally distributed
Residual analysis - ANSWER-Residual analysis can be used to check if a model is
appropriate. If the model is appropriate, the residuals will have no pattern.
Detecting non-planar data (also called non-linearity) - ANSWER-(1) A curved residuals
versus fitted values plot
(2) Curved residuals vs explanatory variables plot
(3) Pairs plot/ co plot should be parallel
Remedies for non-planar data: GAM plots (General Additive Models) - ANSWER-Use
GAM plots to decide which variables we need to transform and how. The GAM plots
show the estimated functions of the variables obtained by smoothing. These optimal
transformations are used to suggest simpler transformations of explanatory variables
(e.g. quadratic, splines). This also applies for logistic regression
Remedies for non-planar data: the ladder of powers - ANSWER-Use the Box-Cox plot
to suggest the optimal value by which the response variable should be transformed.
Transforming the response according to the Box-Cox plot may fix non-normality of
residuals, unequal variances in response, non-planarity of model and need to transform
covariates indicated by GAM plots.
,Fitting Splines - ANSWER-An alternative to polynomials are splines. They are piecewise
cubics which join smoothly at knots that give a more flexible fit to the data. Values at
one point are not affected by values at distant points, unlike polynomials.
Detecting non-constant scatter - ANSWER-Plot of residuals versus fitted values shows
curvature
The importance of non-constant scatter - ANSWER-A multiple regression model
specifies that the scatter about the regression plane is uniform. All tests and confidence
intervals rely on this.
Constant scatter - ANSWER-Scatter does not depend on the explanatory variables or
the mean of the response. Scatter is measured by the size of the residuals.
The funnel effect - ANSWER-The funnel effect in residuals versus fitted values plot
occurs when scatter increases as the mean response increases.
Remedies for unequal scatter - ANSWER-(1) Transform the response (ladder of
powers)
(2) Estimate the variances of the observations using weighted least squares
Weighted least squares - ANSWER-Observations need to have constant variance. If the
n observation has variance vnσ² then we can get a valid test by using weighted least
squares, minimising the sum of the weighted squared residuals.
Weighted squared residuals - ANSWER-∑(r²)/v
(sum of residuals squared divided by weighting)
Residual sum of squares - ANSWER-RSS=∑r²
Or the sum of residuals squared
Finding the required weights to minimise weighted squared residuals - ANSWER-(1)
Plot the squared residuals versus the fitted values
(2) Smooth the plot
(3) Estimate the variance of an observation by the smoothed squared residual
(4) Weight is reciprocal of smoothed squared residual
The issue of non-normality - ANSWER-Non-normality is not such a crucial assumption,
but it can be important if the errors have a long-tailed distribution since this will imply
there are several outliers. The normality assumption is important for prediction.
Detecting non-normality: Normal QQ plots - ANSWER-QQ plot
(1) Normal if along line
(2) Short tailed if low values are above line and high values are below, flat
(3) Long tailed if low values are below line and high values are above, steep
(4) Right skewed if smile shaped
, (5) Left skewed if frown shaped
Detecting non-normality: Weisberg-Bingham test - ANSWER-WB is the square of the
correlation of the normal plot, which measures how straight the plot is. It is between 0
and 1. Values close to 1 indicate normality. WB.test computes the p value for the test
statistic, with the null hypothesis being that the sample is normal (small p = not normal)
Remedies for non-normality - ANSWER-Transform the response using the power
transformation
Outlier - ANSWER-An outlier is a point that has a larger or smaller y value than the
model would suggest. It can be due to a genuine large error, ε, or to a typographical
error in recording the data.
High leverage point - ANSWER-A high leverage point is a point with extreme values of
the explanatory variable.
High leverage outlier - ANSWER-A high leverage outlier can attract the fitted plane,
distorting the fit. They will have large HMDs. In extreme cases, it may not have a big
residual, meaning we may not be able to tell if it is an outlier. In extreme cases it can
increase R².
Low leverage outlier - ANSWER-Does not distort the fit to the same extent. It usually
has a big residual, and a low HMD. It inflates the standard errors and decreases R².
Detecting high leverage points: Hat matrix diagonals - ANSWER-High leverage points
do not necessarily have large residuals, so it can be difficult to detect them in a residual
plot. They can be detected by calculating hat matrix diagonals, which measure the
amount of pull ith data point has on the fitted regression surface. An HMD larger than
3(k+1)/n is considered extreme.
hatvalues(whatever.lm)
Hat matrix diagonals - ANSWER-Fitted Y = H*Y. The ith diagonal element measures the
extent to which the yi depends on the fitted value.
Recognising large residuals: studentised residuals - ANSWER-We need to standardise
residuals by dividing them by their standard deviations. You can do this by using
internally (standardised) or externally (studentised) residuals. They should be between -
2 and 2 with 95% probability.
s - ANSWER-Estimate of residual variance, σ²
Studentised residuals - ANSWER-var(ei) = (1-hii)σ²
= ei/(sqrt(1-hii)s²)
where s is the usual estimate of residual variance
Modelling Exam-Graded A
Technically correct data - ANSWER-Data is stored in a data frame with suitable column
names. Each column of the data frame is of adequate class
Consistent Data - ANSWER-Missing values, special values, unusual observations and
obvious errors have been dealt with in a suitable manner.
Dealing with missing values, special values and obvious errors - ANSWER-(1)
Removing (seldom)
(2) Imputing a reasonable alternative - the mean of the data, prediction, randomly from
a range, or the nearest neighbour
Potential problems with data (5) - ANSWER-(1) Data are non-planar
(2) There are outliers in the data
(3) There is non-constant scatter
(4) Errors are not independent
(5) Errors are not normally distributed
Residual analysis - ANSWER-Residual analysis can be used to check if a model is
appropriate. If the model is appropriate, the residuals will have no pattern.
Detecting non-planar data (also called non-linearity) - ANSWER-(1) A curved residuals
versus fitted values plot
(2) Curved residuals vs explanatory variables plot
(3) Pairs plot/ co plot should be parallel
Remedies for non-planar data: GAM plots (General Additive Models) - ANSWER-Use
GAM plots to decide which variables we need to transform and how. The GAM plots
show the estimated functions of the variables obtained by smoothing. These optimal
transformations are used to suggest simpler transformations of explanatory variables
(e.g. quadratic, splines). This also applies for logistic regression
Remedies for non-planar data: the ladder of powers - ANSWER-Use the Box-Cox plot
to suggest the optimal value by which the response variable should be transformed.
Transforming the response according to the Box-Cox plot may fix non-normality of
residuals, unequal variances in response, non-planarity of model and need to transform
covariates indicated by GAM plots.
,Fitting Splines - ANSWER-An alternative to polynomials are splines. They are piecewise
cubics which join smoothly at knots that give a more flexible fit to the data. Values at
one point are not affected by values at distant points, unlike polynomials.
Detecting non-constant scatter - ANSWER-Plot of residuals versus fitted values shows
curvature
The importance of non-constant scatter - ANSWER-A multiple regression model
specifies that the scatter about the regression plane is uniform. All tests and confidence
intervals rely on this.
Constant scatter - ANSWER-Scatter does not depend on the explanatory variables or
the mean of the response. Scatter is measured by the size of the residuals.
The funnel effect - ANSWER-The funnel effect in residuals versus fitted values plot
occurs when scatter increases as the mean response increases.
Remedies for unequal scatter - ANSWER-(1) Transform the response (ladder of
powers)
(2) Estimate the variances of the observations using weighted least squares
Weighted least squares - ANSWER-Observations need to have constant variance. If the
n observation has variance vnσ² then we can get a valid test by using weighted least
squares, minimising the sum of the weighted squared residuals.
Weighted squared residuals - ANSWER-∑(r²)/v
(sum of residuals squared divided by weighting)
Residual sum of squares - ANSWER-RSS=∑r²
Or the sum of residuals squared
Finding the required weights to minimise weighted squared residuals - ANSWER-(1)
Plot the squared residuals versus the fitted values
(2) Smooth the plot
(3) Estimate the variance of an observation by the smoothed squared residual
(4) Weight is reciprocal of smoothed squared residual
The issue of non-normality - ANSWER-Non-normality is not such a crucial assumption,
but it can be important if the errors have a long-tailed distribution since this will imply
there are several outliers. The normality assumption is important for prediction.
Detecting non-normality: Normal QQ plots - ANSWER-QQ plot
(1) Normal if along line
(2) Short tailed if low values are above line and high values are below, flat
(3) Long tailed if low values are below line and high values are above, steep
(4) Right skewed if smile shaped
, (5) Left skewed if frown shaped
Detecting non-normality: Weisberg-Bingham test - ANSWER-WB is the square of the
correlation of the normal plot, which measures how straight the plot is. It is between 0
and 1. Values close to 1 indicate normality. WB.test computes the p value for the test
statistic, with the null hypothesis being that the sample is normal (small p = not normal)
Remedies for non-normality - ANSWER-Transform the response using the power
transformation
Outlier - ANSWER-An outlier is a point that has a larger or smaller y value than the
model would suggest. It can be due to a genuine large error, ε, or to a typographical
error in recording the data.
High leverage point - ANSWER-A high leverage point is a point with extreme values of
the explanatory variable.
High leverage outlier - ANSWER-A high leverage outlier can attract the fitted plane,
distorting the fit. They will have large HMDs. In extreme cases, it may not have a big
residual, meaning we may not be able to tell if it is an outlier. In extreme cases it can
increase R².
Low leverage outlier - ANSWER-Does not distort the fit to the same extent. It usually
has a big residual, and a low HMD. It inflates the standard errors and decreases R².
Detecting high leverage points: Hat matrix diagonals - ANSWER-High leverage points
do not necessarily have large residuals, so it can be difficult to detect them in a residual
plot. They can be detected by calculating hat matrix diagonals, which measure the
amount of pull ith data point has on the fitted regression surface. An HMD larger than
3(k+1)/n is considered extreme.
hatvalues(whatever.lm)
Hat matrix diagonals - ANSWER-Fitted Y = H*Y. The ith diagonal element measures the
extent to which the yi depends on the fitted value.
Recognising large residuals: studentised residuals - ANSWER-We need to standardise
residuals by dividing them by their standard deviations. You can do this by using
internally (standardised) or externally (studentised) residuals. They should be between -
2 and 2 with 95% probability.
s - ANSWER-Estimate of residual variance, σ²
Studentised residuals - ANSWER-var(ei) = (1-hii)σ²
= ei/(sqrt(1-hii)s²)
where s is the usual estimate of residual variance
