📔
Ch 9: The Linear Model
Class VOS
Type Chapter
Materials Discovering Statistics Using IBM SPSS Statistics
Reviewed
An introduction to the linear model (regression)
💡 Outcome = ax + b = b₁X + b₀
A linear model can be described entirely by a constant (b₀) and by parameters
associated with each predictor (bₛ).
Residuals (= deviations) = the differences between what a linear model predicts and
the observed data.
Hierarchical regression = you select the predictors based on past work and decide in
which order to enter them into the model. Known predictors in the model first.
Residual sum of squares (SSᵣ)
!: Sometimes the predicted value of the outcome is less than the actual value and
sometimes it is greater. Consequently, some residuals are positive but others are
negative, and if we summed them up they would cancel out → the solution is to
square them before we add them up; same concept applies to the fit of the
model.
→ residual sum of squares SSᵣ: if the squared differences are large, the model is
not representative of the data (there is a lot of error in the prediction).
Ch 9: The Linear Model 1
, Goodness of fit
Goodness of fit = The values of b that define the model of best fit we assess how
well this model fits the data. You need to compare the baseline model with the
new model with new predictor(s).
Total sum of squares (SSₜ) = squared differences to give the sum of squared
differences. The sum of squared differences is known as the total sum of squares
and represents how good the mean is as a model of the observed outcome
scores.
🔢 R^2 = SSₘ / SSₜ
!: R^2 represents the variance of the model SSₘ.
If SSₘ is large, the linear model is very different from using the mean to predict
the outcome variable → implies that the linear model has made a big
improvement to predicting the outcome variable.
If SSₘ is small, the linear model is little better than using the mean.
F-test
F = the amount of systematic variance divided y the amount of unsystematic
variance, the model compared to the error in the model.
F in case of the goodness of fit = F is based upon the ratio of the improvement
due to the model (SSₘ) and the error of the model (SSₜ).
Mean sum of squares = the sum of squares divided by the associated degrees of
freedom. The degrees of freedom are the number of predictors in the model (k).
SSᵣ = number of observations (N) - number of parameters being estimated (b)
Bias in linear models
Questions to ask yourself:
1. Is the model influenced by a small number of cases?
→ look at outliers and/or influential cases
2. Does the model generalize to other samples?
→ Additivity and linearity: the outcome variable should be linearly related to any
predictors, and, with several predictors, their combined effect is best described by
Ch 9: The Linear Model 2
, adding their effects together. (Basically the assumptions)
Generalization = criitical additional step; we must restrict any conclusions to the
sample used.
Standardized residuals = residuals converted to z-scores and so are expressed
in standard deviation units.
Multicollinearity
Multicollinearity = a strong correlation between two or more predictors.
Perfect collinearity = at least one predictor is perfectly linear combination of the
others. It becomes impossible to obtain unique estimates of the regression
coefficients, because there are an infinite number of combinations of coefficients
that would work equally well.
→ If two predictors are perfectly correlated, then the values of b for each variable
are interchangible.
Three problems related to collinearity:
1. Untrustworthy b’s: As collinearity increases, so do the standard errors of the
b coefficients.
2. Limit size of R: R is a measure of the correlation between predicted values of
the outcome and the observed valyes and that R^2 indicates the variance in
the outcome for which the model counts.
3. Importance of predictors: Multicollineariy between predictors amkes it difficult
to assess the individual importance of a predictor.
Ch 9: The Linear Model 3
Ch 9: The Linear Model
Class VOS
Type Chapter
Materials Discovering Statistics Using IBM SPSS Statistics
Reviewed
An introduction to the linear model (regression)
💡 Outcome = ax + b = b₁X + b₀
A linear model can be described entirely by a constant (b₀) and by parameters
associated with each predictor (bₛ).
Residuals (= deviations) = the differences between what a linear model predicts and
the observed data.
Hierarchical regression = you select the predictors based on past work and decide in
which order to enter them into the model. Known predictors in the model first.
Residual sum of squares (SSᵣ)
!: Sometimes the predicted value of the outcome is less than the actual value and
sometimes it is greater. Consequently, some residuals are positive but others are
negative, and if we summed them up they would cancel out → the solution is to
square them before we add them up; same concept applies to the fit of the
model.
→ residual sum of squares SSᵣ: if the squared differences are large, the model is
not representative of the data (there is a lot of error in the prediction).
Ch 9: The Linear Model 1
, Goodness of fit
Goodness of fit = The values of b that define the model of best fit we assess how
well this model fits the data. You need to compare the baseline model with the
new model with new predictor(s).
Total sum of squares (SSₜ) = squared differences to give the sum of squared
differences. The sum of squared differences is known as the total sum of squares
and represents how good the mean is as a model of the observed outcome
scores.
🔢 R^2 = SSₘ / SSₜ
!: R^2 represents the variance of the model SSₘ.
If SSₘ is large, the linear model is very different from using the mean to predict
the outcome variable → implies that the linear model has made a big
improvement to predicting the outcome variable.
If SSₘ is small, the linear model is little better than using the mean.
F-test
F = the amount of systematic variance divided y the amount of unsystematic
variance, the model compared to the error in the model.
F in case of the goodness of fit = F is based upon the ratio of the improvement
due to the model (SSₘ) and the error of the model (SSₜ).
Mean sum of squares = the sum of squares divided by the associated degrees of
freedom. The degrees of freedom are the number of predictors in the model (k).
SSᵣ = number of observations (N) - number of parameters being estimated (b)
Bias in linear models
Questions to ask yourself:
1. Is the model influenced by a small number of cases?
→ look at outliers and/or influential cases
2. Does the model generalize to other samples?
→ Additivity and linearity: the outcome variable should be linearly related to any
predictors, and, with several predictors, their combined effect is best described by
Ch 9: The Linear Model 2
, adding their effects together. (Basically the assumptions)
Generalization = criitical additional step; we must restrict any conclusions to the
sample used.
Standardized residuals = residuals converted to z-scores and so are expressed
in standard deviation units.
Multicollinearity
Multicollinearity = a strong correlation between two or more predictors.
Perfect collinearity = at least one predictor is perfectly linear combination of the
others. It becomes impossible to obtain unique estimates of the regression
coefficients, because there are an infinite number of combinations of coefficients
that would work equally well.
→ If two predictors are perfectly correlated, then the values of b for each variable
are interchangible.
Three problems related to collinearity:
1. Untrustworthy b’s: As collinearity increases, so do the standard errors of the
b coefficients.
2. Limit size of R: R is a measure of the correlation between predicted values of
the outcome and the observed valyes and that R^2 indicates the variance in
the outcome for which the model counts.
3. Importance of predictors: Multicollineariy between predictors amkes it difficult
to assess the individual importance of a predictor.
Ch 9: The Linear Model 3
