UNIVERSITY OF LIMPOPO
SSTB031 STUDY GUIDE
APPLIED LINEAR REGRESSION
Compiled
By
Mr Maluleke H
2022
, Table of Contents
CHAPTER 1: SIMPLE LINEAR REGRESSION AND CORRELATION ................. 4
1. Introduction.................................................................................................... 4
1.1 Simple Linear Regression Model ............................................................. 4
1.2 Estimation of the Regression Parameters................................................ 6
1.3. Correlation ............................................................................................ 11
What does the standard error means? ............................................................ 15
The sampling distribution of the slope ( 1 ) of the regression model ............... 15
Hypothesis testing for 0 and 1 ................................................................... 17
CHAPTER 2: DIAGNOSTICS FOR SIMLPLE LINEAR REGRESSION .............. 19
2.1 Introduction................................................................................................ 19
2.2 Residual Analysis ...................................................................................... 19
Examination of Residuals ................................................................................ 20
Different Patterns of Residual plots ............................................................. 21
2.3 Identification of Outliers ............................................................................. 22
2.4 Detection of Influential observation ........................................................... 23
2.3.1 Leverage procedure ............................................................................ 23
See an electronic book!!! ............................................................................. 23
2.3.2 Deleted residual method ..................................................................... 23
2.3.3 Cook’s distance .................................................................................. 24
CHAPTER 3: MULTIPLE REGRESSION MODEL .............................................. 26
3.1 Multiple Linear Regression Model .......................................................... 26
Matrix Approach to Regression Analysis ..................................................... 26
3.2 Estimation of Regression coefficients .................................................... 28
3.3 Test for the significant of the overall model............................................ 31
3.4 Test for the significant of the regression coefficient ............................... 32
3.5 Inferences about the mean response .................................................... 33
3.6 Inferences about the individual response fitted values .......................... 34
3.7 Multiple Coefficient of determination (R2)............................................... 34
3.8 Testing Portions of the Multiple Regression Model ................................ 35
2
,CHAPTER 4: DIAGNOSTICS FOR MULTIPLE REGRESSION ......................... 40
4.1 Introduction................................................................................................ 40
4.2 Residual Analysis ...................................................................................... 40
Examination of Residuals ................................................................................ 41
different patterns of the residual plots: as in simple linear regression .......... 42
4.3 Identification of Outliers ............................................................................. 43
4.4 Detection of Influential observation ........................................................... 43
4.4.1 Leverage procedure ............................................................................ 43
4.4.2 Deleted residual method ..................................................................... 44
4.4.3 Cook’s distance .................................................................................. 44
4.5 Collinearity................................................................................................. 45
CHAPTER 5: MODEL-BUILDING ....................................................................... 46
5.1 Backward elimination ................................................................................ 46
5.2 Forward elimination ................................................................................... 46
5.3 Stepwise Regression ................................................................................. 46
Appendices ......................................................................................................... 47
Appendix A: Class Examples .......................................................................... 47
Appendix B: Time table ....................................................................................... 51
Appendix C: Module Outline ............................................................................... 52
3
, CHAPTER 1: SIMPLE LINEAR REGRESSION AND
CORRELATION
1.1 Introduction
We have dealt with data that involved a single variable x. In this section, we shall
deal with paired variables x and y. Paired variables means that, for each value of y
there is a corresponding value of x. Here's an example of paired variables:
x 24 15 17 32 19 18 25 34
y 22 11 14 30 17 12 23 31
When confronted with paired data, we are often confronted with two questions:
Is there a relationship between the variable x and its counterpart y, and
If so what is the exact nature of the relationship?
Can you predict the value of y given the value of x?
1.2 Simple Linear Regression Model
Regression explores the expression of this relationship with the use of a regression
Line.
We can establish this statistical relationship in the form of a linear equation. This
equation is used to predict the value of one variable given the value of its partner.
The equation is known as a regression line. The analysis designed to derive an
equation for the line that best models the relationship between the dependent and
independent variables is called the regression analysis. This equation has the
mathematical form:
yi 0 1 xi + i (0)
where yi is the value of the dependent variable for the ith observation.
xi is the value of the independent variable for the ith observation.
i is a random error and
4
SSTB031 STUDY GUIDE
APPLIED LINEAR REGRESSION
Compiled
By
Mr Maluleke H
2022
, Table of Contents
CHAPTER 1: SIMPLE LINEAR REGRESSION AND CORRELATION ................. 4
1. Introduction.................................................................................................... 4
1.1 Simple Linear Regression Model ............................................................. 4
1.2 Estimation of the Regression Parameters................................................ 6
1.3. Correlation ............................................................................................ 11
What does the standard error means? ............................................................ 15
The sampling distribution of the slope ( 1 ) of the regression model ............... 15
Hypothesis testing for 0 and 1 ................................................................... 17
CHAPTER 2: DIAGNOSTICS FOR SIMLPLE LINEAR REGRESSION .............. 19
2.1 Introduction................................................................................................ 19
2.2 Residual Analysis ...................................................................................... 19
Examination of Residuals ................................................................................ 20
Different Patterns of Residual plots ............................................................. 21
2.3 Identification of Outliers ............................................................................. 22
2.4 Detection of Influential observation ........................................................... 23
2.3.1 Leverage procedure ............................................................................ 23
See an electronic book!!! ............................................................................. 23
2.3.2 Deleted residual method ..................................................................... 23
2.3.3 Cook’s distance .................................................................................. 24
CHAPTER 3: MULTIPLE REGRESSION MODEL .............................................. 26
3.1 Multiple Linear Regression Model .......................................................... 26
Matrix Approach to Regression Analysis ..................................................... 26
3.2 Estimation of Regression coefficients .................................................... 28
3.3 Test for the significant of the overall model............................................ 31
3.4 Test for the significant of the regression coefficient ............................... 32
3.5 Inferences about the mean response .................................................... 33
3.6 Inferences about the individual response fitted values .......................... 34
3.7 Multiple Coefficient of determination (R2)............................................... 34
3.8 Testing Portions of the Multiple Regression Model ................................ 35
2
,CHAPTER 4: DIAGNOSTICS FOR MULTIPLE REGRESSION ......................... 40
4.1 Introduction................................................................................................ 40
4.2 Residual Analysis ...................................................................................... 40
Examination of Residuals ................................................................................ 41
different patterns of the residual plots: as in simple linear regression .......... 42
4.3 Identification of Outliers ............................................................................. 43
4.4 Detection of Influential observation ........................................................... 43
4.4.1 Leverage procedure ............................................................................ 43
4.4.2 Deleted residual method ..................................................................... 44
4.4.3 Cook’s distance .................................................................................. 44
4.5 Collinearity................................................................................................. 45
CHAPTER 5: MODEL-BUILDING ....................................................................... 46
5.1 Backward elimination ................................................................................ 46
5.2 Forward elimination ................................................................................... 46
5.3 Stepwise Regression ................................................................................. 46
Appendices ......................................................................................................... 47
Appendix A: Class Examples .......................................................................... 47
Appendix B: Time table ....................................................................................... 51
Appendix C: Module Outline ............................................................................... 52
3
, CHAPTER 1: SIMPLE LINEAR REGRESSION AND
CORRELATION
1.1 Introduction
We have dealt with data that involved a single variable x. In this section, we shall
deal with paired variables x and y. Paired variables means that, for each value of y
there is a corresponding value of x. Here's an example of paired variables:
x 24 15 17 32 19 18 25 34
y 22 11 14 30 17 12 23 31
When confronted with paired data, we are often confronted with two questions:
Is there a relationship between the variable x and its counterpart y, and
If so what is the exact nature of the relationship?
Can you predict the value of y given the value of x?
1.2 Simple Linear Regression Model
Regression explores the expression of this relationship with the use of a regression
Line.
We can establish this statistical relationship in the form of a linear equation. This
equation is used to predict the value of one variable given the value of its partner.
The equation is known as a regression line. The analysis designed to derive an
equation for the line that best models the relationship between the dependent and
independent variables is called the regression analysis. This equation has the
mathematical form:
yi 0 1 xi + i (0)
where yi is the value of the dependent variable for the ith observation.
xi is the value of the independent variable for the ith observation.
i is a random error and
4
