STAT 4230/6230
Introduction to Regression Chapter 2
In many applications, we would like to predict the value of some variable y based on one or
more other variables x1,...,xk. The variable y that we would like to predict is called the
response or dependent variable, while the x-variables are called the explanatory,
predictor or independent variables.
Example:
Suppose we want to predict the finishing times of runners in the Peachtree Road Race, a 6.2-mile
(10K) event held annually in Atlanta on July 4th. The response variable, y, represents the finishing
times of the runners in the race. Let’s consider a few scenarios:
1. We can collect data on the finishing times of all participants who have run this race in the
past. However, if the entire dataset is not available, we might need to randomly select a
sample of finishing times from previous races.
Using this sample, we can predict the finishing time for each individual in this year’s race
based on the average or expected finishing time from the sample. The prediction equation
would then be: 𝐸(𝑦) = 𝑦&
Using the average finishing time as a predictor for each runner's finishing time in this
year’s race is not entirely accurate, as it does not account for individual differences and
some important explanatory variables. This introduces some random error, which we can
estimate once the race occurs.
Thus, the finishing time for each individual in the race can be expressed as:
𝑦! = 𝐸(𝑦) + 𝜀! , 𝑓𝑜𝑟 𝑖 = 1, ⋯ , 𝑛
Alternatively, this can be written as:
2. The prediction for finishing times considered in Scenario 1 does not account for
important explanatory (predictor) variables such as the temperature on race day (𝑥" ),
, humidity level (𝑥# ), or the number of miles run per week during the preceding two
months (𝑥$ ). The process of identifying a mathematical model that relates y (finishing
time) to a set of predictor variables and best fits the data is part of a broader method
called regression analysis.
For instance, suppose the finishing times (y) of runners depended only on the temperature on
race day (𝑥" ). In this case, we can create a graph of the collected data on these two variables.
Observing the relationship in the plot allows us to determine a mathematical equation or model
that best represents the data.
If the relationship between y and (𝑥" ) is linear, we can determine a straight line that best fits
the data.
Figure 1: Scatterplot
This linear regression model predicts the expected or mean finishing time, E(y), and includes
parameters 𝛽% (the intercept) and 𝛽" (the slope):
𝐸(𝑦) = 𝛽% + 𝛽" 𝑥"
This regression model predicts the average or expected finishing time for a given temperature.
For example, if we were to examine finishing times for all races where the temperature was 70
degrees, the model would provide the average or expected finishing time for that temperature.
Similarly, for a temperature of 75 degrees, the model would predict the corresponding average or
expected finishing time.
Page 2
Introduction to Regression Chapter 2
In many applications, we would like to predict the value of some variable y based on one or
more other variables x1,...,xk. The variable y that we would like to predict is called the
response or dependent variable, while the x-variables are called the explanatory,
predictor or independent variables.
Example:
Suppose we want to predict the finishing times of runners in the Peachtree Road Race, a 6.2-mile
(10K) event held annually in Atlanta on July 4th. The response variable, y, represents the finishing
times of the runners in the race. Let’s consider a few scenarios:
1. We can collect data on the finishing times of all participants who have run this race in the
past. However, if the entire dataset is not available, we might need to randomly select a
sample of finishing times from previous races.
Using this sample, we can predict the finishing time for each individual in this year’s race
based on the average or expected finishing time from the sample. The prediction equation
would then be: 𝐸(𝑦) = 𝑦&
Using the average finishing time as a predictor for each runner's finishing time in this
year’s race is not entirely accurate, as it does not account for individual differences and
some important explanatory variables. This introduces some random error, which we can
estimate once the race occurs.
Thus, the finishing time for each individual in the race can be expressed as:
𝑦! = 𝐸(𝑦) + 𝜀! , 𝑓𝑜𝑟 𝑖 = 1, ⋯ , 𝑛
Alternatively, this can be written as:
2. The prediction for finishing times considered in Scenario 1 does not account for
important explanatory (predictor) variables such as the temperature on race day (𝑥" ),
, humidity level (𝑥# ), or the number of miles run per week during the preceding two
months (𝑥$ ). The process of identifying a mathematical model that relates y (finishing
time) to a set of predictor variables and best fits the data is part of a broader method
called regression analysis.
For instance, suppose the finishing times (y) of runners depended only on the temperature on
race day (𝑥" ). In this case, we can create a graph of the collected data on these two variables.
Observing the relationship in the plot allows us to determine a mathematical equation or model
that best represents the data.
If the relationship between y and (𝑥" ) is linear, we can determine a straight line that best fits
the data.
Figure 1: Scatterplot
This linear regression model predicts the expected or mean finishing time, E(y), and includes
parameters 𝛽% (the intercept) and 𝛽" (the slope):
𝐸(𝑦) = 𝛽% + 𝛽" 𝑥"
This regression model predicts the average or expected finishing time for a given temperature.
For example, if we were to examine finishing times for all races where the temperature was 70
degrees, the model would provide the average or expected finishing time for that temperature.
Similarly, for a temperature of 75 degrees, the model would predict the corresponding average or
expected finishing time.
Page 2
