Links: Index, Third Year, Fall 2025 Term 1, STAT 306 - Finding Relationships in Data
2026-2027 STAT 306 - Finding Relationships in Data 03-09 QUESTIONS AND ANSWERS University of British
Columbia
#classnotes
Subject: MATH307 MATH340 #STAT306 PHYS119
Topic :: Simple linear regression (SLR)
Slide 1 - Relationships between two variables
Deterministic relations: value of one variable is entirely determined by the value of another
variable - there is no uncertainty.
Stochastic relations: situations in which the outcome of a given input cannot be precisely
predicted - there is uncertainty.
This is due to some variables not being considered, or noise/error.
eg. Relationship between height and weight - other factors such as age, lifestyle isn't
considered and impacts the relationship.
Typically includes an error term (ϵ) to account for variability in the response/outcome.
Types of relations:
Linear relation
Quadratic relation
Exponential relation
Y : response variable, output variable, NOT dependent variable
X: predictors, feature, input variable, covariate, regressor, NOT independent variable
iClicker
Proportion Z test - for proportions of 2 populations
Paired t-test - determines the mean difference between 2 sets
Two-sample t-test - test whether the unknown population means of two groups are equal or
not.
, Chi-square test - test whether the unknown population means of two groups are equal or
not.
ANOVA - compare the means of two or more groups by analyzing variance.
A straight line model could be used as a reasonable approximation for a relation, this is
called simple linear regression.
Slide 2 - Simple linear regression
Explanatory variable (X) : the variable used to predict the response.
Response variable (Y ): what we are trying to predict/explain
There is a probability distribution of Y for each value of X
The means of these distributions are linearly relation to x
x are constants/variables
X is a random variable
The explanatory variable x is assumed to be fixed for each individual/sample
The model relating X and Y is given by:
Yi = β0 + β1xi + ϵi
xi is the value of the explanatory variable for the ith sample (a scalar/constant)
β0, β1 are parameters (constants)
ϵi is a random error term (random variable)
Yi the response variable for the ith sample (random variable)
the distribution is based on the error term - which assumes to be normal
β0 = intercept, value of Y when X = 0
β1 = slope, how much change in Y to expect for a unit increase in X
ϵi = captures the variability of the response not explained by the model.
has a distribution (assume to be normal),
mean E[ϵ] = 0,
variance (all errors ei have the same variance)
The output from a model when given an input
-> mean of the distribution based on the input and the standard the deviation of the input. It
is not a singular number,
it is the mean of the distribution of what the output could be.
Given that the E[ϵ] = 0, when trying to predict the value of the output given an input using
the model, the output would follow the form "output + ϵ" but since the output is the mean of
, the distribution this would result in "mean + 0".
The regression line is the conditional average of Y for a given value of X = x
E[Y |X = x] = y = β0 + β1x
Yi = β0 + β1x + ϵi
is a given point, which is off the line (because of the error term, each error term is different
for each point)
We can estimate the values of X in a population using β0 and β1 aka β^ 0and β^1 with a
sample.
Residual (ei)
ei = Yi − Y^i
difference between the i-th observed point and the point on the regression line.
Residual sum of squares (RSS): sum of squared difference between the observed point
and the expected point.
We want to find the values of β0 and β1 to minimize the RSS.
Residuals are sensitive to outliers.
-> Outliers will "drag" the regression line towards the outlier -> increasing/decrease the
slope and intercept -> the residuals are also greatly impacted.
Low-x, high-y outlier → higher intercept, smaller slope.
High-x, high-y outlier → lower intercept, larger slope.
Center-x, high-y outlier → mainly increases intercept.
We can find the values of β0 and β1 that minimizes the value of RSS using algebra.
eg.
Interpretation of β0
"an increase of 1 unit in x is associated with an expected increase of β1 in y."
Y^ = β0^ + β1 ^ x: predicted value
2026-2027 STAT 306 - Finding Relationships in Data 03-09 QUESTIONS AND ANSWERS University of British
Columbia
#classnotes
Subject: MATH307 MATH340 #STAT306 PHYS119
Topic :: Simple linear regression (SLR)
Slide 1 - Relationships between two variables
Deterministic relations: value of one variable is entirely determined by the value of another
variable - there is no uncertainty.
Stochastic relations: situations in which the outcome of a given input cannot be precisely
predicted - there is uncertainty.
This is due to some variables not being considered, or noise/error.
eg. Relationship between height and weight - other factors such as age, lifestyle isn't
considered and impacts the relationship.
Typically includes an error term (ϵ) to account for variability in the response/outcome.
Types of relations:
Linear relation
Quadratic relation
Exponential relation
Y : response variable, output variable, NOT dependent variable
X: predictors, feature, input variable, covariate, regressor, NOT independent variable
iClicker
Proportion Z test - for proportions of 2 populations
Paired t-test - determines the mean difference between 2 sets
Two-sample t-test - test whether the unknown population means of two groups are equal or
not.
, Chi-square test - test whether the unknown population means of two groups are equal or
not.
ANOVA - compare the means of two or more groups by analyzing variance.
A straight line model could be used as a reasonable approximation for a relation, this is
called simple linear regression.
Slide 2 - Simple linear regression
Explanatory variable (X) : the variable used to predict the response.
Response variable (Y ): what we are trying to predict/explain
There is a probability distribution of Y for each value of X
The means of these distributions are linearly relation to x
x are constants/variables
X is a random variable
The explanatory variable x is assumed to be fixed for each individual/sample
The model relating X and Y is given by:
Yi = β0 + β1xi + ϵi
xi is the value of the explanatory variable for the ith sample (a scalar/constant)
β0, β1 are parameters (constants)
ϵi is a random error term (random variable)
Yi the response variable for the ith sample (random variable)
the distribution is based on the error term - which assumes to be normal
β0 = intercept, value of Y when X = 0
β1 = slope, how much change in Y to expect for a unit increase in X
ϵi = captures the variability of the response not explained by the model.
has a distribution (assume to be normal),
mean E[ϵ] = 0,
variance (all errors ei have the same variance)
The output from a model when given an input
-> mean of the distribution based on the input and the standard the deviation of the input. It
is not a singular number,
it is the mean of the distribution of what the output could be.
Given that the E[ϵ] = 0, when trying to predict the value of the output given an input using
the model, the output would follow the form "output + ϵ" but since the output is the mean of
, the distribution this would result in "mean + 0".
The regression line is the conditional average of Y for a given value of X = x
E[Y |X = x] = y = β0 + β1x
Yi = β0 + β1x + ϵi
is a given point, which is off the line (because of the error term, each error term is different
for each point)
We can estimate the values of X in a population using β0 and β1 aka β^ 0and β^1 with a
sample.
Residual (ei)
ei = Yi − Y^i
difference between the i-th observed point and the point on the regression line.
Residual sum of squares (RSS): sum of squared difference between the observed point
and the expected point.
We want to find the values of β0 and β1 to minimize the RSS.
Residuals are sensitive to outliers.
-> Outliers will "drag" the regression line towards the outlier -> increasing/decrease the
slope and intercept -> the residuals are also greatly impacted.
Low-x, high-y outlier → higher intercept, smaller slope.
High-x, high-y outlier → lower intercept, larger slope.
Center-x, high-y outlier → mainly increases intercept.
We can find the values of β0 and β1 that minimizes the value of RSS using algebra.
eg.
Interpretation of β0
"an increase of 1 unit in x is associated with an expected increase of β1 in y."
Y^ = β0^ + β1 ^ x: predicted value
