Research Methods in Communication With this estimation we can also define the
Science residuals:
Lecture 1: Introduction and linear
regression
Example wages and education
The residuals show how bad our estimation
is!
• Y hat = what model says
• Y = what reality is
• B0/B1 hat = things that come out of
your model
• B0 is the point where the line
crosses the Y axis
Scatterplot, for seeing the relationship • B1 is the slope
between 2 continuous variables
Regression results
- Intercept is B0
- Dependent variable = wage
- B0 = 4.97 expected wage if educ =
0
Simple regression - B1 = 0.79 increase in wage if
education increases by 1 year
Our model (= approximation of reality) is:
Wage = 4.97 + .079 * educ
Significance – is the coefficient
We don’t observe B0, B1 nor Ei
statistically different from 0?
Instead we estimate them along with the
dependent variable: - B0: t = 9.305, p < 0.000 → it is!
- B1: t = 20.284, p < 0.000 → it is!
,Generally, the interpretations of b are On residuals
- B0 expected Y (Y hat) if X = 0 To answer that question we need to
- B1 change in Y hat if X increases understand better what residuals are
in 1 unit
We assume that they:
Standardized regression
- Have mean zero
We might actually want a standardizes - Are not related with X
regression. Why?
We’ll see that the behavior of their
- Sometimes X = 0 doesn’t make a variance is very important too!
lot of sense. E.g. age = 0
What are they conceptually?
- B1 depends on the units of X which
makes it difficult to interpret - The part of Y not explained by X;
how far is our prediction of Y from
How does the standardized regression
the real Y
works?
Example
- We replace Y and X with their
standardized (Z) versions
- Remember that:
- Zx and Zy have mean 0 and
standard deviation (sd) 1
We can write this regression as
What happens now?
- Since both variables have mean 0
→ B0 = 0
- Our new coefficient interpretation
is that if X increases by 1 unit sd, Regression line
then Y increases by B1 sd’s The sum of the residuals is expected to be
Regression line 0
There are many possible lines → which We define the regression line by making
one should we draw? Worded differently, ALL the residuals as low as possible!
which Bhat0 and Bhat1 ‘fit’ best our data? Should we reduce
We need to construct the line that best - Not really, remember that there are
approximates reality positive and negative residuals
- They might cancel out
,To avoid this issue we will square the sum: The TSS can be easily seen when we
predict Y only with its average
- It’s the sum of the square distance
- This is termed the Residual Sum of
in the plot!
Squares (RSS)
- Least Squares Method = Besides the totals, sometimes is useful to
minimizing the RSS seen the mean errors
Splitting the Sum of Squares - To do this we divide by the degrees
of freedom
The total variance of Y is…
Degrees of freedom = number of
independent pieces of information used to
- Practically: the ‘error’ you make calculate a statistic
when predicting Y with Yflathat
- Formally: how much variance of Y
is there to explain K = the number of independent variables
The variance of Y which CAN be N = sample
explained by X is…
= the mean squared total error
- Practically: what the regression
The square root of the
explains
is also referred as RMSE
The variance of Y which CANNOT be
explained by X is… - It’s one of the most common
measures of regression quality!
Coefficient of determination R-squared
- Practically: what the regression
does NOT explain R2 is the percentage (between 0 and 1) of
the total variance that is explained by the
regression…
How can we see these results?
So, the percentage of unexplained
RSS = 756.5755 SSE = 8451.138 TSS = variance is…
9207.713
We can also see the SSE in the ANOVA
table
, Residuals and regression quality Lecture 2: Multiple Regression
Keep in mind that we assumed linearity In multiple regression our interest is to
explain Y as a function of several
- The conditional means of Y are
independent variables
best shown in a line
- In other words: we assume that The new model can be written as
Back to the example: let’s explain wage by
What if I don’t have linearity (see plot with education and age
conditional means in red below)?
Causality
- MSE is not an appropriate measure
We are interested in the question: is there a
for assessing the regression quality
causal effect of X on Y?
Requirements for causality
- X and Y are associated
- X (independent) is realized earlier
than Y (dependent)
- We have excluded all other
alternative explanations of Y
Multiple regression aim
Keep in mind that…
As we will see assumption are very - Correlation does not imply
important in regression! causation
- The third requirement for causation
is the most difficult to fulfil
We use multiple regression because we
want to comply with the third requirement
We exclude alternative explanations by
controlling for several variable.
- However, it’s not that simple
We should think about controls in multiple
regression falling within three cases or
scenarios
Science residuals:
Lecture 1: Introduction and linear
regression
Example wages and education
The residuals show how bad our estimation
is!
• Y hat = what model says
• Y = what reality is
• B0/B1 hat = things that come out of
your model
• B0 is the point where the line
crosses the Y axis
Scatterplot, for seeing the relationship • B1 is the slope
between 2 continuous variables
Regression results
- Intercept is B0
- Dependent variable = wage
- B0 = 4.97 expected wage if educ =
0
Simple regression - B1 = 0.79 increase in wage if
education increases by 1 year
Our model (= approximation of reality) is:
Wage = 4.97 + .079 * educ
Significance – is the coefficient
We don’t observe B0, B1 nor Ei
statistically different from 0?
Instead we estimate them along with the
dependent variable: - B0: t = 9.305, p < 0.000 → it is!
- B1: t = 20.284, p < 0.000 → it is!
,Generally, the interpretations of b are On residuals
- B0 expected Y (Y hat) if X = 0 To answer that question we need to
- B1 change in Y hat if X increases understand better what residuals are
in 1 unit
We assume that they:
Standardized regression
- Have mean zero
We might actually want a standardizes - Are not related with X
regression. Why?
We’ll see that the behavior of their
- Sometimes X = 0 doesn’t make a variance is very important too!
lot of sense. E.g. age = 0
What are they conceptually?
- B1 depends on the units of X which
makes it difficult to interpret - The part of Y not explained by X;
how far is our prediction of Y from
How does the standardized regression
the real Y
works?
Example
- We replace Y and X with their
standardized (Z) versions
- Remember that:
- Zx and Zy have mean 0 and
standard deviation (sd) 1
We can write this regression as
What happens now?
- Since both variables have mean 0
→ B0 = 0
- Our new coefficient interpretation
is that if X increases by 1 unit sd, Regression line
then Y increases by B1 sd’s The sum of the residuals is expected to be
Regression line 0
There are many possible lines → which We define the regression line by making
one should we draw? Worded differently, ALL the residuals as low as possible!
which Bhat0 and Bhat1 ‘fit’ best our data? Should we reduce
We need to construct the line that best - Not really, remember that there are
approximates reality positive and negative residuals
- They might cancel out
,To avoid this issue we will square the sum: The TSS can be easily seen when we
predict Y only with its average
- It’s the sum of the square distance
- This is termed the Residual Sum of
in the plot!
Squares (RSS)
- Least Squares Method = Besides the totals, sometimes is useful to
minimizing the RSS seen the mean errors
Splitting the Sum of Squares - To do this we divide by the degrees
of freedom
The total variance of Y is…
Degrees of freedom = number of
independent pieces of information used to
- Practically: the ‘error’ you make calculate a statistic
when predicting Y with Yflathat
- Formally: how much variance of Y
is there to explain K = the number of independent variables
The variance of Y which CAN be N = sample
explained by X is…
= the mean squared total error
- Practically: what the regression
The square root of the
explains
is also referred as RMSE
The variance of Y which CANNOT be
explained by X is… - It’s one of the most common
measures of regression quality!
Coefficient of determination R-squared
- Practically: what the regression
does NOT explain R2 is the percentage (between 0 and 1) of
the total variance that is explained by the
regression…
How can we see these results?
So, the percentage of unexplained
RSS = 756.5755 SSE = 8451.138 TSS = variance is…
9207.713
We can also see the SSE in the ANOVA
table
, Residuals and regression quality Lecture 2: Multiple Regression
Keep in mind that we assumed linearity In multiple regression our interest is to
explain Y as a function of several
- The conditional means of Y are
independent variables
best shown in a line
- In other words: we assume that The new model can be written as
Back to the example: let’s explain wage by
What if I don’t have linearity (see plot with education and age
conditional means in red below)?
Causality
- MSE is not an appropriate measure
We are interested in the question: is there a
for assessing the regression quality
causal effect of X on Y?
Requirements for causality
- X and Y are associated
- X (independent) is realized earlier
than Y (dependent)
- We have excluded all other
alternative explanations of Y
Multiple regression aim
Keep in mind that…
As we will see assumption are very - Correlation does not imply
important in regression! causation
- The third requirement for causation
is the most difficult to fulfil
We use multiple regression because we
want to comply with the third requirement
We exclude alternative explanations by
controlling for several variable.
- However, it’s not that simple
We should think about controls in multiple
regression falling within three cases or
scenarios
