Partial Effects in Multiple regression
Learning Objectives:
● Compute and interpret effect sizes for single predictors in the multiple regression model
● Test an hypothesis about single (or sets of) predictors in the multiple regression model
● Draw a conclusion about hypotheses for single (or sets) of parameters in the multiple
regression model
Eg Is class size associated with academic performance among schools with a similar percentage
of students receiving free meals?
→ controlling for percentage of free meals (moderator relationship)
(1) Multiple Regression
1.1 Effect Sizes in Multiple Regression
We cannot use b to judge the strength of the partial association between x and y as b depends on the
scale on which x and y were measured
Eg if one predictor is measured in metres and another predictor is measured in kilograms,
their coefficients will be on different scales
→ hence, we inspect effect size instead
1. Standardised regression coefficient: b*
2
2. Squared partial correlation: r p
3. Change in explained variation: △ R 2
1.1.1 Standardised Regression Coefficient (b*)
We can scale each of the b coefficient in the multiple regression model using the:
● SD of the respective predictor (x)
● SD of the outcome variable (y)
Hence, b* is the amount of SDs y is expected to change when x i increases with 1 SD (controlling for
all other predictors in the model)
Rule of thumb for interpretation
● 0 - .10: negligible
● .10 - .30: small
● .30 - .50: moderate
● .50 ≤ large
Eg
^
AP=9.981 −0.067 ∗ PFM +0.003 ∗CS
sx 1 9.068
For PFM: b 1∗¿ b1 ( )=−0.067( )=−0.91
sy 0.667
, 2
1.1.2 Squared Partial Correlation (r p )
Represents the proportion of variance in y not associated with any other x's that is
explained by x 1
In a model with 2 predictors, the partial correlation between x 1 and y, controlling for x 2:
r yx 1−r yx2 r x1 x2
r yx1. x 2=
√❑
Which is easier to compute when squared:
2 R 2−r x22 proportion variation∈ y uniquely explained by x 1
r yx1. x 2 = 2 →
1−r x2 proportion variation∈ y not explained by x 2
Rule of thumb to interpret:
● 0 - .01: negligible
● .01 - .06: small
● .06 - .14: moderate
● .14 ≤ large
Eg
2 2
2 R −r x2 0.845−(−0.919)2 0.004
r yx1. x 2 = = = = 0.002
1−r x2
2
1−(−0.919)
2
0.155
Interpretation: class size explains 0.2% of the differences in academic performance that were not yet
explained by the percentage of students with free meals. This is a negligible effect.
1.1.3 R-Squared Change ( △ R 2)
The difference in explained variation when comparing two models
1. Complete model with all predictors
Eg ^
y c =a+b1 x1 +b 2 x 2
2. Reduced model which includes all predictors, apart from the one for which
we want to know the partial effect
Eg ^
y r =a+b 1 x 1
R-squared change: △ R 2=Rc 2−Rr 2
Learning Objectives:
● Compute and interpret effect sizes for single predictors in the multiple regression model
● Test an hypothesis about single (or sets of) predictors in the multiple regression model
● Draw a conclusion about hypotheses for single (or sets) of parameters in the multiple
regression model
Eg Is class size associated with academic performance among schools with a similar percentage
of students receiving free meals?
→ controlling for percentage of free meals (moderator relationship)
(1) Multiple Regression
1.1 Effect Sizes in Multiple Regression
We cannot use b to judge the strength of the partial association between x and y as b depends on the
scale on which x and y were measured
Eg if one predictor is measured in metres and another predictor is measured in kilograms,
their coefficients will be on different scales
→ hence, we inspect effect size instead
1. Standardised regression coefficient: b*
2
2. Squared partial correlation: r p
3. Change in explained variation: △ R 2
1.1.1 Standardised Regression Coefficient (b*)
We can scale each of the b coefficient in the multiple regression model using the:
● SD of the respective predictor (x)
● SD of the outcome variable (y)
Hence, b* is the amount of SDs y is expected to change when x i increases with 1 SD (controlling for
all other predictors in the model)
Rule of thumb for interpretation
● 0 - .10: negligible
● .10 - .30: small
● .30 - .50: moderate
● .50 ≤ large
Eg
^
AP=9.981 −0.067 ∗ PFM +0.003 ∗CS
sx 1 9.068
For PFM: b 1∗¿ b1 ( )=−0.067( )=−0.91
sy 0.667
, 2
1.1.2 Squared Partial Correlation (r p )
Represents the proportion of variance in y not associated with any other x's that is
explained by x 1
In a model with 2 predictors, the partial correlation between x 1 and y, controlling for x 2:
r yx 1−r yx2 r x1 x2
r yx1. x 2=
√❑
Which is easier to compute when squared:
2 R 2−r x22 proportion variation∈ y uniquely explained by x 1
r yx1. x 2 = 2 →
1−r x2 proportion variation∈ y not explained by x 2
Rule of thumb to interpret:
● 0 - .01: negligible
● .01 - .06: small
● .06 - .14: moderate
● .14 ≤ large
Eg
2 2
2 R −r x2 0.845−(−0.919)2 0.004
r yx1. x 2 = = = = 0.002
1−r x2
2
1−(−0.919)
2
0.155
Interpretation: class size explains 0.2% of the differences in academic performance that were not yet
explained by the percentage of students with free meals. This is a negligible effect.
1.1.3 R-Squared Change ( △ R 2)
The difference in explained variation when comparing two models
1. Complete model with all predictors
Eg ^
y c =a+b1 x1 +b 2 x 2
2. Reduced model which includes all predictors, apart from the one for which
we want to know the partial effect
Eg ^
y r =a+b 1 x 1
R-squared change: △ R 2=Rc 2−Rr 2
