October 2020
Summary Advanced Statistics
Table of Contents
1 ANCOVA .......................................................................................................................................... 2
2 MANOVA ....................................................................................................................................... 16
3 Discriminant Analysis .................................................................................................................... 42
4 Repeated Measures ...................................................................................................................... 59
5 RM-MANOVA ................................................................................................................................ 70
6 Bayesian Statistics ......................................................................................................................... 80
7 Matrix Algebra ............................................................................................................................ 121
~1~
,October 2020
1 ANCOVA
ANOVA model where you want to study the differences between groups with a covariate
(continuous predictor)
There are several ways of controlling the sources of error variance:
- Random assignment of subjects to groups.
- Add more factors to the experiment.
- Hold constant throughout the experiment so it doesn’t have effect
- Counterbalance effects.
These are experimental controls.
- Goal: Reduction of error
Alternatively, statistical controls are also possible:
- Add a factor (categorical IV) to the model: Blocking.
- Add a covariate (continuous IV) to the model: ANCOVA.
Help to explain more variance and reduce error variance
Why ANCOVA
1. Eliminate systematic differences (bias) between the experimental groups.
- This is especially relevant in nonexperimental (observational) or quasi-experimental studies.
- You remove initial differences between the groups on particular covariate / criteria
2. Reduce within group (i.e., error) variance.
- By making the groups more homogeneous.
- Less error → more power.
- Increase explained variance and increase power
ANCOVA is one step into the direction of explaining more variance and reducing error variance.
ANCOVA in experimental studies (Lab controlled)
- Systematic differences are already dealt with through random assignment of subjects to
groups, counterbalancing effects, . . .
- Therefore, bias due to variables outside the experiment is the same for all groups.
- In theory, the systematic differences between groups that remain are due to manipulation
of the experiment factors.
- ANCOVA is typically not used to eliminate systematic differences in experiment studies. But
when groups are small, systematic differences are more possible and ANCOVA can help
removing them.
Why use ANCOVA?
~2~
,October 2020
ANCOVA is still useful to reduce error variance and increase power
ANCOVA in nonexperimental studies
- Natural or intact groups (class rooms)
- Any systematic differences are due to either initial differences on some variables,
manipulation of treatment levels (factors) or a confounding of both
Example Class room:
You cannot separate them and control, some students might be better
Why use ANCOVA?
ANCOVA may reduce the systematic differences
- Use covariate to adjust the group means for initial differences
- Groups start equally on the covariates
As a bonus, ANCOVA is still useful to reduce the error variance
But there are some things we have to consider! There is no free lunch
Covariate
- Predictor
- Usually continuous variable (interval level)
- It is a source of variation not controlled for in the experiment, that we believe to have an
effect on the DV
- The idea is to remove initial DV-score differences due to the covariate before running the
ANOVA. This is referred to as statistically adjusting the DV
- To be useful, the covariate should be linearly related with the DV
- Use the relationship covariate – DV to make better predictions of the means in the groups
(less bias, less uncertainty)
Adjust the group means
How ANCOVA works
ANCOVA combines regression and ANOVA
The two conceptual steps of an ANCOVA:
1. Perform a regression analysis to predict the DV using the covariate. The residuals of this analysis
are corrected values of the DV
2. Perform an ANOVA on the corrected DV (residuals) to compare the groups
- Usually it is done in one step though
ANCOVA combines both steps in one model.
You can do it manually by means of hierarchical regression:
~3~
, October 2020
Step 1: Enter the covariate: Adjusting the means
Step 2: Enter the categorical factor, grouping factor: ANOVA on adjusted means
How much does explained variance change from step 1 to step 2?
There is an F test for this, Hierarchical regression
Perform F test of increase in R2 between Step 1 and Step 2 you do the ANCOVA
Example:
Factor with two levels: Two different instruction methods
Outcome is learning measure, how much students learned
Covariate is ability test provided at the beginning to make sure that groups do not differ too much
from each other from the beginning
Running the ANOVA:
~4~
Summary Advanced Statistics
Table of Contents
1 ANCOVA .......................................................................................................................................... 2
2 MANOVA ....................................................................................................................................... 16
3 Discriminant Analysis .................................................................................................................... 42
4 Repeated Measures ...................................................................................................................... 59
5 RM-MANOVA ................................................................................................................................ 70
6 Bayesian Statistics ......................................................................................................................... 80
7 Matrix Algebra ............................................................................................................................ 121
~1~
,October 2020
1 ANCOVA
ANOVA model where you want to study the differences between groups with a covariate
(continuous predictor)
There are several ways of controlling the sources of error variance:
- Random assignment of subjects to groups.
- Add more factors to the experiment.
- Hold constant throughout the experiment so it doesn’t have effect
- Counterbalance effects.
These are experimental controls.
- Goal: Reduction of error
Alternatively, statistical controls are also possible:
- Add a factor (categorical IV) to the model: Blocking.
- Add a covariate (continuous IV) to the model: ANCOVA.
Help to explain more variance and reduce error variance
Why ANCOVA
1. Eliminate systematic differences (bias) between the experimental groups.
- This is especially relevant in nonexperimental (observational) or quasi-experimental studies.
- You remove initial differences between the groups on particular covariate / criteria
2. Reduce within group (i.e., error) variance.
- By making the groups more homogeneous.
- Less error → more power.
- Increase explained variance and increase power
ANCOVA is one step into the direction of explaining more variance and reducing error variance.
ANCOVA in experimental studies (Lab controlled)
- Systematic differences are already dealt with through random assignment of subjects to
groups, counterbalancing effects, . . .
- Therefore, bias due to variables outside the experiment is the same for all groups.
- In theory, the systematic differences between groups that remain are due to manipulation
of the experiment factors.
- ANCOVA is typically not used to eliminate systematic differences in experiment studies. But
when groups are small, systematic differences are more possible and ANCOVA can help
removing them.
Why use ANCOVA?
~2~
,October 2020
ANCOVA is still useful to reduce error variance and increase power
ANCOVA in nonexperimental studies
- Natural or intact groups (class rooms)
- Any systematic differences are due to either initial differences on some variables,
manipulation of treatment levels (factors) or a confounding of both
Example Class room:
You cannot separate them and control, some students might be better
Why use ANCOVA?
ANCOVA may reduce the systematic differences
- Use covariate to adjust the group means for initial differences
- Groups start equally on the covariates
As a bonus, ANCOVA is still useful to reduce the error variance
But there are some things we have to consider! There is no free lunch
Covariate
- Predictor
- Usually continuous variable (interval level)
- It is a source of variation not controlled for in the experiment, that we believe to have an
effect on the DV
- The idea is to remove initial DV-score differences due to the covariate before running the
ANOVA. This is referred to as statistically adjusting the DV
- To be useful, the covariate should be linearly related with the DV
- Use the relationship covariate – DV to make better predictions of the means in the groups
(less bias, less uncertainty)
Adjust the group means
How ANCOVA works
ANCOVA combines regression and ANOVA
The two conceptual steps of an ANCOVA:
1. Perform a regression analysis to predict the DV using the covariate. The residuals of this analysis
are corrected values of the DV
2. Perform an ANOVA on the corrected DV (residuals) to compare the groups
- Usually it is done in one step though
ANCOVA combines both steps in one model.
You can do it manually by means of hierarchical regression:
~3~
, October 2020
Step 1: Enter the covariate: Adjusting the means
Step 2: Enter the categorical factor, grouping factor: ANOVA on adjusted means
How much does explained variance change from step 1 to step 2?
There is an F test for this, Hierarchical regression
Perform F test of increase in R2 between Step 1 and Step 2 you do the ANCOVA
Example:
Factor with two levels: Two different instruction methods
Outcome is learning measure, how much students learned
Covariate is ability test provided at the beginning to make sure that groups do not differ too much
from each other from the beginning
Running the ANOVA:
~4~
