Lecture 2: Repeated-measures ANOVA (Analysis of variance – within subject)
- Design where subjects are submitted to repeated measurements
o Subjects are tested more than once, e.g., at t1, t2, and t3
OR
o Subjects are submitted to more than one treatment at once, for example
multiple experimental conditions
- Benefits
o Allows for reduction of variation between subjects and zooms in on the effect
of the treatment within subjects (i.e., within subject variation)
▪ Every participant brings their own noise
o Need fewer participants to test an effect (because you eliminate between-
subject variation)
→ gives you more statistical power
Between vs within subject design
- Between: the larger the variation within groups, the smaller the chance of a
difference between groups
- Within: variation within a group is unimportant: only variation within subjects is
important
o Only look at how participants vary between each condition
Disadvantages
- Carry-over effect
o Treatment at t1 has an effect on t2, e.g., pill given at t1 has not worn off
o Solution: enlarge interval between t1 & t2
- Test- or learning-effect
o Test results are influenced (positively/negatively) by testing itself and not by
treatment
▪ Participants get better at doing tests over time / getting sick of doing
same test over and over → get worse
o Solution:
▪ Randomize the tests (counterbalancing); randomize order of stimuli
▪ Add a control group
Methodological issues
- History: External occurrence between t1 and t2
- Maturation: people change over time
- Solution to both: control group
- Participants become aware of the manipulation
o May respond in ways that are less natural/how they think you want them to
answer
o Solution: add filler stimuli to distract them & hide manipulation from
participants
Repeated-measures: Basic Idea
- Compares 2 types of variation to test the equality of means → are the means in
different conditions equal?
- Comparison is based on ratio of variations
1
, - If the treatment variation is significantly larger than the random variation, then at
least one mean deviates from another mean
- Measures of variance are obtained by breaking down the total variance
o Only interested in variation within participants
1) Variation due to treatment: SSM
2) Random variation (random noise, e.g., distraction, tiredness): SSR
Assumptions
1) Normality
o Dependent variable(s) is/are normally distributed
o ANOVA is robust to violations
2) Homogeneity of variances (sphericity)
o Whether variance of difference between conditions is equal/ DVs have equal
variance in each condition
o ANOVA is robust to violations if n’s are equal
3) Residuals are random & independent
o Individual difference should not interact with treatment error
o Treatment effect is independent of individual differences
What if assumptions are violated?
- Most important assumption is equality of variances (sphericity); there are three tests
1) Mauchly’s test of sphericity (within variance, more than 2 levels)
o Variance of different conditions is equal
o If you only have two factors, ignore this
o Based on this test, one can conclude whether a within-subject test is
allowed (sphericity assumed)
o If violated (i.e., test is significant)→ alternative F-ratios need to be used
▪ P-values by Huyhn-Feldt, if epsilon >.75
▪ P-values by Greenhouse-Geisser, if epsilon is <.75
▪ Use multivariate tests, if sphericity is not relevant (when more than 1
DV)
2) Box’s M test (when there is more than 1 dependent variable) → not relevant for
us (only relevant when mixed design with more than one DV)
o Tests whether DVs are related to each other & different groups → want the
DV to be related in the same way in different groups
o Disadvantages → test is sensitive to violations of normality & sample size
(only nec. when 2 or more DVs)
o Ignore results of this if n is equal across groups
3) Levene’s test of equality of error variance (when there is a between subject
variable → mixed design)
o Tests equal variances across groups/error between groups should be equal
o If significant → cannot assume that variation is due to experimental design
but could be due to too much variance between groups in general → use
Dunnett’s T3, otherwise e.g., Tukey (or okay if n is qual across groups)
▪ Need to report whether it was sign. and which test you used instead
➔ mixed design: Mauchly’s and Levene’s test
➔ within design: only Mauchly’s test
2
, Output interpretation example 1 – one within subject factor
- Variable view: one column per condition, e.g., four lists that people read → four
columns
→ gives idea of what might be found in the data
▪ significant → assumption
of sphericity not met
▪ Check Epsilon value < .75
→ use F-ratio by
Greenhouse-Geisser
▪ G-G reporting: (F(1.55,
13.94) = 69.31, p < .001,
np2 = .855) → don’t forget
effect size
▪ Alternative F-ratio does
not influence what is
found
▪ This table is more
important than the
Multivariate Test table
because there is only on
DV
- Conclusion: at least one list has deviant mean percentage compared to other means
▪ table shows whether DV
can be described with
linear, quadratic, or cubic
function
▪ In this case: both linear &
cubic can describe data
▪ Only important if repeated measures are
actually repeated measured in time, or there is
an equal difference between levels
▪ shows variation in data
between participants
▪ Conclusion: participants
differ a lot → large
individual variation (error)
3
- Design where subjects are submitted to repeated measurements
o Subjects are tested more than once, e.g., at t1, t2, and t3
OR
o Subjects are submitted to more than one treatment at once, for example
multiple experimental conditions
- Benefits
o Allows for reduction of variation between subjects and zooms in on the effect
of the treatment within subjects (i.e., within subject variation)
▪ Every participant brings their own noise
o Need fewer participants to test an effect (because you eliminate between-
subject variation)
→ gives you more statistical power
Between vs within subject design
- Between: the larger the variation within groups, the smaller the chance of a
difference between groups
- Within: variation within a group is unimportant: only variation within subjects is
important
o Only look at how participants vary between each condition
Disadvantages
- Carry-over effect
o Treatment at t1 has an effect on t2, e.g., pill given at t1 has not worn off
o Solution: enlarge interval between t1 & t2
- Test- or learning-effect
o Test results are influenced (positively/negatively) by testing itself and not by
treatment
▪ Participants get better at doing tests over time / getting sick of doing
same test over and over → get worse
o Solution:
▪ Randomize the tests (counterbalancing); randomize order of stimuli
▪ Add a control group
Methodological issues
- History: External occurrence between t1 and t2
- Maturation: people change over time
- Solution to both: control group
- Participants become aware of the manipulation
o May respond in ways that are less natural/how they think you want them to
answer
o Solution: add filler stimuli to distract them & hide manipulation from
participants
Repeated-measures: Basic Idea
- Compares 2 types of variation to test the equality of means → are the means in
different conditions equal?
- Comparison is based on ratio of variations
1
, - If the treatment variation is significantly larger than the random variation, then at
least one mean deviates from another mean
- Measures of variance are obtained by breaking down the total variance
o Only interested in variation within participants
1) Variation due to treatment: SSM
2) Random variation (random noise, e.g., distraction, tiredness): SSR
Assumptions
1) Normality
o Dependent variable(s) is/are normally distributed
o ANOVA is robust to violations
2) Homogeneity of variances (sphericity)
o Whether variance of difference between conditions is equal/ DVs have equal
variance in each condition
o ANOVA is robust to violations if n’s are equal
3) Residuals are random & independent
o Individual difference should not interact with treatment error
o Treatment effect is independent of individual differences
What if assumptions are violated?
- Most important assumption is equality of variances (sphericity); there are three tests
1) Mauchly’s test of sphericity (within variance, more than 2 levels)
o Variance of different conditions is equal
o If you only have two factors, ignore this
o Based on this test, one can conclude whether a within-subject test is
allowed (sphericity assumed)
o If violated (i.e., test is significant)→ alternative F-ratios need to be used
▪ P-values by Huyhn-Feldt, if epsilon >.75
▪ P-values by Greenhouse-Geisser, if epsilon is <.75
▪ Use multivariate tests, if sphericity is not relevant (when more than 1
DV)
2) Box’s M test (when there is more than 1 dependent variable) → not relevant for
us (only relevant when mixed design with more than one DV)
o Tests whether DVs are related to each other & different groups → want the
DV to be related in the same way in different groups
o Disadvantages → test is sensitive to violations of normality & sample size
(only nec. when 2 or more DVs)
o Ignore results of this if n is equal across groups
3) Levene’s test of equality of error variance (when there is a between subject
variable → mixed design)
o Tests equal variances across groups/error between groups should be equal
o If significant → cannot assume that variation is due to experimental design
but could be due to too much variance between groups in general → use
Dunnett’s T3, otherwise e.g., Tukey (or okay if n is qual across groups)
▪ Need to report whether it was sign. and which test you used instead
➔ mixed design: Mauchly’s and Levene’s test
➔ within design: only Mauchly’s test
2
, Output interpretation example 1 – one within subject factor
- Variable view: one column per condition, e.g., four lists that people read → four
columns
→ gives idea of what might be found in the data
▪ significant → assumption
of sphericity not met
▪ Check Epsilon value < .75
→ use F-ratio by
Greenhouse-Geisser
▪ G-G reporting: (F(1.55,
13.94) = 69.31, p < .001,
np2 = .855) → don’t forget
effect size
▪ Alternative F-ratio does
not influence what is
found
▪ This table is more
important than the
Multivariate Test table
because there is only on
DV
- Conclusion: at least one list has deviant mean percentage compared to other means
▪ table shows whether DV
can be described with
linear, quadratic, or cubic
function
▪ In this case: both linear &
cubic can describe data
▪ Only important if repeated measures are
actually repeated measured in time, or there is
an equal difference between levels
▪ shows variation in data
between participants
▪ Conclusion: participants
differ a lot → large
individual variation (error)
3
