Repeated Measures lecture notes
Lecture 1 Review of ANOVA
Univariate = 1 dependent variable (DV)
Multivariate = multiple DVs
Lectures are most important! Background is in book, still important.
Recall ANOVA
Between-factor one-way ANOVA:
Purpose: Comparison of group means (independent populations).
Factor, e.g., gender, for females and males.
One-way means 1 factor like gender, or intervention (group with intervention, and group without), or
educational level with three levels (low, average, high).
→ two way is with two factors, e.g., gender and educational level in the design. A participant is
always put in a group. Between subject-variable, e.g., you a female of male.
Within-variable: pops up in different moments/categories, e.g., within factor is time, before and after
treatment.
To wat extent do the means differ, e.g., between high and low education.
µj = population mean of the group
= subject-specific residual
SS = the variability in sum of scores.
SS partition: SST = SSG + SSE
SSG – between groups, explained part
SSE – within groups, unexplained part
F = MSG/MSE =
95% confidence interval (CI) = 95% sure that the population mean will fall between the sample mean
and 95% CI interval.
SS/df = means square (MS)
,F = mean square / residual
Example one-way ANOVA
- Study on the effects of instructional material on how well students learn statistical concepts.
- Variables:
o DV continuous: Y (test scores on statistical concepts)
o IV discrete: group (2) (instructional conditions)
- Perform an univariate ANOVA:
o Test whether the two population means are equal
o ANOVA table:
SS, df, MS, F, p-value, Partial eta squared (.01: Small; .06: Medium; .14: Large effect
size)
Samples scores on Y per group + output
Significance test and effect size
p > .05 HO = not rejected, no significant difference.
Small sample = lower power, could give larger effect size
- P-value: indicates the significance of a factor.
o What is the probability of these samples means or more extreme if the population
means would be equal in the population?
- Effect size: indicates the size of the effect
o In ANOVA: How large is the difference between the groups in the population?
o Population means relative to within group variable. How much do groups differ from
each other? The further apart the normal distributions are, the bigger the effect size.
o Effect size measures in ANOVA
▪ ɳ2 = SSeffect/SStotal: proportion of variance explained of effect
▪ Partial ɳ2: proportion of variance explained, after accounting for variance
explained by possible other factors
▪ And other measures
,Follow-up on significant ANOVA
What to do if the omnibus F test rejects H0?
- Evidence that at least 1 group differs from the other groups, based on one or more effects
(main/interaction). One group significantly differs, where is the difference?
Via:
o Visual inspection
o (Muliple) comparisons (tests or CI’s)
1. Planned → contrasts
2. Post hoc comparisons
Assumptions ANOVA
1. Independent observations
2. Within each group the scores are normally distributed
a. Check per group via QQ-plot or test on skewness and kurtosis
3. The variances of the scores are equal across all groups
a. Check sample variances between groups: max/min <2 is ok
b. Levene’s test: be cautious, use of significant test to confirm H0. → quite dangerous
Experimental designs
Experiments have 3 characteristics:
1. Manipulation of treatment levels:
– researcher controls nature and timing of each treatment level
2. Random assignment of cases to levels (groups):
– to remove bias
– average out differences among cases
3. Control of extraneous variables:
– only treatment level changes during experiment
Observational: apparently groups differ from each other.
Experimental: you can infer causality.
How to control extraneous variables:
- Hold them constant
- Counter effect their effects
- Turn them into an extra factor
When all 3 characteristics hold (i.e., manipulation, random assignment, control), differences in scores
are attributed to differences in treatment levels.
Proof of causal relationship? → still hazardous until study is successfully replicated
, Between subject design
Differences due to treatments are tested between groups of subjects: Different cases in every level.
Designs:
- Experimental: Cases are randomly assigned to
treatment levels
- Nonexperimental (also denoted: correlational
or observational): No random assignment
(e.g., gender; patient/control)
- Factorial designs:
o Treatment levels are determined by
more than one factor
o Main effects of each factor, and interaction(s)
Factorial ANOVA
- Usually more than one factor (defining different groups)
o For two factors: then a x b groups, and main effects and interaction effects can be
tested. → is denoted: two-way ANOVA.
▪ Main effects are best interpreted when there is NO interactions between
variables.
- Why several factors?
o Statistical reason: Reduction of error variance
o Substantive reason: Study interplay between variables
Source of variance
Identifying source of variance
1. List each factor as source
2. Examine each combination of factors: complete crossed → include interactions as source
3. When effect is repeated, with different instances, at every level or another factor → include
factor as source
Main effects are best interpreted when there is NO interaction between variables.
Example:
- Factor A, factor B, and subjects S
- A and B completely crossed: A, B, AB, and S
- Different S, at each level of A and B: A, B, AB, and S(AB)
Lecture 1 Review of ANOVA
Univariate = 1 dependent variable (DV)
Multivariate = multiple DVs
Lectures are most important! Background is in book, still important.
Recall ANOVA
Between-factor one-way ANOVA:
Purpose: Comparison of group means (independent populations).
Factor, e.g., gender, for females and males.
One-way means 1 factor like gender, or intervention (group with intervention, and group without), or
educational level with three levels (low, average, high).
→ two way is with two factors, e.g., gender and educational level in the design. A participant is
always put in a group. Between subject-variable, e.g., you a female of male.
Within-variable: pops up in different moments/categories, e.g., within factor is time, before and after
treatment.
To wat extent do the means differ, e.g., between high and low education.
µj = population mean of the group
= subject-specific residual
SS = the variability in sum of scores.
SS partition: SST = SSG + SSE
SSG – between groups, explained part
SSE – within groups, unexplained part
F = MSG/MSE =
95% confidence interval (CI) = 95% sure that the population mean will fall between the sample mean
and 95% CI interval.
SS/df = means square (MS)
,F = mean square / residual
Example one-way ANOVA
- Study on the effects of instructional material on how well students learn statistical concepts.
- Variables:
o DV continuous: Y (test scores on statistical concepts)
o IV discrete: group (2) (instructional conditions)
- Perform an univariate ANOVA:
o Test whether the two population means are equal
o ANOVA table:
SS, df, MS, F, p-value, Partial eta squared (.01: Small; .06: Medium; .14: Large effect
size)
Samples scores on Y per group + output
Significance test and effect size
p > .05 HO = not rejected, no significant difference.
Small sample = lower power, could give larger effect size
- P-value: indicates the significance of a factor.
o What is the probability of these samples means or more extreme if the population
means would be equal in the population?
- Effect size: indicates the size of the effect
o In ANOVA: How large is the difference between the groups in the population?
o Population means relative to within group variable. How much do groups differ from
each other? The further apart the normal distributions are, the bigger the effect size.
o Effect size measures in ANOVA
▪ ɳ2 = SSeffect/SStotal: proportion of variance explained of effect
▪ Partial ɳ2: proportion of variance explained, after accounting for variance
explained by possible other factors
▪ And other measures
,Follow-up on significant ANOVA
What to do if the omnibus F test rejects H0?
- Evidence that at least 1 group differs from the other groups, based on one or more effects
(main/interaction). One group significantly differs, where is the difference?
Via:
o Visual inspection
o (Muliple) comparisons (tests or CI’s)
1. Planned → contrasts
2. Post hoc comparisons
Assumptions ANOVA
1. Independent observations
2. Within each group the scores are normally distributed
a. Check per group via QQ-plot or test on skewness and kurtosis
3. The variances of the scores are equal across all groups
a. Check sample variances between groups: max/min <2 is ok
b. Levene’s test: be cautious, use of significant test to confirm H0. → quite dangerous
Experimental designs
Experiments have 3 characteristics:
1. Manipulation of treatment levels:
– researcher controls nature and timing of each treatment level
2. Random assignment of cases to levels (groups):
– to remove bias
– average out differences among cases
3. Control of extraneous variables:
– only treatment level changes during experiment
Observational: apparently groups differ from each other.
Experimental: you can infer causality.
How to control extraneous variables:
- Hold them constant
- Counter effect their effects
- Turn them into an extra factor
When all 3 characteristics hold (i.e., manipulation, random assignment, control), differences in scores
are attributed to differences in treatment levels.
Proof of causal relationship? → still hazardous until study is successfully replicated
, Between subject design
Differences due to treatments are tested between groups of subjects: Different cases in every level.
Designs:
- Experimental: Cases are randomly assigned to
treatment levels
- Nonexperimental (also denoted: correlational
or observational): No random assignment
(e.g., gender; patient/control)
- Factorial designs:
o Treatment levels are determined by
more than one factor
o Main effects of each factor, and interaction(s)
Factorial ANOVA
- Usually more than one factor (defining different groups)
o For two factors: then a x b groups, and main effects and interaction effects can be
tested. → is denoted: two-way ANOVA.
▪ Main effects are best interpreted when there is NO interactions between
variables.
- Why several factors?
o Statistical reason: Reduction of error variance
o Substantive reason: Study interplay between variables
Source of variance
Identifying source of variance
1. List each factor as source
2. Examine each combination of factors: complete crossed → include interactions as source
3. When effect is repeated, with different instances, at every level or another factor → include
factor as source
Main effects are best interpreted when there is NO interaction between variables.
Example:
- Factor A, factor B, and subjects S
- A and B completely crossed: A, B, AB, and S
- Different S, at each level of A and B: A, B, AB, and S(AB)
