0HM120 – Advanced Data Analysis
Complete file: theory
Content
General notes
Significance with different tests
Link research question to certain test
Lecture 1: Descriptive and inferential statistics
Describing a population
Estimating a population mean
Hypothesis testing
Examples
Effect size
Confidence intervals
The counter null-hypothesis
Power
Determining the minimal effect size
Lecture 2: Introduction to ANOVA
One-way between subjects ANOVA
Assumptions
Post-hoc testing
Two-way ANOVA
Hypothesis testing
The difference of difference interpretation
Post-hoc testing
Correlated factors
Factorial ANOVA
Post-hoc comparisons
Lecture 3: Repeated and mixed ANOVA
Repeated measures ANOVA the factorial case
Assumptions
Post-hoc comparisons
Mixed ANOVA
Assumptions
Lecture 4: Introduction to regression
Population covariance
Assumptions
Limitations
Partial and semi partial correlation
Simple regression – Continuous predictor
Assumptions
Outliers
Simple regression – Categorical predictor
Multiple regression – Continuous predictors
Unique variance
Multicollinearity
Multiple regression – Categorical and Continuous predictors
Lecture 5: Moderated regression
1
, Moderated multiple regression – Continuous moderator
Moderated multiple regression – Categorical moderator
Moderated multiple regression – Continuous moderator with centering
Post-hoc testing
Assumptions
Lecture 6: Introduction to ANCOVA
Recap ANOVA
Traditional ANCOVA
Increasing power of F-test
Controlling for systematic bias
Assumptions
Modern ANCOVA
Assumptions
Intact groups
Lecture 7: Advanced topics in ANCOVA
Similarities and differences between methods
AN(C)OVA vs (Moderated) regression
Lord’s paradox
Lecture 8: Introduction to contrast analysis
2x4 between subjects design
Post-hoc testing
ANOVA vs Contrast analysis
Contrast analysis – one-way between-subjects design
Contrast analysis – between-subjects design
Contrast analysis – Membership defined by multiple factors
Lecture 9: Contrast analysis for repeated and mixed designs
Contrast analysis – Repeated measures design
Contrast analysis – Mixed design
Lecture 10: Mediation analysis
Mediation analysis – One mediator
Different types of mediation
Multicollinearity
Mediation analysis – Multiple mediators
Lecture 11: Moderated mediation analysis
Moderated mediation analysis
Post-hoc testing
Lecture 12: Meta analytical thinking
Mixed results
Introduction to meta-analysis
Bias detection
2
,General notes
Significance with different tests
H0
P < 0.05 statistically significant, reject H0, difference found
P > 0.05 not statistically significant, accept H0, no difference found
Homogeneity of variance
P < 0.05 statistically significant, reject H0, variances are not equal, heterogeneity
P > 0.05 not statistically significant, accept H0, variances are equal, homogeneity
Normality – sktest and swilk
P < 0.05 statistically significant, reject H0, not normally distributed
P > 0.05 not statistically significant, accept H0, normally distributed
ANCOVA assumption: homogeneity of regression slopes, testing interaction (factor*covariate)
P < 0.05 statistically significant, slopes are not the same, heterogeneity
P > 0.05 not statistically significant, slopes are the same, homogeneity of regression slopes
F (dfvariable, dferror) = …. with p = … which is < / > 0, so we can accept/reject H0 that states…
Overview: link research question to a certain test
Comparing your sample to a number.
e.g. Is the average height of adult Dutch males 173 cm?
→ one sample t-test
Comparing two means with each other
e.g. Do people who get drug A recover faster than people who get drug B?
→ independent samples t-test
Does independent variable X affect dependent variable Y? (causation) Answer questions about
differences between groups, or the effect of the factors that define group membership.
e.g. does manipulation of seating location affect educational performance?
Group membership is defined by a single factor.
→ one-way between subjects ANOVA
Each participant is a member of all groups (all group consist of the same participants)
→ one-way within subjects (repeated measures) ANOVA
Do independent variable X1 and independent variable X2 influence dependent variable Y?
e.g. How do exercise (2 categories) and medication (3 categories) affect weight loss? 2x3 design with
first order interactions. If you have a 2x2x3 design, you also have second order interactions.
Group membership is defined by multiple factors.
→ factorial between subjects ANOVA (two-way ANOVA)
3
, Each participant is a member of all groups (all group consist of the same participants)
→ factorial within subjects (repeated measures) ANOVA
When participants are member of some of the groups. Both between measurements and
within (repeated measurements)
→ Mixed ANOVA
Predicting (estimate) dependent variable Y by means of (on the basis of) a single predictor X. How is
X related to Y?
→Simple regression
Predicting dependent variable Y by means of predictor X1 and predictor X2. (more than one
predictor)
→multiple regression
What is the correlation between variable X1 and variable Y when the variation in Y due to variable X2
is removed?
→Partial correlation
What is the correlation between variable X1 and variable Y when the variation in Y due to variable X2
is ignored? What is the unique contributesion of X1 to the explanation of variance in Y?
→semipartial (part) correlation
Test wheter the effect of a predictor X1 on the outcome of variable Y is dependent on another
predictor X2.
→moderated multiple regression
The effect of variables X1 and X2 on Y, whilst correcting for a confound/covariate (not controlled or
randomly assigned, but measured, that correlates with the dependent variable).
Partial out the covariate
→traditional ANCOVA (fisher)
Include the covariate as a continuous predictor to the model
→modern ANCOVA
Test a specific, clear theory-driven explanation/expectation/prediction directly against the empirical
data.
→contrast analysis
Can mediator M explain an observed effect of the predictor X on the dependent variable Y?
e.g. determine the effect of city size on the crime rate through the degree of anonymity.
→mediation with one single mediator
Or with multiple Ms
→mediation with multiple mediators
The moderator Z moderates both the direct effect of X on Y as the indirect effect of X on Y through
the mediator M.
→Moderated mediation analysis
4
Complete file: theory
Content
General notes
Significance with different tests
Link research question to certain test
Lecture 1: Descriptive and inferential statistics
Describing a population
Estimating a population mean
Hypothesis testing
Examples
Effect size
Confidence intervals
The counter null-hypothesis
Power
Determining the minimal effect size
Lecture 2: Introduction to ANOVA
One-way between subjects ANOVA
Assumptions
Post-hoc testing
Two-way ANOVA
Hypothesis testing
The difference of difference interpretation
Post-hoc testing
Correlated factors
Factorial ANOVA
Post-hoc comparisons
Lecture 3: Repeated and mixed ANOVA
Repeated measures ANOVA the factorial case
Assumptions
Post-hoc comparisons
Mixed ANOVA
Assumptions
Lecture 4: Introduction to regression
Population covariance
Assumptions
Limitations
Partial and semi partial correlation
Simple regression – Continuous predictor
Assumptions
Outliers
Simple regression – Categorical predictor
Multiple regression – Continuous predictors
Unique variance
Multicollinearity
Multiple regression – Categorical and Continuous predictors
Lecture 5: Moderated regression
1
, Moderated multiple regression – Continuous moderator
Moderated multiple regression – Categorical moderator
Moderated multiple regression – Continuous moderator with centering
Post-hoc testing
Assumptions
Lecture 6: Introduction to ANCOVA
Recap ANOVA
Traditional ANCOVA
Increasing power of F-test
Controlling for systematic bias
Assumptions
Modern ANCOVA
Assumptions
Intact groups
Lecture 7: Advanced topics in ANCOVA
Similarities and differences between methods
AN(C)OVA vs (Moderated) regression
Lord’s paradox
Lecture 8: Introduction to contrast analysis
2x4 between subjects design
Post-hoc testing
ANOVA vs Contrast analysis
Contrast analysis – one-way between-subjects design
Contrast analysis – between-subjects design
Contrast analysis – Membership defined by multiple factors
Lecture 9: Contrast analysis for repeated and mixed designs
Contrast analysis – Repeated measures design
Contrast analysis – Mixed design
Lecture 10: Mediation analysis
Mediation analysis – One mediator
Different types of mediation
Multicollinearity
Mediation analysis – Multiple mediators
Lecture 11: Moderated mediation analysis
Moderated mediation analysis
Post-hoc testing
Lecture 12: Meta analytical thinking
Mixed results
Introduction to meta-analysis
Bias detection
2
,General notes
Significance with different tests
H0
P < 0.05 statistically significant, reject H0, difference found
P > 0.05 not statistically significant, accept H0, no difference found
Homogeneity of variance
P < 0.05 statistically significant, reject H0, variances are not equal, heterogeneity
P > 0.05 not statistically significant, accept H0, variances are equal, homogeneity
Normality – sktest and swilk
P < 0.05 statistically significant, reject H0, not normally distributed
P > 0.05 not statistically significant, accept H0, normally distributed
ANCOVA assumption: homogeneity of regression slopes, testing interaction (factor*covariate)
P < 0.05 statistically significant, slopes are not the same, heterogeneity
P > 0.05 not statistically significant, slopes are the same, homogeneity of regression slopes
F (dfvariable, dferror) = …. with p = … which is < / > 0, so we can accept/reject H0 that states…
Overview: link research question to a certain test
Comparing your sample to a number.
e.g. Is the average height of adult Dutch males 173 cm?
→ one sample t-test
Comparing two means with each other
e.g. Do people who get drug A recover faster than people who get drug B?
→ independent samples t-test
Does independent variable X affect dependent variable Y? (causation) Answer questions about
differences between groups, or the effect of the factors that define group membership.
e.g. does manipulation of seating location affect educational performance?
Group membership is defined by a single factor.
→ one-way between subjects ANOVA
Each participant is a member of all groups (all group consist of the same participants)
→ one-way within subjects (repeated measures) ANOVA
Do independent variable X1 and independent variable X2 influence dependent variable Y?
e.g. How do exercise (2 categories) and medication (3 categories) affect weight loss? 2x3 design with
first order interactions. If you have a 2x2x3 design, you also have second order interactions.
Group membership is defined by multiple factors.
→ factorial between subjects ANOVA (two-way ANOVA)
3
, Each participant is a member of all groups (all group consist of the same participants)
→ factorial within subjects (repeated measures) ANOVA
When participants are member of some of the groups. Both between measurements and
within (repeated measurements)
→ Mixed ANOVA
Predicting (estimate) dependent variable Y by means of (on the basis of) a single predictor X. How is
X related to Y?
→Simple regression
Predicting dependent variable Y by means of predictor X1 and predictor X2. (more than one
predictor)
→multiple regression
What is the correlation between variable X1 and variable Y when the variation in Y due to variable X2
is removed?
→Partial correlation
What is the correlation between variable X1 and variable Y when the variation in Y due to variable X2
is ignored? What is the unique contributesion of X1 to the explanation of variance in Y?
→semipartial (part) correlation
Test wheter the effect of a predictor X1 on the outcome of variable Y is dependent on another
predictor X2.
→moderated multiple regression
The effect of variables X1 and X2 on Y, whilst correcting for a confound/covariate (not controlled or
randomly assigned, but measured, that correlates with the dependent variable).
Partial out the covariate
→traditional ANCOVA (fisher)
Include the covariate as a continuous predictor to the model
→modern ANCOVA
Test a specific, clear theory-driven explanation/expectation/prediction directly against the empirical
data.
→contrast analysis
Can mediator M explain an observed effect of the predictor X on the dependent variable Y?
e.g. determine the effect of city size on the crime rate through the degree of anonymity.
→mediation with one single mediator
Or with multiple Ms
→mediation with multiple mediators
The moderator Z moderates both the direct effect of X on Y as the indirect effect of X on Y through
the mediator M.
→Moderated mediation analysis
4
