0HV110 - BRM 3
The notes have been ordered as following: per week, then per lecturer and then counting the
lectures. So each lecturer has a lecture which number matches the week number.
Contents
Week 1...................................................................................................................................................4
Lecture 1 Snijders – Introduction lecture...........................................................................................4
Lecture 1 Lakens................................................................................................................................4
Video 1.0 – Intro............................................................................................................................4
Video 1.1 – Frequentism likelihoods Bayesian...............................................................................4
Video 1.2 – What is a p-value........................................................................................................5
Video 1.3 – Type 1 and Type 2 errors............................................................................................6
Assignment 1.1..............................................................................................................................7
Assignment 1.2 – Understanding p-values.....................................................................................8
Week 2.................................................................................................................................................10
Lecture 2 Snijders............................................................................................................................10
Part 1 of Logistic regression.........................................................................................................10
Part 2 of Logistic regression.........................................................................................................11
Hands-on lecture.........................................................................................................................12
Lecture 2 Lakens..............................................................................................................................13
Video 2.1 – Likelihoods................................................................................................................13
Video 2.2 – Binomial Bayesian Inference.....................................................................................13
Video 2.3 – Bayesian thinking......................................................................................................14
Assignment 2.1 – Likelihoods.......................................................................................................15
Assignment 2.2 – Bayesian statistics............................................................................................16
Week 3.................................................................................................................................................17
Lecture 3 Snijders............................................................................................................................17
Part 3 of Logistic regression.........................................................................................................17
Part 4 of Logistic regression.........................................................................................................17
Part 5 of Logistic regression.........................................................................................................18
Hands-on lecture.........................................................................................................................19
Lecture 3 Lakens..............................................................................................................................21
Video 3.1 – Type 1 Errors.............................................................................................................21
Video 3.2 – Type 2 Error control..................................................................................................22
Video 3.3 – Pre-registration.........................................................................................................23
Assignment 3.1 – The positive predictive value...........................................................................24
, Assignment 3.2 – Optional stopping............................................................................................24
Week 4.................................................................................................................................................26
Lecture 4 Snijders - Sneaky Stata.....................................................................................................26
Multiple regression......................................................................................................................26
Hands-on lecture Sneaky Stata....................................................................................................28
Lecture 4 Lakens..............................................................................................................................28
Video 4.1 Effect Sizes...................................................................................................................28
Video 4.2 Cohen’s d.....................................................................................................................29
Video 4.3 Correlations (r values)..................................................................................................30
Assignment 4.1 Effect sizes Cohen’s d and r................................................................................31
Assignment 4.2 Guessing the effect.............................................................................................32
Week 5.................................................................................................................................................32
Lecture 5 Snijders............................................................................................................................32
Multilevel regression part 1 + 2...................................................................................................32
Hands-on lecture on multi-level regression.................................................................................34
Lecture 5 Daniel Lakens...................................................................................................................35
Video 5.1 Confidence intervals....................................................................................................35
Video 5.2 Sample Size Justification..............................................................................................36
Video 5.3 P-curve analysis...........................................................................................................37
Assignment 5.1 – Confidence Intervals and Capture Percentages...............................................38
Assignment 5.2 Random Variation and Power Analysis...............................................................39
Week 6.................................................................................................................................................40
Lecture 6 Chris Snijders....................................................................................................................40
Multilevel regression part 3,4,5...................................................................................................40
Hands-on lecture multi-level regression part 2............................................................................42
Lecture 6 Daniel Lakens...................................................................................................................44
Video 6.1 Philosophy of Science..................................................................................................44
Video 6.2 The null is always false.................................................................................................45
Video 6.3 Theory construction.....................................................................................................46
Assignment 6.1 Equivalence testing.............................................................................................47
Week 7.................................................................................................................................................48
Lecture 7 Chris Snijders....................................................................................................................48
Exploratory factor analysis summary (by James Gaskin)..............................................................48
Factor analysis lecture part 1: introduction factor analysis.........................................................49
Factor analysis lecture part 2: extraction, rotation, calculation...................................................50
Factor analysis lecture part 3: principle component analysis vs factor analysis..........................52
, Factor analysis lecture part 4: assumptions and sample size.......................................................53
Hands-on lecture factor analysis..................................................................................................53
Week 7 Daniel Lakens......................................................................................................................55
Video 7.1 Replications..................................................................................................................55
Video 7.2 Publication bias............................................................................................................56
Video 7.3 Open science................................................................................................................57
Video 7.4 Scientific integrity........................................................................................................59
Assignment 7.2 Applied research ethics......................................................................................59
General take-aways.............................................................................................................................60
, Week 1
Lecture 1 Snijders – Introduction lecture
The homepage of the Canvas page shows the course structure.
Part 1
- Watch pre-recorded lecture before Wednesday each week
- Wednesday live lectures show how exercises should be handled
- Friday morning there is the opportunity to ask questions about all exercises
Part 2
- Nothing live
- Weekly assignment + homework
Lecture 1 Lakens
Video 1.0 – Intro
The part of Lakens in this course is aimed to improve our statistical inferences. This means confusion
is prevented and understanding of statistics is improved.
Problems in science related to statistics nowadays:
- Too small sample sizes
- Flexible analysis of data, resulting in flukes in data interpreted as true effects
- Publication bias: mainly research showing an effect is published, while research not showing
an effect is not
Video 1.1 – Frequentism likelihoods Bayesian
There are often multiple action paths in statistics to find the same result.
- Path of action:
o Use p-values to accept or reject the null hypothesis (Neyman-Pearson)
o Does not say anything about the current test, but gives more information in the long
run
- Path of knowledge: likelihoods
o Plotting the likelihood function of different hypotheses, and use this to find the
likelihood of the data
- Path of belief/ devotion: estimating the data based on prior beliefs
o Bayesian statistics
For this course, each approach can be chosen when seeming most useful, or can even be combined.
Quiz questions
1) If we reject the null hypothesis based on p < alpha, we:
a. Can be certain our conclusions is correct for the current test
b. We can’t know whether we are right or wrong in the current test, but we will not
be wrong too often over a large number of tests
c. Are making a mistake: we should have rejected the alternative hypothesis
2) Which approach allows you to incorporate your prior belief in your statistical inferences?
a. Frequentist statistics
b. Bayesian statistics
c. A likelihood approach
The notes have been ordered as following: per week, then per lecturer and then counting the
lectures. So each lecturer has a lecture which number matches the week number.
Contents
Week 1...................................................................................................................................................4
Lecture 1 Snijders – Introduction lecture...........................................................................................4
Lecture 1 Lakens................................................................................................................................4
Video 1.0 – Intro............................................................................................................................4
Video 1.1 – Frequentism likelihoods Bayesian...............................................................................4
Video 1.2 – What is a p-value........................................................................................................5
Video 1.3 – Type 1 and Type 2 errors............................................................................................6
Assignment 1.1..............................................................................................................................7
Assignment 1.2 – Understanding p-values.....................................................................................8
Week 2.................................................................................................................................................10
Lecture 2 Snijders............................................................................................................................10
Part 1 of Logistic regression.........................................................................................................10
Part 2 of Logistic regression.........................................................................................................11
Hands-on lecture.........................................................................................................................12
Lecture 2 Lakens..............................................................................................................................13
Video 2.1 – Likelihoods................................................................................................................13
Video 2.2 – Binomial Bayesian Inference.....................................................................................13
Video 2.3 – Bayesian thinking......................................................................................................14
Assignment 2.1 – Likelihoods.......................................................................................................15
Assignment 2.2 – Bayesian statistics............................................................................................16
Week 3.................................................................................................................................................17
Lecture 3 Snijders............................................................................................................................17
Part 3 of Logistic regression.........................................................................................................17
Part 4 of Logistic regression.........................................................................................................17
Part 5 of Logistic regression.........................................................................................................18
Hands-on lecture.........................................................................................................................19
Lecture 3 Lakens..............................................................................................................................21
Video 3.1 – Type 1 Errors.............................................................................................................21
Video 3.2 – Type 2 Error control..................................................................................................22
Video 3.3 – Pre-registration.........................................................................................................23
Assignment 3.1 – The positive predictive value...........................................................................24
, Assignment 3.2 – Optional stopping............................................................................................24
Week 4.................................................................................................................................................26
Lecture 4 Snijders - Sneaky Stata.....................................................................................................26
Multiple regression......................................................................................................................26
Hands-on lecture Sneaky Stata....................................................................................................28
Lecture 4 Lakens..............................................................................................................................28
Video 4.1 Effect Sizes...................................................................................................................28
Video 4.2 Cohen’s d.....................................................................................................................29
Video 4.3 Correlations (r values)..................................................................................................30
Assignment 4.1 Effect sizes Cohen’s d and r................................................................................31
Assignment 4.2 Guessing the effect.............................................................................................32
Week 5.................................................................................................................................................32
Lecture 5 Snijders............................................................................................................................32
Multilevel regression part 1 + 2...................................................................................................32
Hands-on lecture on multi-level regression.................................................................................34
Lecture 5 Daniel Lakens...................................................................................................................35
Video 5.1 Confidence intervals....................................................................................................35
Video 5.2 Sample Size Justification..............................................................................................36
Video 5.3 P-curve analysis...........................................................................................................37
Assignment 5.1 – Confidence Intervals and Capture Percentages...............................................38
Assignment 5.2 Random Variation and Power Analysis...............................................................39
Week 6.................................................................................................................................................40
Lecture 6 Chris Snijders....................................................................................................................40
Multilevel regression part 3,4,5...................................................................................................40
Hands-on lecture multi-level regression part 2............................................................................42
Lecture 6 Daniel Lakens...................................................................................................................44
Video 6.1 Philosophy of Science..................................................................................................44
Video 6.2 The null is always false.................................................................................................45
Video 6.3 Theory construction.....................................................................................................46
Assignment 6.1 Equivalence testing.............................................................................................47
Week 7.................................................................................................................................................48
Lecture 7 Chris Snijders....................................................................................................................48
Exploratory factor analysis summary (by James Gaskin)..............................................................48
Factor analysis lecture part 1: introduction factor analysis.........................................................49
Factor analysis lecture part 2: extraction, rotation, calculation...................................................50
Factor analysis lecture part 3: principle component analysis vs factor analysis..........................52
, Factor analysis lecture part 4: assumptions and sample size.......................................................53
Hands-on lecture factor analysis..................................................................................................53
Week 7 Daniel Lakens......................................................................................................................55
Video 7.1 Replications..................................................................................................................55
Video 7.2 Publication bias............................................................................................................56
Video 7.3 Open science................................................................................................................57
Video 7.4 Scientific integrity........................................................................................................59
Assignment 7.2 Applied research ethics......................................................................................59
General take-aways.............................................................................................................................60
, Week 1
Lecture 1 Snijders – Introduction lecture
The homepage of the Canvas page shows the course structure.
Part 1
- Watch pre-recorded lecture before Wednesday each week
- Wednesday live lectures show how exercises should be handled
- Friday morning there is the opportunity to ask questions about all exercises
Part 2
- Nothing live
- Weekly assignment + homework
Lecture 1 Lakens
Video 1.0 – Intro
The part of Lakens in this course is aimed to improve our statistical inferences. This means confusion
is prevented and understanding of statistics is improved.
Problems in science related to statistics nowadays:
- Too small sample sizes
- Flexible analysis of data, resulting in flukes in data interpreted as true effects
- Publication bias: mainly research showing an effect is published, while research not showing
an effect is not
Video 1.1 – Frequentism likelihoods Bayesian
There are often multiple action paths in statistics to find the same result.
- Path of action:
o Use p-values to accept or reject the null hypothesis (Neyman-Pearson)
o Does not say anything about the current test, but gives more information in the long
run
- Path of knowledge: likelihoods
o Plotting the likelihood function of different hypotheses, and use this to find the
likelihood of the data
- Path of belief/ devotion: estimating the data based on prior beliefs
o Bayesian statistics
For this course, each approach can be chosen when seeming most useful, or can even be combined.
Quiz questions
1) If we reject the null hypothesis based on p < alpha, we:
a. Can be certain our conclusions is correct for the current test
b. We can’t know whether we are right or wrong in the current test, but we will not
be wrong too often over a large number of tests
c. Are making a mistake: we should have rejected the alternative hypothesis
2) Which approach allows you to incorporate your prior belief in your statistical inferences?
a. Frequentist statistics
b. Bayesian statistics
c. A likelihood approach
