Research Methods and Techniques II:
quantitative methods
Prof. Tim Vantilborgh
2023-2024
Amandine Vico
1 Amandine Vico
,Inhoudstafel
1. Introduction to scientific reasoning ................................................................................................ 7
1.1 Psychology is a way of thinking ............................................................................................. 7
1.1.1 Producing versus consuming research .............................................................................. 7
1.1.2 How scientists work........................................................................................................... 7
1.2 Sources of information ........................................................................................................ 11
1.2.1 Research versus experience ............................................................................................ 12
1.2.2 Research versus intuition ................................................................................................ 13
1.2.3 Authority figures ............................................................................................................. 14
1.2.4 Overview ......................................................................................................................... 14
1.2.5 Scientific sources ............................................................................................................. 14
2. Foundations of scientific research ................................................................................................ 16
2.1 Three claims, four validities ................................................................................................. 16
2.1.1 Variables .......................................................................................................................... 16
2.1.2 Three claims .................................................................................................................... 17
2.1.3 The four big validities ...................................................................................................... 20
2.1.4 Conditions for causality ................................................................................................... 21
2.1.5 Prioritizing types of validity ............................................................................................. 22
2.2 Ethical guidelines for psychology research .......................................................................... 23
2.2.1 Historical examples ......................................................................................................... 23
2.2.2 The Belmont report ......................................................................................................... 23
2.2.3 APA guidelines ................................................................................................................. 24
2.2.8 Research fraud ..................................................................................................................... 25
2.2.9 Questionable research practices .......................................................................................... 25
2.2.10 Reasons and solutions ........................................................................................................ 26
2.2.11 Open science ...................................................................................................................... 27
2.2.12 Animal research ................................................................................................................. 27
3. Survey research ................................................................................................................................. 28
3.1 Identifying good measurement ................................................................................................... 28
3.1.1 Different ways to measure a variable................................................................................... 28
3.1.2 Operationalizing ................................................................................................................... 28
3.1.3 Construct validity and reliability .......................................................................................... 29
3.1.4 Evaluating reliability and construct validity articles ............................................................. 31
3.2 Surveys and observations ........................................................................................................... 33
3.2.1 Surveys ................................................................................................................................. 33
3.2.2 Experience sampling methods ............................................................................................. 33
2 Amandine Vico
, 3.2.3 Single-measures versus multi-item measures ..................................................................... 33
3.2.4 Scales.................................................................................................................................... 33
3.2.5 Observations ........................................................................................................................ 37
4. Sampling ........................................................................................................................................ 39
4.1 Some key terms ........................................................................................................................... 39
4.2 Key issue ...................................................................................................................................... 39
4.3 Illustration ................................................................................................................................... 40
4.4 Sampling bias .............................................................................................................................. 40
4.5 Margin-of-error ........................................................................................................................... 40
4.6 Sampling techniques ................................................................................................................... 41
4.6.1 Biased vs unbiased sampling techniques ............................................................................. 41
4.6.2 Probability sampling............................................................................................................. 42
4.6.3 Biased sampling ................................................................................................................... 43
4.6.4 Quota sampling .................................................................................................................... 45
4.7 Why is external validity important? ............................................................................................ 45
4.8 How to evaluate the external validity ......................................................................................... 45
4.9 WEIRD samples ........................................................................................................................... 46
4.10 Online paid panels .................................................................................................................... 46
4.11 Sample size ................................................................................................................................ 47
4.12 Response rate............................................................................................................................ 47
5. Bivariate correlational research .................................................................................................... 49
5.1 Three examples.................................................................................................................... 49
5.1.1 Mehl et al. (2010): eavesdropping on happiness ............................................................ 49
5.1.2 Cacioppo et al. (2013) ..................................................................................................... 50
5.1.3 Siddarth et al. (2018) ....................................................................................................... 51
5.2 Definition ............................................................................................................................. 51
5.3 Evaluating validity ................................................................................................................ 51
5.3.1 Construct validity ............................................................................................................ 51
5.3.2 Statistical validity ............................................................................................................. 52
5.3.3 Internal validity................................................................................................................ 55
5.3.4 External validity ............................................................................................................... 57
5.4 Moderator ........................................................................................................................... 57
6. Multivariate correlational research............................................................................................... 59
6.1 Criteria for causality............................................................................................................. 59
6.2 Longitudinal designs ............................................................................................................ 59
6.2.1 Exercise............................................................................................................................ 61
3 Amandine Vico
, 6.3 Multiple regression .............................................................................................................. 62
6.3.1 Rule out third variables ................................................................................................... 63
6.3.2 Patterns and parsimony................................................................................................... 67
6.4 Mediation ............................................................................................................................ 68
6.4.1 Mediator versus confounder ........................................................................................... 68
6.4.2 Mediator versus moderator ............................................................................................ 69
6.5 Controlling for confounders................................................................................................. 70
7. Introduction to simple experimental research ............................................................................. 71
7.1 Two examples ...................................................................................................................... 71
7.1.1 Mueller & Oppenheimer (2014)...................................................................................... 71
7.1.2 Van Kleef, Shimizu, & Wansink (2012) ............................................................................. 71
7.2 Experimental variables ........................................................................................................ 72
7.3 Simple experiment............................................................................................................... 72
7.3.1 Example in class............................................................................................................... 72
7.4 Experiments ......................................................................................................................... 73
7.4.1 Causality .......................................................................................................................... 73
7.4.2 Threats to internal validity .............................................................................................. 73
7.4.3 Experimental designs ...................................................................................................... 75
7.4.4 Advantages within-subjects designs ............................................................................... 77
7.4.5 Threats internal validity with within-subject designs...................................................... 77
7.4.6 Counterbalancing ............................................................................................................ 78
7.4.7 Downsides within-subjects designs................................................................................. 78
7.4.8 Is a pretest/posttest design a within-subjects design? NO ............................................. 79
7.5 Evaluating for aspects of validity ......................................................................................... 79
7.5.1 Illustration: Mueller & Oppenheimer (2014) .................................................................. 79
8. Experiments - Confounding and obscuring variables.................................................................... 82
8.1 Examples of bad experiments.............................................................................................. 82
8.1.1 Example 1 ........................................................................................................................ 82
8.1.2 Example 2 ........................................................................................................................ 82
8.1.3 Example 3 ........................................................................................................................ 82
8.2 Threats to internal validity................................................................................................... 83
8.2.1 Maturation ...................................................................................................................... 83
8.2.2 history threat ................................................................................................................... 83
8.2.3 regression to the mean ................................................................................................... 83
8.2.4 attrition............................................................................................................................ 84
8.2.5 testing .............................................................................................................................. 84
4 Amandine Vico
quantitative methods
Prof. Tim Vantilborgh
2023-2024
Amandine Vico
1 Amandine Vico
,Inhoudstafel
1. Introduction to scientific reasoning ................................................................................................ 7
1.1 Psychology is a way of thinking ............................................................................................. 7
1.1.1 Producing versus consuming research .............................................................................. 7
1.1.2 How scientists work........................................................................................................... 7
1.2 Sources of information ........................................................................................................ 11
1.2.1 Research versus experience ............................................................................................ 12
1.2.2 Research versus intuition ................................................................................................ 13
1.2.3 Authority figures ............................................................................................................. 14
1.2.4 Overview ......................................................................................................................... 14
1.2.5 Scientific sources ............................................................................................................. 14
2. Foundations of scientific research ................................................................................................ 16
2.1 Three claims, four validities ................................................................................................. 16
2.1.1 Variables .......................................................................................................................... 16
2.1.2 Three claims .................................................................................................................... 17
2.1.3 The four big validities ...................................................................................................... 20
2.1.4 Conditions for causality ................................................................................................... 21
2.1.5 Prioritizing types of validity ............................................................................................. 22
2.2 Ethical guidelines for psychology research .......................................................................... 23
2.2.1 Historical examples ......................................................................................................... 23
2.2.2 The Belmont report ......................................................................................................... 23
2.2.3 APA guidelines ................................................................................................................. 24
2.2.8 Research fraud ..................................................................................................................... 25
2.2.9 Questionable research practices .......................................................................................... 25
2.2.10 Reasons and solutions ........................................................................................................ 26
2.2.11 Open science ...................................................................................................................... 27
2.2.12 Animal research ................................................................................................................. 27
3. Survey research ................................................................................................................................. 28
3.1 Identifying good measurement ................................................................................................... 28
3.1.1 Different ways to measure a variable................................................................................... 28
3.1.2 Operationalizing ................................................................................................................... 28
3.1.3 Construct validity and reliability .......................................................................................... 29
3.1.4 Evaluating reliability and construct validity articles ............................................................. 31
3.2 Surveys and observations ........................................................................................................... 33
3.2.1 Surveys ................................................................................................................................. 33
3.2.2 Experience sampling methods ............................................................................................. 33
2 Amandine Vico
, 3.2.3 Single-measures versus multi-item measures ..................................................................... 33
3.2.4 Scales.................................................................................................................................... 33
3.2.5 Observations ........................................................................................................................ 37
4. Sampling ........................................................................................................................................ 39
4.1 Some key terms ........................................................................................................................... 39
4.2 Key issue ...................................................................................................................................... 39
4.3 Illustration ................................................................................................................................... 40
4.4 Sampling bias .............................................................................................................................. 40
4.5 Margin-of-error ........................................................................................................................... 40
4.6 Sampling techniques ................................................................................................................... 41
4.6.1 Biased vs unbiased sampling techniques ............................................................................. 41
4.6.2 Probability sampling............................................................................................................. 42
4.6.3 Biased sampling ................................................................................................................... 43
4.6.4 Quota sampling .................................................................................................................... 45
4.7 Why is external validity important? ............................................................................................ 45
4.8 How to evaluate the external validity ......................................................................................... 45
4.9 WEIRD samples ........................................................................................................................... 46
4.10 Online paid panels .................................................................................................................... 46
4.11 Sample size ................................................................................................................................ 47
4.12 Response rate............................................................................................................................ 47
5. Bivariate correlational research .................................................................................................... 49
5.1 Three examples.................................................................................................................... 49
5.1.1 Mehl et al. (2010): eavesdropping on happiness ............................................................ 49
5.1.2 Cacioppo et al. (2013) ..................................................................................................... 50
5.1.3 Siddarth et al. (2018) ....................................................................................................... 51
5.2 Definition ............................................................................................................................. 51
5.3 Evaluating validity ................................................................................................................ 51
5.3.1 Construct validity ............................................................................................................ 51
5.3.2 Statistical validity ............................................................................................................. 52
5.3.3 Internal validity................................................................................................................ 55
5.3.4 External validity ............................................................................................................... 57
5.4 Moderator ........................................................................................................................... 57
6. Multivariate correlational research............................................................................................... 59
6.1 Criteria for causality............................................................................................................. 59
6.2 Longitudinal designs ............................................................................................................ 59
6.2.1 Exercise............................................................................................................................ 61
3 Amandine Vico
, 6.3 Multiple regression .............................................................................................................. 62
6.3.1 Rule out third variables ................................................................................................... 63
6.3.2 Patterns and parsimony................................................................................................... 67
6.4 Mediation ............................................................................................................................ 68
6.4.1 Mediator versus confounder ........................................................................................... 68
6.4.2 Mediator versus moderator ............................................................................................ 69
6.5 Controlling for confounders................................................................................................. 70
7. Introduction to simple experimental research ............................................................................. 71
7.1 Two examples ...................................................................................................................... 71
7.1.1 Mueller & Oppenheimer (2014)...................................................................................... 71
7.1.2 Van Kleef, Shimizu, & Wansink (2012) ............................................................................. 71
7.2 Experimental variables ........................................................................................................ 72
7.3 Simple experiment............................................................................................................... 72
7.3.1 Example in class............................................................................................................... 72
7.4 Experiments ......................................................................................................................... 73
7.4.1 Causality .......................................................................................................................... 73
7.4.2 Threats to internal validity .............................................................................................. 73
7.4.3 Experimental designs ...................................................................................................... 75
7.4.4 Advantages within-subjects designs ............................................................................... 77
7.4.5 Threats internal validity with within-subject designs...................................................... 77
7.4.6 Counterbalancing ............................................................................................................ 78
7.4.7 Downsides within-subjects designs................................................................................. 78
7.4.8 Is a pretest/posttest design a within-subjects design? NO ............................................. 79
7.5 Evaluating for aspects of validity ......................................................................................... 79
7.5.1 Illustration: Mueller & Oppenheimer (2014) .................................................................. 79
8. Experiments - Confounding and obscuring variables.................................................................... 82
8.1 Examples of bad experiments.............................................................................................. 82
8.1.1 Example 1 ........................................................................................................................ 82
8.1.2 Example 2 ........................................................................................................................ 82
8.1.3 Example 3 ........................................................................................................................ 82
8.2 Threats to internal validity................................................................................................... 83
8.2.1 Maturation ...................................................................................................................... 83
8.2.2 history threat ................................................................................................................... 83
8.2.3 regression to the mean ................................................................................................... 83
8.2.4 attrition............................................................................................................................ 84
8.2.5 testing .............................................................................................................................. 84
4 Amandine Vico
