Table of Contents
1 Research Methods and Techniques – Practical Information................7
1.1 Language of Instruction.............................................................7
1.2 Course Modules.........................................................................8
1.3 Instructor Team.........................................................................8
1.4 Lecture Schedule & Topics..........................................................8
1.5 Participation & Group Registration.............................................8
1.6 Practical Sessions......................................................................8
1.7 Required Materials.....................................................................9
1.8 Assessment & Evaluation...........................................................9
2 Psychology Is a Way of Thinking.....................................................10
2.1 Producing Versus Consuming Research.....................................10
2.2 How Scientists Work................................................................11
2.2.1 Key Principles of Scientific Practice..................................................................11
2.3 Empiricism...............................................................................12
2.4 Scientists Test Theories...........................................................12
2.4.1 Theory-Data Cycle...........................................................................................12
2.4.2 Exploratory (Inductive) vs. Confirmatory (Deductive) Research......................15
2.4.3 Characteristics of Good Theories.....................................................................15
2.5 Scientists Form a Community....................................................15
2.6 Fundamental Versus Applied Research......................................16
2.7 Continuously Evolving Science..................................................16
2.8 Publishing in Science................................................................16
3 Sources of Information..................................................................17
3.1 Researchers and Their Methods................................................18
3.2 Research Versus Experience.....................................................18
3.3 Research Versus Intuition.........................................................19
3.4 Authority Figures.....................................................................19
3.5 Scientific Sources.....................................................................19
3.5.1 Breakdown of the Diagram:.............................................................................20
3.6 Hierarchy of Evidence...............................................................22
4 Three Claims, Four Validities..........................................................23
4.1 Variables: Descriptions and Definitions.....................................23
4.2 The Three Types of Research Claims.........................................24
4.2.1 Frequency Claims............................................................................................ 24
4.2.2 Association Claims........................................................................................... 24
4.2.3 Causal Claims..................................................................................................25
4.3 The Four Big Validities.............................................................25
4.4 Applying Validity to Different Research Claims..........................26
pg. 1
, 4.5 Conditions for Causality...........................................................27
4.6 Spurious Associations and Balancing Validity............................28
4.7 correllation does not equal caustion..........................................28
5 Ethical Guidelines for Psychology Research.....................................29
5.1 Historical Ethical Violations in Research....................................29
5.1.1 The Tuskegee Syphilis Study (1932-1972).......................................................29
5.1.2 Milgram’s Obedience Experiment (1961).........................................................29
5.1.3 1.3 Invasion of Personal Space........................................................................29
5.2 The Belmont Report: Ethical Principles in Research....................30
5.2.1 Three Core Ethical Principles:..........................................................................30
5.3 APA Ethical Guidelines in Psychology........................................30
5.3.1 Five General APA Principles.............................................................................30
5.3.2 APA guidelines - Standard 8:............................................................................30
6 Identifying Good Measurement......................................................35
6.1 Ways to Measure a Variable......................................................35
6.1.1 Example: Measuring Stress in Students During Exams....................................35
6.2 Construct vs. Operational Definitions........................................35
6.2.1 Conceptual vs. Operationalization...................................................................35
6.2.2 Choosing the Right Operationalization.............................................................36
6.3 Reliability and Construct Validity..............................................36
6.3.1 Construct Validity............................................................................................ 36
6.3.2 Reliability: Measuring Consistency...................................................................37
6.3.3 Types of Reliability:......................................................................................... 38
6.3.4 Evaluating Reliability and Validity in Research................................................38
7 Surveys and Observations in Research Methods..............................40
7.1 Surveys...................................................................................40
7.1.1 Types of Surveys............................................................................................. 40
7.2 Experience Sampling Methods (ESM).........................................41
7.3 Single- vs. Multi-Item Measures................................................41
7.4 scales......................................................................................42
7.5 Choosing the Question Format..................................................42
7.5.1 Likert Rating Scale...........................................................................................42
7.5.2 Visual Analogue Scale (VAS)............................................................................42
7.5.3 Semantic Differential Scale..............................................................................42
7.5.4 Guttman Scale.................................................................................................43
7.5.5 Writing Well-Written Questions........................................................................43
7.5.6 Encouraging Accurate Responses....................................................................43
7.5.7 Observations in Research................................................................................44
8 Sampling.......................................................................................45
8.1 Sampling Concepts and Key Terms............................................45
8.2 Key issue.................................................................................46
8.2.1 Generalizability & External Validity..................................................................46
8.3 Sampling Bias..........................................................................47
8.3.1 Key Concepts................................................................................................... 47
8.3.2 Ways a Sample May Be Biased........................................................................48
8.4 Margin of Error........................................................................48
pg. 2
, 8.4.1 What Is Margin of Error?..................................................................................48
8.4.2 Margin of Error by Sample Size........................................................................49
8.5 Sampling Techniques...............................................................49
8.5.1 Population of Interest......................................................................................49
8.5.2 Representative Sampling.................................................................................49
8.5.3 Biased Sampling..............................................................................................49
8.5.4 Biased vs Unbiased Sampling Techniques.......................................................49
8.6 Probability Sampling................................................................50
8.6.1 Types of Probability Sampling..........................................................................50
8.6.2 Biased sampling.............................................................................................. 54
8.6.3 Types of Biased Sampling................................................................................54
8.7 External Validity......................................................................58
8.8 WEIRD Samples........................................................................58
8.9 Online Paid Panels...................................................................59
8.10 Sample Size...........................................................................59
8.11 Response Rate.......................................................................60
8.12 Response Rates (Advanced)....................................................60
8.12.1 General Trends Over Time.............................................................................60
8.12.2 Detailed Findings by Holtom et al..................................................................60
8.12.3 From Anseel et al. (2010).............................................................................61
9 Bivariate correlational research.....................................................61
9.1 Three examples.......................................................................61
9.1.1 Study 1: Mehl et al. (2010) – Eavesdropping on Happiness.............................61
9.1.2 Study 2: Cacioppo et al. (2013) – Online Dating and Relationship Satisfaction 62
9.1.3 Study 3: Siddarth et al. (2018) – Sitting Time and Brain Structure..................63
9.1.4 Cross-Study Comparison & Takeaways...........................................................63
9.1.5 Final Reflections.............................................................................................64
9.2 Bivariate Correlational Research Explained................................64
9.2.1 What Is It?....................................................................................................... 64
9.2.2 Case Example: Cacioppo et al. (2013)............................................................64
9.3 Statistical Validity in Bivariate Correlational Research...............65
9.3.1 What is the Effect Size?...................................................................................66
9.3.2 How Accurate is the Effect Estimate?..............................................................66
9.3.3 Is the Effect Statistically Significant?..............................................................66
9.3.4 Has the Effect Been Replicated?......................................................................66
9.3.5 Do Outliers Influence the Estimate?.................................................................66
9.3.6 Is There Range Restriction?............................................................................67
9.3.7 Is the Relationship Linear?...............................................................................67
9.3.8 Summary Checklist: Ensuring Statistical Validity............................................67
9.4 Internal Validity in Bivariate Correlational Research..................67
9.4.1 Criteria for Causal Inference............................................................................67
9.4.2 Summary Takeaway........................................................................................68
9.5 External Validity in Bivariate Correlational Research..................69
9.5.1 How was the sample selected?........................................................................69
9.5.2 To which population are they trying to generalize?.........................................69
9.5.3 Is external validity important when investigating relationships?.....................69
9.5.4 The Role of Moderators....................................................................................69
9.5.5 Summary Table............................................................................................... 69
9.5.6 Summary Takeaway........................................................................................70
9.6 Moderators in Bivariate Correlational Research.........................70
pg. 3
, 9.6.1 What Is a Moderator?.......................................................................................70
9.6.2 Summary Table.............................................................................................. 71
10 Multivariate correlational research...............................................72
10.1 Criteria for Causality in Multivariate Correlational Research.....72
10.1.1 Final Takeaway............................................................................................. 73
10.2 Longitudinal Designs in Multivariate Correlational Research.....73
10.2.1 What Is a Longitudinal Design?.....................................................................73
10.2.2 Case Study.................................................................................................... 74
10.2.3 Longitudinal Designs vs. Causality Criteria....................................................74
10.2.4 Why Use Longitudinal Design (vs. Experiments)?..........................................74
10.2.5 Summary: Interpreting Longitudinal Paths...................................................75
10.3 Multiple Regression — Ruling Out Third Variables....................75
10.4 Core Concepts........................................................................75
10.4.1 Case Examples.............................................................................................. 75
10.4.2 Summary: What Does Regression Offer?.......................................................77
10.4.3 Summary of Multiple Regression Strengths...................................................77
10.5 Can Multiple Regression Prove Causality?................................77
10.5.1 Popular Misinterpretations in Media..............................................................77
10.5.2 Patterns and Parsimony: Example of Smoking and Cancer...........................78
10.5.3 Key Concepts Explained................................................................................78
10.6 Mediation, Moderation & Confounding: Understanding Variable
Relationships in Research..............................................................78
10.6.1 What is Mediation?........................................................................................79
10.6.2 Mediator vs. Confounder................................................................................79
10.6.3 Mediator vs. Moderator..................................................................................79
10.6.4 Mediation, Moderation, and Third-variable Summary Chart...........................79
10.7 Controlling for Confounders: Ensuring Valid Causal Inference...80
10.7.1 Methods to Control for Confounding..............................................................80
10.7.2 Summary: Why Control Confounders?...........................................................81
11 Simple experimental research......................................................81
11.1 Two Real-World Examples.......................................................81
11.1.1 Summary Comparison: Experimental Design & Insights................................82
11.2 Experimental Variables...........................................................83
11.2.1 Independent Variable (IV)..............................................................................83
11.2.2 Dependent Variable (DV)...............................................................................83
11.2.3 Control Variables...........................................................................................83
11.2.4 Critical Principle............................................................................................. 83
11.2.5 Summary....................................................................................................... 83
11.3 Potential Outcomes Framework & Average Treatment Effect
(ATE)............................................................................................84
11.3.1 What is the Causal Effect?.............................................................................84
11.3.2 How Is the ATE Measured?.............................................................................84
11.3.3 Why This Works:............................................................................................ 84
11.3.4 Interpretation:............................................................................................... 84
11.3.5 Key Benefits of the ATE Framework...............................................................84
11.3.6 Final Summary...............................................................................................85
11.4 Establishing Causality in Experiments.....................................85
11.4.1 Covariance..................................................................................................... 85
11.4.2 Temporal Precedence....................................................................................85
11.4.3 Ruling Out Alternative Explanations..............................................................85
11.4.4 Systematic vs. Non-Systematic Variance.......................................................86
pg. 4
1 Research Methods and Techniques – Practical Information................7
1.1 Language of Instruction.............................................................7
1.2 Course Modules.........................................................................8
1.3 Instructor Team.........................................................................8
1.4 Lecture Schedule & Topics..........................................................8
1.5 Participation & Group Registration.............................................8
1.6 Practical Sessions......................................................................8
1.7 Required Materials.....................................................................9
1.8 Assessment & Evaluation...........................................................9
2 Psychology Is a Way of Thinking.....................................................10
2.1 Producing Versus Consuming Research.....................................10
2.2 How Scientists Work................................................................11
2.2.1 Key Principles of Scientific Practice..................................................................11
2.3 Empiricism...............................................................................12
2.4 Scientists Test Theories...........................................................12
2.4.1 Theory-Data Cycle...........................................................................................12
2.4.2 Exploratory (Inductive) vs. Confirmatory (Deductive) Research......................15
2.4.3 Characteristics of Good Theories.....................................................................15
2.5 Scientists Form a Community....................................................15
2.6 Fundamental Versus Applied Research......................................16
2.7 Continuously Evolving Science..................................................16
2.8 Publishing in Science................................................................16
3 Sources of Information..................................................................17
3.1 Researchers and Their Methods................................................18
3.2 Research Versus Experience.....................................................18
3.3 Research Versus Intuition.........................................................19
3.4 Authority Figures.....................................................................19
3.5 Scientific Sources.....................................................................19
3.5.1 Breakdown of the Diagram:.............................................................................20
3.6 Hierarchy of Evidence...............................................................22
4 Three Claims, Four Validities..........................................................23
4.1 Variables: Descriptions and Definitions.....................................23
4.2 The Three Types of Research Claims.........................................24
4.2.1 Frequency Claims............................................................................................ 24
4.2.2 Association Claims........................................................................................... 24
4.2.3 Causal Claims..................................................................................................25
4.3 The Four Big Validities.............................................................25
4.4 Applying Validity to Different Research Claims..........................26
pg. 1
, 4.5 Conditions for Causality...........................................................27
4.6 Spurious Associations and Balancing Validity............................28
4.7 correllation does not equal caustion..........................................28
5 Ethical Guidelines for Psychology Research.....................................29
5.1 Historical Ethical Violations in Research....................................29
5.1.1 The Tuskegee Syphilis Study (1932-1972).......................................................29
5.1.2 Milgram’s Obedience Experiment (1961).........................................................29
5.1.3 1.3 Invasion of Personal Space........................................................................29
5.2 The Belmont Report: Ethical Principles in Research....................30
5.2.1 Three Core Ethical Principles:..........................................................................30
5.3 APA Ethical Guidelines in Psychology........................................30
5.3.1 Five General APA Principles.............................................................................30
5.3.2 APA guidelines - Standard 8:............................................................................30
6 Identifying Good Measurement......................................................35
6.1 Ways to Measure a Variable......................................................35
6.1.1 Example: Measuring Stress in Students During Exams....................................35
6.2 Construct vs. Operational Definitions........................................35
6.2.1 Conceptual vs. Operationalization...................................................................35
6.2.2 Choosing the Right Operationalization.............................................................36
6.3 Reliability and Construct Validity..............................................36
6.3.1 Construct Validity............................................................................................ 36
6.3.2 Reliability: Measuring Consistency...................................................................37
6.3.3 Types of Reliability:......................................................................................... 38
6.3.4 Evaluating Reliability and Validity in Research................................................38
7 Surveys and Observations in Research Methods..............................40
7.1 Surveys...................................................................................40
7.1.1 Types of Surveys............................................................................................. 40
7.2 Experience Sampling Methods (ESM).........................................41
7.3 Single- vs. Multi-Item Measures................................................41
7.4 scales......................................................................................42
7.5 Choosing the Question Format..................................................42
7.5.1 Likert Rating Scale...........................................................................................42
7.5.2 Visual Analogue Scale (VAS)............................................................................42
7.5.3 Semantic Differential Scale..............................................................................42
7.5.4 Guttman Scale.................................................................................................43
7.5.5 Writing Well-Written Questions........................................................................43
7.5.6 Encouraging Accurate Responses....................................................................43
7.5.7 Observations in Research................................................................................44
8 Sampling.......................................................................................45
8.1 Sampling Concepts and Key Terms............................................45
8.2 Key issue.................................................................................46
8.2.1 Generalizability & External Validity..................................................................46
8.3 Sampling Bias..........................................................................47
8.3.1 Key Concepts................................................................................................... 47
8.3.2 Ways a Sample May Be Biased........................................................................48
8.4 Margin of Error........................................................................48
pg. 2
, 8.4.1 What Is Margin of Error?..................................................................................48
8.4.2 Margin of Error by Sample Size........................................................................49
8.5 Sampling Techniques...............................................................49
8.5.1 Population of Interest......................................................................................49
8.5.2 Representative Sampling.................................................................................49
8.5.3 Biased Sampling..............................................................................................49
8.5.4 Biased vs Unbiased Sampling Techniques.......................................................49
8.6 Probability Sampling................................................................50
8.6.1 Types of Probability Sampling..........................................................................50
8.6.2 Biased sampling.............................................................................................. 54
8.6.3 Types of Biased Sampling................................................................................54
8.7 External Validity......................................................................58
8.8 WEIRD Samples........................................................................58
8.9 Online Paid Panels...................................................................59
8.10 Sample Size...........................................................................59
8.11 Response Rate.......................................................................60
8.12 Response Rates (Advanced)....................................................60
8.12.1 General Trends Over Time.............................................................................60
8.12.2 Detailed Findings by Holtom et al..................................................................60
8.12.3 From Anseel et al. (2010).............................................................................61
9 Bivariate correlational research.....................................................61
9.1 Three examples.......................................................................61
9.1.1 Study 1: Mehl et al. (2010) – Eavesdropping on Happiness.............................61
9.1.2 Study 2: Cacioppo et al. (2013) – Online Dating and Relationship Satisfaction 62
9.1.3 Study 3: Siddarth et al. (2018) – Sitting Time and Brain Structure..................63
9.1.4 Cross-Study Comparison & Takeaways...........................................................63
9.1.5 Final Reflections.............................................................................................64
9.2 Bivariate Correlational Research Explained................................64
9.2.1 What Is It?....................................................................................................... 64
9.2.2 Case Example: Cacioppo et al. (2013)............................................................64
9.3 Statistical Validity in Bivariate Correlational Research...............65
9.3.1 What is the Effect Size?...................................................................................66
9.3.2 How Accurate is the Effect Estimate?..............................................................66
9.3.3 Is the Effect Statistically Significant?..............................................................66
9.3.4 Has the Effect Been Replicated?......................................................................66
9.3.5 Do Outliers Influence the Estimate?.................................................................66
9.3.6 Is There Range Restriction?............................................................................67
9.3.7 Is the Relationship Linear?...............................................................................67
9.3.8 Summary Checklist: Ensuring Statistical Validity............................................67
9.4 Internal Validity in Bivariate Correlational Research..................67
9.4.1 Criteria for Causal Inference............................................................................67
9.4.2 Summary Takeaway........................................................................................68
9.5 External Validity in Bivariate Correlational Research..................69
9.5.1 How was the sample selected?........................................................................69
9.5.2 To which population are they trying to generalize?.........................................69
9.5.3 Is external validity important when investigating relationships?.....................69
9.5.4 The Role of Moderators....................................................................................69
9.5.5 Summary Table............................................................................................... 69
9.5.6 Summary Takeaway........................................................................................70
9.6 Moderators in Bivariate Correlational Research.........................70
pg. 3
, 9.6.1 What Is a Moderator?.......................................................................................70
9.6.2 Summary Table.............................................................................................. 71
10 Multivariate correlational research...............................................72
10.1 Criteria for Causality in Multivariate Correlational Research.....72
10.1.1 Final Takeaway............................................................................................. 73
10.2 Longitudinal Designs in Multivariate Correlational Research.....73
10.2.1 What Is a Longitudinal Design?.....................................................................73
10.2.2 Case Study.................................................................................................... 74
10.2.3 Longitudinal Designs vs. Causality Criteria....................................................74
10.2.4 Why Use Longitudinal Design (vs. Experiments)?..........................................74
10.2.5 Summary: Interpreting Longitudinal Paths...................................................75
10.3 Multiple Regression — Ruling Out Third Variables....................75
10.4 Core Concepts........................................................................75
10.4.1 Case Examples.............................................................................................. 75
10.4.2 Summary: What Does Regression Offer?.......................................................77
10.4.3 Summary of Multiple Regression Strengths...................................................77
10.5 Can Multiple Regression Prove Causality?................................77
10.5.1 Popular Misinterpretations in Media..............................................................77
10.5.2 Patterns and Parsimony: Example of Smoking and Cancer...........................78
10.5.3 Key Concepts Explained................................................................................78
10.6 Mediation, Moderation & Confounding: Understanding Variable
Relationships in Research..............................................................78
10.6.1 What is Mediation?........................................................................................79
10.6.2 Mediator vs. Confounder................................................................................79
10.6.3 Mediator vs. Moderator..................................................................................79
10.6.4 Mediation, Moderation, and Third-variable Summary Chart...........................79
10.7 Controlling for Confounders: Ensuring Valid Causal Inference...80
10.7.1 Methods to Control for Confounding..............................................................80
10.7.2 Summary: Why Control Confounders?...........................................................81
11 Simple experimental research......................................................81
11.1 Two Real-World Examples.......................................................81
11.1.1 Summary Comparison: Experimental Design & Insights................................82
11.2 Experimental Variables...........................................................83
11.2.1 Independent Variable (IV)..............................................................................83
11.2.2 Dependent Variable (DV)...............................................................................83
11.2.3 Control Variables...........................................................................................83
11.2.4 Critical Principle............................................................................................. 83
11.2.5 Summary....................................................................................................... 83
11.3 Potential Outcomes Framework & Average Treatment Effect
(ATE)............................................................................................84
11.3.1 What is the Causal Effect?.............................................................................84
11.3.2 How Is the ATE Measured?.............................................................................84
11.3.3 Why This Works:............................................................................................ 84
11.3.4 Interpretation:............................................................................................... 84
11.3.5 Key Benefits of the ATE Framework...............................................................84
11.3.6 Final Summary...............................................................................................85
11.4 Establishing Causality in Experiments.....................................85
11.4.1 Covariance..................................................................................................... 85
11.4.2 Temporal Precedence....................................................................................85
11.4.3 Ruling Out Alternative Explanations..............................................................85
11.4.4 Systematic vs. Non-Systematic Variance.......................................................86
pg. 4
