Research methods and techniques II
quantitative methods
Content: HC + WPO
HOC 1: Introduction to scientific reasoning ............................................................................................................ 6
Psychology As a way of thinking ......................................................................................................................... 6
Producing VS. Consuming research................................................................................................................. 6
The Importance of Being a Critical Research Consumer ................................................................................. 7
How scientists work ............................................................................................................................................ 9
Empiricism ....................................................................................................................................................... 9
Scientists test theories .................................................................................................................................. 10
Scientists form a community............................................................................................................................. 12
Fundamental versus applied research .............................................................................................................. 13
Theories Are Continuously Tested, Modified, and Falsified.............................................................................. 14
Publishing .......................................................................................................................................................... 15
Communication ................................................................................................................................................. 16
HOC1: Sources of information .............................................................................................................................. 17
Research versus experience .............................................................................................................................. 18
Research versus intuition .................................................................................................................................. 20
Authority figures ............................................................................................................................................... 22
Overview ........................................................................................................................................................... 23
HOC 2: Foundations of research ........................................................................................................................... 24
Overview ........................................................................................................................................................... 24
Three Claims and Four Validities ....................................................................................................................... 24
Variables........................................................................................................................................................ 25
Three Claims .................................................................................................................................................. 26
The four big validities .................................................................................................................................... 31
Four types of validity and frequency claims.................................................................................................. 32
Four types of validity and association claims ................................................................................................ 32
Conditions for Causality ................................................................................................................................ 33
Spurious Associations: A Warning About Mistaking Correlation for Causation............................................ 36
Correlation Does Not Equal Causation .......................................................................................................... 36
Prioritizing Types of Validity.......................................................................................................................... 36
2. Ethical guidelines for psychology research ................................................................................................... 37
The Belmont Report ...................................................................................................................................... 38
APA Ethical Guidelines .................................................................................................................................. 38
Stephanie De Wacker 1
Extra notes thanks to: Hülya Yazici
, Institutional Review Boards (IRBs) ................................................................................................................ 39
Informed Consent ......................................................................................................................................... 39
Deception in Research .................................................................................................................................. 39
Debriefing ...................................................................................................................................................... 40
Research Fraud.................................................................................................................................................. 40
Prevalence of Questionable Research Practices (QRPs) ............................................................................... 41
Reasons and Solutions for Research Fraud ................................................................................................... 41
Open Science (Crüwell et al., 2019) .............................................................................................................. 41
Animal Research and Ethics .......................................................................................................................... 42
HOC 3: Survey research ......................................................................................................................................... 43
Overview ........................................................................................................................................................... 43
Identifying good measurement......................................................................................................................... 43
Different way’s to measure a variable .......................................................................................................... 43
Operationalizing ............................................................................................................................................ 43
Construct validity and reliability ................................................................................................................... 44
Reliability ....................................................................................................................................................... 44
Construct validity .......................................................................................................................................... 46
Evaluating reliability and construct validity in articles .................................................................................. 47
Surveys and observations ................................................................................................................................. 48
Experience sampling methods (ESM) ............................................................................................................ 48
Single- vs. Multi-Item Measures ................................................................................................................... 49
Scales ............................................................................................................................................................. 49
Likert Scale .................................................................................................................................................... 50
Visual analogue scale (VAS)........................................................................................................................... 51
Semantic Differential Scale ........................................................................................................................... 51
Guttman Scale ............................................................................................................................................... 51
Writing well-worded questions ..................................................................................................................... 52
Encouraging Accurate Answers ..................................................................................................................... 52
Hardy & Ford (2014) found: .......................................................................................................................... 52
Observations ................................................................................................................................................. 53
HOC 4: Sampling.................................................................................................................................................... 55
Overview ........................................................................................................................................................... 55
Sampling ........................................................................................................................................................ 55
Key Issues ...................................................................................................................................................... 56
Illustration ..................................................................................................................................................... 56
Sampling bias ................................................................................................................................................ 58
Ways a sample may be biased ...................................................................................................................... 58
Stephanie De Wacker 2
Extra notes thanks to: Hülya Yazici
, Margin-of-error ............................................................................................................................................. 58
Biased vs unbiased sampling techniques ...................................................................................................... 59
Representative Sampling: ............................................................................................................................. 59
Probability sampling...................................................................................................................................... 60
Simple random sample ................................................................................................................................. 60
Cluster sample............................................................................................................................................... 60
Stratified random sample ............................................................................................................................. 60
Oversampling ................................................................................................................................................ 60
Raking example ............................................................................................................................................. 60
Systematic sampling ...................................................................................................................................... 61
Multistage sampling ...................................................................................................................................... 61
Biased sampling ............................................................................................................................................ 61
Purposive sampling ....................................................................................................................................... 62
Snowball sampling ........................................................................................................................................ 62
Convenience sample ..................................................................................................................................... 63
Quota sampling ............................................................................................................................................. 63
Exercise: ........................................................................................................................................................ 65
When is external validity important? ............................................................................................................ 66
WEIRD samples ............................................................................................................................................. 67
Online paid panels......................................................................................................................................... 67
Sample size .................................................................................................................................................... 68
Response rate ............................................................................................................................................... 68
HOC 5: Bivariate correlational research ................................................................................................................ 69
Overview ........................................................................................................................................................... 69
Three examples ................................................................................................................................................. 69
Mehl et al. (2010): Eavesdropping on happiness .......................................................................................... 69
Cacioppo et al. (2013) ................................................................................................................................... 70
Siddarth et al. (2018)..................................................................................................................................... 71
Bivariate correlational research .................................................................................................................... 71
Evaluating Validity ......................................................................................................................................... 71
Bivariate Correlational Research: Strengths & Weaknesses ......................................................................... 72
Statistical validity .......................................................................................................................................... 73
What Does p < .05 Mean? ............................................................................................................................. 74
Additional Factors Influencing Statistical Validity ......................................................................................... 75
Internal validity ............................................................................................................................................. 76
External validity ............................................................................................................................................. 77
Moderators and Generalization .................................................................................................................... 77
Stephanie De Wacker 3
Extra notes thanks to: Hülya Yazici
,HOC 6: Multivariate correlational research .......................................................................................................... 78
Criteria for causality .......................................................................................................................................... 78
Longitudinal designs.......................................................................................................................................... 79
Exercise ............................................................................................................................................................. 80
Multiple regression ........................................................................................................................................... 81
Mediation ...................................................................................................................................................... 85
Controlling for confounders .......................................................................................................................... 88
HOC 7: Simple experimental research .................................................................................................................. 89
Two Examples ................................................................................................................................................... 89
Mueller & Oppenheimer (2014).................................................................................................................... 89
Van Kleef, Shimizu, & Wansink (2012) .......................................................................................................... 90
Experimental variables .................................................................................................................................. 90
Potential outcomes framework .................................................................................................................... 91
Experiments .................................................................................................................................................. 92
Evaluating four aspects of validity .................................................................................................................... 98
HOC 8: Confounding and obscuring variables ..................................................................................................... 101
Examples of bad experiments ......................................................................................................................... 101
Threats to internal validity (12)....................................................................................................................... 101
Null-effects ...................................................................................................................................................... 103
Power .............................................................................................................................................................. 104
HOC 9: Experiments with more than 1 independent variable ............................................................................ 106
Experiment with 1 independent variable........................................................................................................ 106
Factorial designs ......................................................................................................................................... 106
Main Effects vs. Interaction Effects ............................................................................................................. 106
Design variants ............................................................................................................................................ 107
HOC 10: Quasi-experiments, small-N designs, & replication .............................................................................. 108
Quasi-experiments .......................................................................................................................................... 108
Small-N experiment .................................................................................................................................... 109
Replication .................................................................................................................................................. 110
Types ........................................................................................................................................................... 111
Meta-analysis .............................................................................................................................................. 112
Generalizability ........................................................................................................................................... 113
WPO 1: part 1 & Part 2 ........................................................................................................................................ 114
Part 1: Basic concepts & methods in research ................................................................................................ 114
PART 2: validities & Reliabilities ...................................................................................................................... 115
Choice of items............................................................................................................................................ 115
Items ........................................................................................................................................................... 115
Stephanie De Wacker 4
Extra notes thanks to: Hülya Yazici
, Unidimensional & Multi-item scales ........................................................................................................... 115
Choice of Answering Categories ................................................................................................................. 115
WPO 2: relationships between variables ............................................................................................................ 116
Choosing and interpreting a statistical test .................................................................................................... 116
WPO 3: Research designs (1) .............................................................................................................................. 118
Part 1: Confounding variables in research ...................................................................................................... 118
Part 2: between-subjects versus within-subject designs ................................................................................. 118
Counterbalancing ........................................................................................................................................ 119
WPO 4: Research Designs (2) .............................................................................................................................. 120
Counterbalancing ............................................................................................................................................ 120
Factors and Factorial Designs.......................................................................................................................... 120
Quasi-Experimental Design ............................................................................................................................. 120
Stephanie De Wacker 5
Extra notes thanks to: Hülya Yazici
,HOC 1: Introduction to scientific reasoning
Psychology As a way of thinking
Psychology as a way of thinking and a broad discussion of what scientific research is.
We learn certain principles because psychology is based on research conducted by psychologists. Like other
scientists, psychologists are empiricists. An empiricist draws conclusions based on systematic observations.
Producing VS. Consuming research
Why this course? (Research in Psychology)
o Psychologists rely on scientific research to make informed decisions, e.g., choosing the best treatment
for a patient.
o To do this, you must be able to find, understand, and critically evaluate research (identify
strengths/weaknesses).
o Psychologists are empiricists: they draw conclusions based on systematic observation.
Example – Facilitated Communication (FC):
o FC is a technique where a facilitator helps a patient with severe autism type words.
o Research showed that the facilitator unintentionally guided the answers.
o Typed words matched what the facilitator saw, not the patient → FC is not reliable or effective.
o Despite the evidence, some still believe in FC → highlights the importance of cri cal thinking and
scientific evidence.
Two roles of psychology students:
1. Producers of research
o Conduct studies, publish, and present findings.
o Examples: brain anatomy, animal behavior, school observations.
2. Consumers of research
o Apply existing research in work, hobbies, relationships, or personal growth.
Why it's important to critically evaluate scientific research
Case: Griet Op de Beeck (author)
o Started practicing therapy without formal qualifications, relying on workshops and trainings.
o Claimed she could treat serious mental health conditions (e.g., depression, PTSD).
o Referenced Internal Family Systems Therapy and claimed it was supported by scientific research.
While this sounds convincing due to the mention of a study, critical evaluation reveals weak evidence:
The study:
o Involved a small sample (10–20 participants) → risk of false positives.
o Had no control group, making it impossible to rule out alternative explanations for reduced stress
levels.
o Only pre-post design: measured stress before and after the intervention.
Conclusion: The claim that the therapy works is not strongly supported by scientific evidence. This shows the
importance of being able to read and critically assess research.
Stephanie De Wacker 6
Extra notes thanks to: Hülya Yazici
,Why the role of the researcher matters (Book)
Even if you don’t aim to become a psychologist, you’ll likely need to:
o Write APA-style papers
o Conduct research in practica or other assignments
o Understand key research concepts: random assignment, accurate measurement, and graph
interpretation
Skills gained through research are also career-relevant:
o Organizing & analyzing data
o Measuring behavior precisely
o Understanding ethical practices (e.g., protecting anonymity)
If you join a research lab, you might:
o Assign participants to groups
o Code behavior
o Enter data
o Write reports
Conclusion: This hands-on experience deepens your understanding of research and prepares you for both
academic and professional success.
The Importance of Being a Critical Research Consumer
Case Study: Daryl Bem – Feeling the Future (2011)
Bem claimed that people are able to predict future events, supported by 9 experiments.
Example: Participants memorized a word list, then recalled words. Later, a computer randomly
selected words. Surprisingly, the recalled words often matched the ones selected after the test
suggesting precognitive memory.
Though results seemed convincing, serious concerns emerged:
o Selective reporting: Only experiments with significant results were published.
o Data manipulation (p-hacking): Bem altered statistical analyses until he found support for his
hypothesis.
o Failed replication: Other researchers could not replicate his findings.
This case became a key example in the replication crisis the realization that many psychological studies,
especially in social psychology, are difficult or impossible to replicate.
The replication crisis sparked the Open Science Movement, encouraging greater transparency and rigor in
research.
Conclusion: Even published research should be evaluated critically.
Why the Role of Research Consumer Is Important (Book)
Most psychology students do not become full-time researchers, but everyone should learn to
evaluate research critically.
In everyday life, we constantly encounter research-based claims in the news, online, in politics, and
healthcare.
Some studies are accurate and useful; others are misleading or false. A research methods course helps
you ask the right questions and assess the quality of a study.
Stephanie De Wacker 7
Extra notes thanks to: Hülya Yazici
, Applications in various fields:
Healthcare and social work:
o Clinical psychologists, social workers, and therapists rely on research to determine which
treatments are effective.
o Licensing often requires knowledge of evidence-based practices.
Education:
o Teachers use research to choose effective teaching methods.
Business:
o Companies analyze data to predict sales trends and understand consumer behavior.
The book emphasizes a mindset of "interrogating information like a detective". It teaches you how to use
systematic rules to evaluate research a valuable skill in many professional settings.
The Benefits of Being a Critical Consumer of Research
Being a critical consumer of research helps you make informed decisions even when something sounds
intuitively correct.
Case Study: Scared Straight Program
Imagine working in a juvenile detention center and hearing about Scared Straight, a program where at-risk
teens visit prisons and hear about the harsh realities of prison life from inmates. The goal is to scare teens
away from crime.
The program seems logical, and your employer considers partnering with the state prison system. Before
moving forward, you help investigate the research behind it. Surprisingly, empirical studies show that Scared
Straight not only fails to reduce crime but may actually increase it.
Randomized controlled trials assigned teens to either Scared Straight or a control group.
Researchers tracked participants’ criminal records over 6–12 months.
Results: Most studies found no benefit. In fact, some showed participants in the program committed
up to 20% more crimes than those in the control group.
This example shows that even well-intentioned programs can be harmful and underlines the importance of
evidence-based evaluation.
The U.S. Department of Justice has since officially warned against such programs, and the Juvenile Justice and
Delinquency Prevention Act was amended to prevent youth from being exposed to adult inmates.
Case Study: Mindfulness and Academic Performance
Being a critical thinker also helps identify effective programs. Suppose you want to improve your academic
performance and come across a headline: “Mindfulness boosts test scores.”
You read a study by Michael Mrazek and colleagues:
Participants were assigned to either a two-week mindfulness course or a two-week nutrition course.
Only those in the mindfulness group showed fewer distracting thoughts and higher test scores after
the intervention.
After reviewing this and similar studies, you decide to try mindfulness yourself and if you're a teacher or tutor,
you might even recommend it to students.
Conclusion: Being a critical consumer allows you to:
Recognize when programs are backed by evidence.
Avoid investing time or resources in ineffective or harmful interventions.
Make evidence-based decisions that benefit yourself, your future employers, and the people you
work with.
Stephanie De Wacker 8
Extra notes thanks to: Hülya Yazici
quantitative methods
Content: HC + WPO
HOC 1: Introduction to scientific reasoning ............................................................................................................ 6
Psychology As a way of thinking ......................................................................................................................... 6
Producing VS. Consuming research................................................................................................................. 6
The Importance of Being a Critical Research Consumer ................................................................................. 7
How scientists work ............................................................................................................................................ 9
Empiricism ....................................................................................................................................................... 9
Scientists test theories .................................................................................................................................. 10
Scientists form a community............................................................................................................................. 12
Fundamental versus applied research .............................................................................................................. 13
Theories Are Continuously Tested, Modified, and Falsified.............................................................................. 14
Publishing .......................................................................................................................................................... 15
Communication ................................................................................................................................................. 16
HOC1: Sources of information .............................................................................................................................. 17
Research versus experience .............................................................................................................................. 18
Research versus intuition .................................................................................................................................. 20
Authority figures ............................................................................................................................................... 22
Overview ........................................................................................................................................................... 23
HOC 2: Foundations of research ........................................................................................................................... 24
Overview ........................................................................................................................................................... 24
Three Claims and Four Validities ....................................................................................................................... 24
Variables........................................................................................................................................................ 25
Three Claims .................................................................................................................................................. 26
The four big validities .................................................................................................................................... 31
Four types of validity and frequency claims.................................................................................................. 32
Four types of validity and association claims ................................................................................................ 32
Conditions for Causality ................................................................................................................................ 33
Spurious Associations: A Warning About Mistaking Correlation for Causation............................................ 36
Correlation Does Not Equal Causation .......................................................................................................... 36
Prioritizing Types of Validity.......................................................................................................................... 36
2. Ethical guidelines for psychology research ................................................................................................... 37
The Belmont Report ...................................................................................................................................... 38
APA Ethical Guidelines .................................................................................................................................. 38
Stephanie De Wacker 1
Extra notes thanks to: Hülya Yazici
, Institutional Review Boards (IRBs) ................................................................................................................ 39
Informed Consent ......................................................................................................................................... 39
Deception in Research .................................................................................................................................. 39
Debriefing ...................................................................................................................................................... 40
Research Fraud.................................................................................................................................................. 40
Prevalence of Questionable Research Practices (QRPs) ............................................................................... 41
Reasons and Solutions for Research Fraud ................................................................................................... 41
Open Science (Crüwell et al., 2019) .............................................................................................................. 41
Animal Research and Ethics .......................................................................................................................... 42
HOC 3: Survey research ......................................................................................................................................... 43
Overview ........................................................................................................................................................... 43
Identifying good measurement......................................................................................................................... 43
Different way’s to measure a variable .......................................................................................................... 43
Operationalizing ............................................................................................................................................ 43
Construct validity and reliability ................................................................................................................... 44
Reliability ....................................................................................................................................................... 44
Construct validity .......................................................................................................................................... 46
Evaluating reliability and construct validity in articles .................................................................................. 47
Surveys and observations ................................................................................................................................. 48
Experience sampling methods (ESM) ............................................................................................................ 48
Single- vs. Multi-Item Measures ................................................................................................................... 49
Scales ............................................................................................................................................................. 49
Likert Scale .................................................................................................................................................... 50
Visual analogue scale (VAS)........................................................................................................................... 51
Semantic Differential Scale ........................................................................................................................... 51
Guttman Scale ............................................................................................................................................... 51
Writing well-worded questions ..................................................................................................................... 52
Encouraging Accurate Answers ..................................................................................................................... 52
Hardy & Ford (2014) found: .......................................................................................................................... 52
Observations ................................................................................................................................................. 53
HOC 4: Sampling.................................................................................................................................................... 55
Overview ........................................................................................................................................................... 55
Sampling ........................................................................................................................................................ 55
Key Issues ...................................................................................................................................................... 56
Illustration ..................................................................................................................................................... 56
Sampling bias ................................................................................................................................................ 58
Ways a sample may be biased ...................................................................................................................... 58
Stephanie De Wacker 2
Extra notes thanks to: Hülya Yazici
, Margin-of-error ............................................................................................................................................. 58
Biased vs unbiased sampling techniques ...................................................................................................... 59
Representative Sampling: ............................................................................................................................. 59
Probability sampling...................................................................................................................................... 60
Simple random sample ................................................................................................................................. 60
Cluster sample............................................................................................................................................... 60
Stratified random sample ............................................................................................................................. 60
Oversampling ................................................................................................................................................ 60
Raking example ............................................................................................................................................. 60
Systematic sampling ...................................................................................................................................... 61
Multistage sampling ...................................................................................................................................... 61
Biased sampling ............................................................................................................................................ 61
Purposive sampling ....................................................................................................................................... 62
Snowball sampling ........................................................................................................................................ 62
Convenience sample ..................................................................................................................................... 63
Quota sampling ............................................................................................................................................. 63
Exercise: ........................................................................................................................................................ 65
When is external validity important? ............................................................................................................ 66
WEIRD samples ............................................................................................................................................. 67
Online paid panels......................................................................................................................................... 67
Sample size .................................................................................................................................................... 68
Response rate ............................................................................................................................................... 68
HOC 5: Bivariate correlational research ................................................................................................................ 69
Overview ........................................................................................................................................................... 69
Three examples ................................................................................................................................................. 69
Mehl et al. (2010): Eavesdropping on happiness .......................................................................................... 69
Cacioppo et al. (2013) ................................................................................................................................... 70
Siddarth et al. (2018)..................................................................................................................................... 71
Bivariate correlational research .................................................................................................................... 71
Evaluating Validity ......................................................................................................................................... 71
Bivariate Correlational Research: Strengths & Weaknesses ......................................................................... 72
Statistical validity .......................................................................................................................................... 73
What Does p < .05 Mean? ............................................................................................................................. 74
Additional Factors Influencing Statistical Validity ......................................................................................... 75
Internal validity ............................................................................................................................................. 76
External validity ............................................................................................................................................. 77
Moderators and Generalization .................................................................................................................... 77
Stephanie De Wacker 3
Extra notes thanks to: Hülya Yazici
,HOC 6: Multivariate correlational research .......................................................................................................... 78
Criteria for causality .......................................................................................................................................... 78
Longitudinal designs.......................................................................................................................................... 79
Exercise ............................................................................................................................................................. 80
Multiple regression ........................................................................................................................................... 81
Mediation ...................................................................................................................................................... 85
Controlling for confounders .......................................................................................................................... 88
HOC 7: Simple experimental research .................................................................................................................. 89
Two Examples ................................................................................................................................................... 89
Mueller & Oppenheimer (2014).................................................................................................................... 89
Van Kleef, Shimizu, & Wansink (2012) .......................................................................................................... 90
Experimental variables .................................................................................................................................. 90
Potential outcomes framework .................................................................................................................... 91
Experiments .................................................................................................................................................. 92
Evaluating four aspects of validity .................................................................................................................... 98
HOC 8: Confounding and obscuring variables ..................................................................................................... 101
Examples of bad experiments ......................................................................................................................... 101
Threats to internal validity (12)....................................................................................................................... 101
Null-effects ...................................................................................................................................................... 103
Power .............................................................................................................................................................. 104
HOC 9: Experiments with more than 1 independent variable ............................................................................ 106
Experiment with 1 independent variable........................................................................................................ 106
Factorial designs ......................................................................................................................................... 106
Main Effects vs. Interaction Effects ............................................................................................................. 106
Design variants ............................................................................................................................................ 107
HOC 10: Quasi-experiments, small-N designs, & replication .............................................................................. 108
Quasi-experiments .......................................................................................................................................... 108
Small-N experiment .................................................................................................................................... 109
Replication .................................................................................................................................................. 110
Types ........................................................................................................................................................... 111
Meta-analysis .............................................................................................................................................. 112
Generalizability ........................................................................................................................................... 113
WPO 1: part 1 & Part 2 ........................................................................................................................................ 114
Part 1: Basic concepts & methods in research ................................................................................................ 114
PART 2: validities & Reliabilities ...................................................................................................................... 115
Choice of items............................................................................................................................................ 115
Items ........................................................................................................................................................... 115
Stephanie De Wacker 4
Extra notes thanks to: Hülya Yazici
, Unidimensional & Multi-item scales ........................................................................................................... 115
Choice of Answering Categories ................................................................................................................. 115
WPO 2: relationships between variables ............................................................................................................ 116
Choosing and interpreting a statistical test .................................................................................................... 116
WPO 3: Research designs (1) .............................................................................................................................. 118
Part 1: Confounding variables in research ...................................................................................................... 118
Part 2: between-subjects versus within-subject designs ................................................................................. 118
Counterbalancing ........................................................................................................................................ 119
WPO 4: Research Designs (2) .............................................................................................................................. 120
Counterbalancing ............................................................................................................................................ 120
Factors and Factorial Designs.......................................................................................................................... 120
Quasi-Experimental Design ............................................................................................................................. 120
Stephanie De Wacker 5
Extra notes thanks to: Hülya Yazici
,HOC 1: Introduction to scientific reasoning
Psychology As a way of thinking
Psychology as a way of thinking and a broad discussion of what scientific research is.
We learn certain principles because psychology is based on research conducted by psychologists. Like other
scientists, psychologists are empiricists. An empiricist draws conclusions based on systematic observations.
Producing VS. Consuming research
Why this course? (Research in Psychology)
o Psychologists rely on scientific research to make informed decisions, e.g., choosing the best treatment
for a patient.
o To do this, you must be able to find, understand, and critically evaluate research (identify
strengths/weaknesses).
o Psychologists are empiricists: they draw conclusions based on systematic observation.
Example – Facilitated Communication (FC):
o FC is a technique where a facilitator helps a patient with severe autism type words.
o Research showed that the facilitator unintentionally guided the answers.
o Typed words matched what the facilitator saw, not the patient → FC is not reliable or effective.
o Despite the evidence, some still believe in FC → highlights the importance of cri cal thinking and
scientific evidence.
Two roles of psychology students:
1. Producers of research
o Conduct studies, publish, and present findings.
o Examples: brain anatomy, animal behavior, school observations.
2. Consumers of research
o Apply existing research in work, hobbies, relationships, or personal growth.
Why it's important to critically evaluate scientific research
Case: Griet Op de Beeck (author)
o Started practicing therapy without formal qualifications, relying on workshops and trainings.
o Claimed she could treat serious mental health conditions (e.g., depression, PTSD).
o Referenced Internal Family Systems Therapy and claimed it was supported by scientific research.
While this sounds convincing due to the mention of a study, critical evaluation reveals weak evidence:
The study:
o Involved a small sample (10–20 participants) → risk of false positives.
o Had no control group, making it impossible to rule out alternative explanations for reduced stress
levels.
o Only pre-post design: measured stress before and after the intervention.
Conclusion: The claim that the therapy works is not strongly supported by scientific evidence. This shows the
importance of being able to read and critically assess research.
Stephanie De Wacker 6
Extra notes thanks to: Hülya Yazici
,Why the role of the researcher matters (Book)
Even if you don’t aim to become a psychologist, you’ll likely need to:
o Write APA-style papers
o Conduct research in practica or other assignments
o Understand key research concepts: random assignment, accurate measurement, and graph
interpretation
Skills gained through research are also career-relevant:
o Organizing & analyzing data
o Measuring behavior precisely
o Understanding ethical practices (e.g., protecting anonymity)
If you join a research lab, you might:
o Assign participants to groups
o Code behavior
o Enter data
o Write reports
Conclusion: This hands-on experience deepens your understanding of research and prepares you for both
academic and professional success.
The Importance of Being a Critical Research Consumer
Case Study: Daryl Bem – Feeling the Future (2011)
Bem claimed that people are able to predict future events, supported by 9 experiments.
Example: Participants memorized a word list, then recalled words. Later, a computer randomly
selected words. Surprisingly, the recalled words often matched the ones selected after the test
suggesting precognitive memory.
Though results seemed convincing, serious concerns emerged:
o Selective reporting: Only experiments with significant results were published.
o Data manipulation (p-hacking): Bem altered statistical analyses until he found support for his
hypothesis.
o Failed replication: Other researchers could not replicate his findings.
This case became a key example in the replication crisis the realization that many psychological studies,
especially in social psychology, are difficult or impossible to replicate.
The replication crisis sparked the Open Science Movement, encouraging greater transparency and rigor in
research.
Conclusion: Even published research should be evaluated critically.
Why the Role of Research Consumer Is Important (Book)
Most psychology students do not become full-time researchers, but everyone should learn to
evaluate research critically.
In everyday life, we constantly encounter research-based claims in the news, online, in politics, and
healthcare.
Some studies are accurate and useful; others are misleading or false. A research methods course helps
you ask the right questions and assess the quality of a study.
Stephanie De Wacker 7
Extra notes thanks to: Hülya Yazici
, Applications in various fields:
Healthcare and social work:
o Clinical psychologists, social workers, and therapists rely on research to determine which
treatments are effective.
o Licensing often requires knowledge of evidence-based practices.
Education:
o Teachers use research to choose effective teaching methods.
Business:
o Companies analyze data to predict sales trends and understand consumer behavior.
The book emphasizes a mindset of "interrogating information like a detective". It teaches you how to use
systematic rules to evaluate research a valuable skill in many professional settings.
The Benefits of Being a Critical Consumer of Research
Being a critical consumer of research helps you make informed decisions even when something sounds
intuitively correct.
Case Study: Scared Straight Program
Imagine working in a juvenile detention center and hearing about Scared Straight, a program where at-risk
teens visit prisons and hear about the harsh realities of prison life from inmates. The goal is to scare teens
away from crime.
The program seems logical, and your employer considers partnering with the state prison system. Before
moving forward, you help investigate the research behind it. Surprisingly, empirical studies show that Scared
Straight not only fails to reduce crime but may actually increase it.
Randomized controlled trials assigned teens to either Scared Straight or a control group.
Researchers tracked participants’ criminal records over 6–12 months.
Results: Most studies found no benefit. In fact, some showed participants in the program committed
up to 20% more crimes than those in the control group.
This example shows that even well-intentioned programs can be harmful and underlines the importance of
evidence-based evaluation.
The U.S. Department of Justice has since officially warned against such programs, and the Juvenile Justice and
Delinquency Prevention Act was amended to prevent youth from being exposed to adult inmates.
Case Study: Mindfulness and Academic Performance
Being a critical thinker also helps identify effective programs. Suppose you want to improve your academic
performance and come across a headline: “Mindfulness boosts test scores.”
You read a study by Michael Mrazek and colleagues:
Participants were assigned to either a two-week mindfulness course or a two-week nutrition course.
Only those in the mindfulness group showed fewer distracting thoughts and higher test scores after
the intervention.
After reviewing this and similar studies, you decide to try mindfulness yourself and if you're a teacher or tutor,
you might even recommend it to students.
Conclusion: Being a critical consumer allows you to:
Recognize when programs are backed by evidence.
Avoid investing time or resources in ineffective or harmful interventions.
Make evidence-based decisions that benefit yourself, your future employers, and the people you
work with.
Stephanie De Wacker 8
Extra notes thanks to: Hülya Yazici
