Research Methods in Psychology
Chapter 1-14
Vrije Universiteit Amsterdam
Semester 1, Period 1
Chapter 1
Research producers and consumers: share commitment to empiricism
Producers:
Research scientists
Professors
Consumers:
Therapists
Teachers
Entrepreneurs
Counselors
Police officers
Evidence-based treatments: therapies that are supported by research
Empiricism: involves using evidence from the senses or from other instruments that
assist the senses as the basis for conclusions
Aim is to be systematic, rigorous and to make work independently verifiable
Theory: set of statements that describes general principles about how variables
relate to one another
Hypothesis or prediction: the specific outcome the researcher expects to observe in
a study if the theory is accurate
Most researchers test theories with a series of empirical studies, each
designed to test an individual hypothesis
Data: set of observations
The best theories are:
Supported by data from research studies
Multiple studies, variety of methods to address different aspects of
theories
Falsifiable: theory must lead to hypotheses that, when tested, could actually
fail support it
Parsimonious: when theories explain data equally well, scientists opt for the
simpler explanation
Theories don’t prove anything
Scientists evaluate theories based on the weight of the evidence, for and
against
Applied research: done with a practical problem in mind; conducted in a particular
real-world context
Basic research: aims to enhance the general body of knowledge
Generated knowledge may be applied to real-world issues later on
Translational research: use of lessons from basic research to develop and test
applications to health care, psychotherapy or other forms of treatment and
intervention
1
,Basic research > Translational research > Applied research
Research publication
Through scientific journals
Articles are peer-reviewed
Read primarily by other scientists and psychology students
Journalism: contrasts journals in that it includes the kinds of news and commentary
that is read or heard on television, magazines, newspapers and the internet
Not written by scientists, meant for general public, special education is not
required to understand their content
Journalists publishing psychological research may have benefits
Better understanding of psychologists’ work
Better understanding of self and environment > change of habits
However, story should be important and reported accurately
Chapter 2
Experience has no comparison group
Comparison group: enables us to compare what would happen both with and
without the thing we are interested in
Basing conclusions on personal experience is problematic because
daily life usually doesn’t include comparison experiences
Experience is confounded – even if a change occurs, there is no
certainty in what caused it
In contrast: basing conclusions on systematic data collection provides
a comparison group
Confounds: several possible explanations for an outcome in real-world situations
Occurs when you think one thing caused an outcome but in fact other things
changed too
Careful controls are used in a research setting to make sure only one
factor at a time is changed
Confederate: an actor playing a specific role for an experimenter
Behavioral research is probabilistic; its findings are not expected to explain all cases
of all time, but rather a certain proportion
Examples of biased reasoning:
Being swayed by a good story (catharsis, Scared Straight)
Being persuaded by what comes easily to mind
Availability heuristic: states that things that pop up easily in our mind
tend to guide our thinking, might lead us to wrongly estimate the
number of something or how often something happens
Failing to think about what we cannot see: when testing relationships, we
often fail to look for absences
Present/present bias: our failure to consider appropriate comparison
groups – Dr. Rush bloodletting study
2
, Instances where treatment was absent, but outcome was still present
are overlooked
The availability heuristic plays a role in the present/present bias
because instances in the present/present cell of a comparison stand
out
Present/present bias adds tendency to ignore absent cells, which
are essential for testing relationships
To avoid this bias, ask: “Compared to what?”
Focusing on the evidence we like best
Confirmation bias: tendency to look only at information that agrees
with what we already believe
Biased about being biased
Bias blind spot: the belief that we are unlikely to be susceptible to
biases previously described
Makes us trust out faulty reasoning even more
Can make it difficult for us to initiate the scientific theory-data
cycle
To be an empiricist:
Base believes on systematic information from the senses
Strive to interpret the data you collect in an objective way; guard against
common biases
Before taking advice from authorities:
Ask about the source of their ideas
Objective and systematic comparison of conditions?
Reference to research evidence? > if yes, attention worthy
Scientific sources
Journal articles
Empirical journal articles: report results of an empirical research study
Review journal articles: provide a summary of all the published studies
that have been done in one research area
Meta-analysis may be used: quantitative technique that
combines result of many studies and gives a number that
summarizes magnitude > effect size of a relationship
o Valued technique; doesn’t allow cherry-picking and
weighs each study proportionately
Chapters in edited books
Collection of chapters on a common topic; each written by a different
contributor
Not the first time a study is reported, rather a summary of research
and explanation of theory behind it
Good way to find a summary of a specific person’s work
Not as rigorously peer-reviewed as scientific sources
Psychologists and psychology students audience
Full-length books
Common in other disciplines but not so much in psychology
Finding scientific sources
3
, PsycINFO
Google Scholar
Components of an empirical journal article:
Abstract
Summary of the article describing hypotheses, method and major
results
Introduction
Explains topic of study, background for research and the study’s
specific research questions/goals/hypotheses
Method
Participants, materials, procedure and apparatus
Results
Tables and figures summarizing key results
Qualitative and qualitative data
Discussion
Summarizes and discusses the study’s importance, gives alternative
explanations for data and raises new questions
References
Reading with purpose
1. What is the argument?
Abstract, introduction, first paragraph of discussion
2. What is the evidence to support the argument?
Method and results
Wikis as a research source
Not comprehensive in their coverage
No comprehensive reference list
Details might be incorrect until they are fixed
Vandalism is a potential problem
Chapter 3
Variable: something that varies, and which must at least have two levels/values
Constant: something that could potentially vary but has only one level in the study
in question
Measured variable: one whose levels are simply observed and recorded
Manipulated variable: variable a researcher controls, usually by assigning study
participants to different levels of that variable
Some variables can be either measured or manipulated – hair color
Conceptual variables/constructs: abstract concepts defined at the theoretical level
by conceptual definitions
Operational variables/operationalizations: operational definitions of variables
created by testing hypotheses with empirical research
Claim: argument someone is trying to make
Frequency claims: describe a particular rate or degree of a single variable
Variables are always measured, not manipulated
4
, Association claims: argue that one level of a variable is likely to be associated
with a particular level of another variable
Variables that are associated are said to correlate or to be related
When one variable changes, the other tends to change
Correlational study: study in which variables are measured ant the
relationship between them is tested
Positive association/correlation: in which high goes with high,
and low goes with low > scatterplot can be used to represent an
association
Negative association/correlation: high goes with low, and low
goes with high
Zero association/correlation
Causal claims: argue that one of the variables is responsible for changing the
other
Have two variables
Start with a positive or negative association
Causal verbs tend to be more active and forceful, suggesting one
variable comes first in time and then acts on the other
Causal claims that contain tentative language – could, may, seem,
suggest, sometimes, potentially – is still considered a causal claim
It must be established that two variables are correlated
It must be established that the causal variable came first, and
the outcome variable came later
It must be established that no other explanations exist for the
relationship
Validity: refers to the appropriateness of a conclusion or decision
Interrogating frequency claims > focus on construct, external, statistical validity
Construct validity: refers to how well a conceptual variable is operationalized
Asks how well a study measured or manipulated – operationalized – a
variable
To ensure construct validity, it must be established that each variable has
been measured reliably, and that different levels accurately correspond to
true differences
External validity/generalizability: concerns how participants were chosen and how
well they represent the intended population
Concerns how well the results of a study generalize to, or represent, people
or contexts beside those in the original study
Statistical validity: the extent to which a study’s statistical conclusions are accurate
and reasonable
Percentages reported in frequency claims are usually accompanied by a
margin of error estimate
Interrogating association claims > focus on construct, external, statistical validity
Construct validity: assess the construct validity of each variable
External validity: ask whether the claim can be generalized to other populations,
contexts, times, or places
Statistical validity: the extent to which the statistical conclusions are accurate and
reasonable
5
Chapter 1-14
Vrije Universiteit Amsterdam
Semester 1, Period 1
Chapter 1
Research producers and consumers: share commitment to empiricism
Producers:
Research scientists
Professors
Consumers:
Therapists
Teachers
Entrepreneurs
Counselors
Police officers
Evidence-based treatments: therapies that are supported by research
Empiricism: involves using evidence from the senses or from other instruments that
assist the senses as the basis for conclusions
Aim is to be systematic, rigorous and to make work independently verifiable
Theory: set of statements that describes general principles about how variables
relate to one another
Hypothesis or prediction: the specific outcome the researcher expects to observe in
a study if the theory is accurate
Most researchers test theories with a series of empirical studies, each
designed to test an individual hypothesis
Data: set of observations
The best theories are:
Supported by data from research studies
Multiple studies, variety of methods to address different aspects of
theories
Falsifiable: theory must lead to hypotheses that, when tested, could actually
fail support it
Parsimonious: when theories explain data equally well, scientists opt for the
simpler explanation
Theories don’t prove anything
Scientists evaluate theories based on the weight of the evidence, for and
against
Applied research: done with a practical problem in mind; conducted in a particular
real-world context
Basic research: aims to enhance the general body of knowledge
Generated knowledge may be applied to real-world issues later on
Translational research: use of lessons from basic research to develop and test
applications to health care, psychotherapy or other forms of treatment and
intervention
1
,Basic research > Translational research > Applied research
Research publication
Through scientific journals
Articles are peer-reviewed
Read primarily by other scientists and psychology students
Journalism: contrasts journals in that it includes the kinds of news and commentary
that is read or heard on television, magazines, newspapers and the internet
Not written by scientists, meant for general public, special education is not
required to understand their content
Journalists publishing psychological research may have benefits
Better understanding of psychologists’ work
Better understanding of self and environment > change of habits
However, story should be important and reported accurately
Chapter 2
Experience has no comparison group
Comparison group: enables us to compare what would happen both with and
without the thing we are interested in
Basing conclusions on personal experience is problematic because
daily life usually doesn’t include comparison experiences
Experience is confounded – even if a change occurs, there is no
certainty in what caused it
In contrast: basing conclusions on systematic data collection provides
a comparison group
Confounds: several possible explanations for an outcome in real-world situations
Occurs when you think one thing caused an outcome but in fact other things
changed too
Careful controls are used in a research setting to make sure only one
factor at a time is changed
Confederate: an actor playing a specific role for an experimenter
Behavioral research is probabilistic; its findings are not expected to explain all cases
of all time, but rather a certain proportion
Examples of biased reasoning:
Being swayed by a good story (catharsis, Scared Straight)
Being persuaded by what comes easily to mind
Availability heuristic: states that things that pop up easily in our mind
tend to guide our thinking, might lead us to wrongly estimate the
number of something or how often something happens
Failing to think about what we cannot see: when testing relationships, we
often fail to look for absences
Present/present bias: our failure to consider appropriate comparison
groups – Dr. Rush bloodletting study
2
, Instances where treatment was absent, but outcome was still present
are overlooked
The availability heuristic plays a role in the present/present bias
because instances in the present/present cell of a comparison stand
out
Present/present bias adds tendency to ignore absent cells, which
are essential for testing relationships
To avoid this bias, ask: “Compared to what?”
Focusing on the evidence we like best
Confirmation bias: tendency to look only at information that agrees
with what we already believe
Biased about being biased
Bias blind spot: the belief that we are unlikely to be susceptible to
biases previously described
Makes us trust out faulty reasoning even more
Can make it difficult for us to initiate the scientific theory-data
cycle
To be an empiricist:
Base believes on systematic information from the senses
Strive to interpret the data you collect in an objective way; guard against
common biases
Before taking advice from authorities:
Ask about the source of their ideas
Objective and systematic comparison of conditions?
Reference to research evidence? > if yes, attention worthy
Scientific sources
Journal articles
Empirical journal articles: report results of an empirical research study
Review journal articles: provide a summary of all the published studies
that have been done in one research area
Meta-analysis may be used: quantitative technique that
combines result of many studies and gives a number that
summarizes magnitude > effect size of a relationship
o Valued technique; doesn’t allow cherry-picking and
weighs each study proportionately
Chapters in edited books
Collection of chapters on a common topic; each written by a different
contributor
Not the first time a study is reported, rather a summary of research
and explanation of theory behind it
Good way to find a summary of a specific person’s work
Not as rigorously peer-reviewed as scientific sources
Psychologists and psychology students audience
Full-length books
Common in other disciplines but not so much in psychology
Finding scientific sources
3
, PsycINFO
Google Scholar
Components of an empirical journal article:
Abstract
Summary of the article describing hypotheses, method and major
results
Introduction
Explains topic of study, background for research and the study’s
specific research questions/goals/hypotheses
Method
Participants, materials, procedure and apparatus
Results
Tables and figures summarizing key results
Qualitative and qualitative data
Discussion
Summarizes and discusses the study’s importance, gives alternative
explanations for data and raises new questions
References
Reading with purpose
1. What is the argument?
Abstract, introduction, first paragraph of discussion
2. What is the evidence to support the argument?
Method and results
Wikis as a research source
Not comprehensive in their coverage
No comprehensive reference list
Details might be incorrect until they are fixed
Vandalism is a potential problem
Chapter 3
Variable: something that varies, and which must at least have two levels/values
Constant: something that could potentially vary but has only one level in the study
in question
Measured variable: one whose levels are simply observed and recorded
Manipulated variable: variable a researcher controls, usually by assigning study
participants to different levels of that variable
Some variables can be either measured or manipulated – hair color
Conceptual variables/constructs: abstract concepts defined at the theoretical level
by conceptual definitions
Operational variables/operationalizations: operational definitions of variables
created by testing hypotheses with empirical research
Claim: argument someone is trying to make
Frequency claims: describe a particular rate or degree of a single variable
Variables are always measured, not manipulated
4
, Association claims: argue that one level of a variable is likely to be associated
with a particular level of another variable
Variables that are associated are said to correlate or to be related
When one variable changes, the other tends to change
Correlational study: study in which variables are measured ant the
relationship between them is tested
Positive association/correlation: in which high goes with high,
and low goes with low > scatterplot can be used to represent an
association
Negative association/correlation: high goes with low, and low
goes with high
Zero association/correlation
Causal claims: argue that one of the variables is responsible for changing the
other
Have two variables
Start with a positive or negative association
Causal verbs tend to be more active and forceful, suggesting one
variable comes first in time and then acts on the other
Causal claims that contain tentative language – could, may, seem,
suggest, sometimes, potentially – is still considered a causal claim
It must be established that two variables are correlated
It must be established that the causal variable came first, and
the outcome variable came later
It must be established that no other explanations exist for the
relationship
Validity: refers to the appropriateness of a conclusion or decision
Interrogating frequency claims > focus on construct, external, statistical validity
Construct validity: refers to how well a conceptual variable is operationalized
Asks how well a study measured or manipulated – operationalized – a
variable
To ensure construct validity, it must be established that each variable has
been measured reliably, and that different levels accurately correspond to
true differences
External validity/generalizability: concerns how participants were chosen and how
well they represent the intended population
Concerns how well the results of a study generalize to, or represent, people
or contexts beside those in the original study
Statistical validity: the extent to which a study’s statistical conclusions are accurate
and reasonable
Percentages reported in frequency claims are usually accompanied by a
margin of error estimate
Interrogating association claims > focus on construct, external, statistical validity
Construct validity: assess the construct validity of each variable
External validity: ask whether the claim can be generalized to other populations,
contexts, times, or places
Statistical validity: the extent to which the statistical conclusions are accurate and
reasonable
5
