Chapter 3: Three Claims, Four Validities: Interrogation Tools for Consumers of
Research
Learning objectives
1. Differentiate the three types of claims: frequency, association, and causal
2. Ask appropriate questions to help you interrogate each of the four big validities: construct validity,
statistical validity, external validity, and internal validity
3. Explain which validities are most relevant for each of the three types of claims
4. State which kind of research study is required to support a causal claim
Types of claims
Variable = an attribute that varies, having at least two levels (values)
o Constant = an attribute that could potentially vary but that has only one level in the
study in question
o Measured variable / dependent variable = levels are observed and recorded as an
observation, a statement, or a value as it occurs naturally
o Manipulated variable / independent variable = researcher controls and manipulates
which means that participants could end up at any of the levels
Variables that cannot be manipulated that can only be measured
Age
IQ
Variables that cannot be manipulated due to ethics
Trauma experience
Variables that can be manipulated and/or measured
Music lessons
Meditation
1. Construct / conceptual variable = a variable of interest, stated at an abstract level, usually
defined as a part of a formal statement of a psychological theory
Discussing theories through the use of abstract names
A researcher's definition of a variable at the theoretical level
2. Operational definition / operational variable / operationalizations = the specific way in
which a concept of interest is measured or manipulated as a variable in a study
When testing hypotheses with empirical research
Operationalize = to turn a conceptual definition of a variable into a specific
measured variable or manipulated variable in order to conduct a research study
Claim = the argument a journalist, researcher, or scientist is trying to make
o Researchers make claims about theories based on data
1) Frequency claim = a claim that describes a particular rate or degree of a single
variable
Claims about how frequent or common something is
Mentions a percentage, number, or an average
Focus on one variable
Always measured variables (dependent variables)
21% of Canadian Adolescents ages 15-17 reported vaping in the last
month
The global average life expectancy is 73 years
About six in ten black adults say racism is an extremely big problem
for black people in the U.S. today
20% believe the COVID-19 vaccine contains a microchip
Few transgender children change their mind after 5 years study finds
o Observational study
Construct validity
How well did the researchers measure the variable of interest?
,External validity
How did the researchers choose the study's participants?
How well do the participants represent the intended population?
Statistical validity
How much? (i.e., what is the point estimate?)
How precise? (i.e., what is the confidence interval?)
What else is known? (i.e., has the study been replicated?)
2) Association claim = a claim about two variables, in which the value (level) one
variable is said to vary systematically with the value of another variable
o Argues that one level of a variable is likely to be associated with a particular level of
another variable
o Correlate / covary = to occur or vary together systematically, as in the case of two
variables
Variables that are associated/related, meaning that when one variable
changes, the other variable tends to change, too
Positive association / positive correlation = an association in which high
levels of one variable go with high levels of the other variable, and low levels
of one variable go with low levels of the other variable
Negative association / negative correlation / inverse association = an
association in which high levels of one variable go with low levels of the
other variable, and vice versa
Zero association / zero correlation = a lack of systematic association
between two variables
Through a positive or negative association of one variable one can more
accurately predict the level of the other variable (in a mathematical sense)
Using association to make estimates more accurate
o Two variables go together but they do not claim to have a causal relationship
o States a relationship between at least two variables
o Words used: link, associate, predict, be at risk for, people who…. also do …
Regular coffee consumption associated with lower depression risk
Playing an instrument is linked to better cognition
Parents attached to digital devices are more likely to yell at their kids
Gaming time has no link to well-being, study finds
o Correlational study = a study that includes two or more variables, in which all of the
variables are measured
Variables are measured and the relationship between them is tested
o Scatterplot = a graphical representation of an association, in which each dot
represents one participant in the study measured on two variables
Construct validity
How well did the researchers measure the dependent variable?
How well did the researchers measure the independent variable?
External validity
Can is be generalized to other populations?
Can it be generalized to other contexts, times, or places?
Statistical validity
How strong is the estimated association?
How precise is the estimated association? (what is the confidence interval?)
Has the study been replicated?
3) Causal claim = a claim arguing that a specific change in one variable is responsible for
influencing the value of another variable
o Argues that one of the variables is responsible for changing the other variable(s)
, o States a causal relationship between at least two variables
o Causal claims can be based on zero association that reports a lack of cause (e.g.
vaccines do not cause autism / being in daycare does not cause behavior problems)
o Words used: cause, enhance, affect, decrease, change, could, may, seem, suggest,
sometimes, potentially, advice, make, influence
Baby's superpowered scent can affect parents' moods, researchers find
Talking to strangers might make you happier
Logic's song 1-800-273-8255 may have led to hundreds of fewer suicides
Pretending to be Batman helps kids stay on task
To appear more intimidating, just tilt your head down, study suggests
Family meals curb teen eating disorders
o Covariance = the extent to which two variables are observed to go together
established by the results of a study
Results must establish that the variables (causal variable and outcome
variable) are correlated (cannot be zero)
o Temporal precedence = the proposed causal variable comes first in time, before the
proposed outcome variable
Method of the study must ensure that the causal variable came first and the
outcome variable came later
o Internal validity / third-variable criterion = refers to a study's ability to eliminate
alternative explanations for the association
Method of the study must establish that no other explanations exist for the
relationship
o Experiment = a study in which at least one variable is manipulated (independent
variable) and another is measured (dependent variable)
Ensures temporal precedence
Controls alternative explanations and therefore ensures internal validity
Random assingment = the use of a random method to assign
participants into different experimental groups
Increases internal validity by allowing the researchers to
control for many alternative explanations
Construct validity
How well have researchers measured or manipulated the variables in the study?
Statistical validity
What is the estimated effect size?
How large is the difference between groups?
How precise is the estimate?
What is the confidence interval?
Has the study been replicated?
External validity
To what populations, settings, and times can the causal claim be generalized?
How representative is the sample?
How representative are the manipulations and measures?
Internal validity
Was the study an experiment?
Does the study achieve temporal precedence?
Does the study control for alternative explanations by randomly assigning participants to
groups?
Does the study avoid internal validity threats?
Types of validities
Validity = the appropriateness of a conclusion or decision
Research
Learning objectives
1. Differentiate the three types of claims: frequency, association, and causal
2. Ask appropriate questions to help you interrogate each of the four big validities: construct validity,
statistical validity, external validity, and internal validity
3. Explain which validities are most relevant for each of the three types of claims
4. State which kind of research study is required to support a causal claim
Types of claims
Variable = an attribute that varies, having at least two levels (values)
o Constant = an attribute that could potentially vary but that has only one level in the
study in question
o Measured variable / dependent variable = levels are observed and recorded as an
observation, a statement, or a value as it occurs naturally
o Manipulated variable / independent variable = researcher controls and manipulates
which means that participants could end up at any of the levels
Variables that cannot be manipulated that can only be measured
Age
IQ
Variables that cannot be manipulated due to ethics
Trauma experience
Variables that can be manipulated and/or measured
Music lessons
Meditation
1. Construct / conceptual variable = a variable of interest, stated at an abstract level, usually
defined as a part of a formal statement of a psychological theory
Discussing theories through the use of abstract names
A researcher's definition of a variable at the theoretical level
2. Operational definition / operational variable / operationalizations = the specific way in
which a concept of interest is measured or manipulated as a variable in a study
When testing hypotheses with empirical research
Operationalize = to turn a conceptual definition of a variable into a specific
measured variable or manipulated variable in order to conduct a research study
Claim = the argument a journalist, researcher, or scientist is trying to make
o Researchers make claims about theories based on data
1) Frequency claim = a claim that describes a particular rate or degree of a single
variable
Claims about how frequent or common something is
Mentions a percentage, number, or an average
Focus on one variable
Always measured variables (dependent variables)
21% of Canadian Adolescents ages 15-17 reported vaping in the last
month
The global average life expectancy is 73 years
About six in ten black adults say racism is an extremely big problem
for black people in the U.S. today
20% believe the COVID-19 vaccine contains a microchip
Few transgender children change their mind after 5 years study finds
o Observational study
Construct validity
How well did the researchers measure the variable of interest?
,External validity
How did the researchers choose the study's participants?
How well do the participants represent the intended population?
Statistical validity
How much? (i.e., what is the point estimate?)
How precise? (i.e., what is the confidence interval?)
What else is known? (i.e., has the study been replicated?)
2) Association claim = a claim about two variables, in which the value (level) one
variable is said to vary systematically with the value of another variable
o Argues that one level of a variable is likely to be associated with a particular level of
another variable
o Correlate / covary = to occur or vary together systematically, as in the case of two
variables
Variables that are associated/related, meaning that when one variable
changes, the other variable tends to change, too
Positive association / positive correlation = an association in which high
levels of one variable go with high levels of the other variable, and low levels
of one variable go with low levels of the other variable
Negative association / negative correlation / inverse association = an
association in which high levels of one variable go with low levels of the
other variable, and vice versa
Zero association / zero correlation = a lack of systematic association
between two variables
Through a positive or negative association of one variable one can more
accurately predict the level of the other variable (in a mathematical sense)
Using association to make estimates more accurate
o Two variables go together but they do not claim to have a causal relationship
o States a relationship between at least two variables
o Words used: link, associate, predict, be at risk for, people who…. also do …
Regular coffee consumption associated with lower depression risk
Playing an instrument is linked to better cognition
Parents attached to digital devices are more likely to yell at their kids
Gaming time has no link to well-being, study finds
o Correlational study = a study that includes two or more variables, in which all of the
variables are measured
Variables are measured and the relationship between them is tested
o Scatterplot = a graphical representation of an association, in which each dot
represents one participant in the study measured on two variables
Construct validity
How well did the researchers measure the dependent variable?
How well did the researchers measure the independent variable?
External validity
Can is be generalized to other populations?
Can it be generalized to other contexts, times, or places?
Statistical validity
How strong is the estimated association?
How precise is the estimated association? (what is the confidence interval?)
Has the study been replicated?
3) Causal claim = a claim arguing that a specific change in one variable is responsible for
influencing the value of another variable
o Argues that one of the variables is responsible for changing the other variable(s)
, o States a causal relationship between at least two variables
o Causal claims can be based on zero association that reports a lack of cause (e.g.
vaccines do not cause autism / being in daycare does not cause behavior problems)
o Words used: cause, enhance, affect, decrease, change, could, may, seem, suggest,
sometimes, potentially, advice, make, influence
Baby's superpowered scent can affect parents' moods, researchers find
Talking to strangers might make you happier
Logic's song 1-800-273-8255 may have led to hundreds of fewer suicides
Pretending to be Batman helps kids stay on task
To appear more intimidating, just tilt your head down, study suggests
Family meals curb teen eating disorders
o Covariance = the extent to which two variables are observed to go together
established by the results of a study
Results must establish that the variables (causal variable and outcome
variable) are correlated (cannot be zero)
o Temporal precedence = the proposed causal variable comes first in time, before the
proposed outcome variable
Method of the study must ensure that the causal variable came first and the
outcome variable came later
o Internal validity / third-variable criterion = refers to a study's ability to eliminate
alternative explanations for the association
Method of the study must establish that no other explanations exist for the
relationship
o Experiment = a study in which at least one variable is manipulated (independent
variable) and another is measured (dependent variable)
Ensures temporal precedence
Controls alternative explanations and therefore ensures internal validity
Random assingment = the use of a random method to assign
participants into different experimental groups
Increases internal validity by allowing the researchers to
control for many alternative explanations
Construct validity
How well have researchers measured or manipulated the variables in the study?
Statistical validity
What is the estimated effect size?
How large is the difference between groups?
How precise is the estimate?
What is the confidence interval?
Has the study been replicated?
External validity
To what populations, settings, and times can the causal claim be generalized?
How representative is the sample?
How representative are the manipulations and measures?
Internal validity
Was the study an experiment?
Does the study achieve temporal precedence?
Does the study control for alternative explanations by randomly assigning participants to
groups?
Does the study avoid internal validity threats?
Types of validities
Validity = the appropriateness of a conclusion or decision
