Types of Claims
➔ validity: the extent to which the evidence matchesthe claim
◆ 4 types
● external
● internal
● construct
● statistical
➔ external validity: looks at whether the sample isgeneralizable to the population of interest
(representative of the pop.)
◆ looks atsampling method
● describes how the sample was identified and acquired
◆ representative samples
● ideally generalizable to the population
◆ can be established through generalizability across
● people(population validity): the extent to whichthe results can be generalized
to a specific and/or diverse population of interest
● contexts(e
cological validity/mundane reality): theextent to which conditions of
a study generalize to the real-world settings and context
● psychological processes(experimental realism/ psychologicalrealism): extent to
which the results can be generalized to capture a meaningful psychological
processes
● studies(c onceptual replication): extent to whichthe results of the study can be
reproduced across diff samples/populations of people across diff
regions/contexts
➔ construct validity: looks at whether the variablesare appropriately operationalized in terms
of the constructs of interest
◆ operationalizationdescribes how [exactly] the variablesare measured, observed, or
manipulated
◆ the real world phenomenon, characteristic, or attribute the researcher is trying to
make conclusions about is theconstruct
➔ internal validity: looks at whether the research claimvalidates a causal claim
◆ looks into whether a cause and effect relationship was tested by the experiment
◆ aims to establishcausalitywithout the possibilityof aconfound(a third variable that
could impact the relationship between the IV and DV)
➔ statistical validity: looking at whether the conclusionis reasonable given the actual data
◆ bolder claims = need for stronger evidence
◆ aresearch claimis a conclusion that’s made basedon the results of the study
➔ 3 conditions for causality
◆ covariance- the IV must be associated with the changesin the DV
◆ temporal precedence- the IV must trigger the DV (notthe other way around)
◆ internal validity- no confounds exist
➔ key features of anexperimental design
, ◆ the IV is manipulated
● 2 or more conditions to manipulate the IV and observe the different (or not
different) impacts
○ i.e. treatment groups and control group
◆ random assignment
● randomly assigning participants to a condition purely based on chance
◆ experimental control
● everything in the study, excluding the IV, is held constant so any
differences/changes in the DV can be linked to the IV
◆ meets all 3 conditions of casualty
➔ commonresearch designs
◆ descriptive research studies: a representative sampleis used to measure key variables
for the purpose of describing the nature of the variables of interest within the
population
● ideal for frequency claims
● don’t support causal claims
◆ correlational research studies: measuring multiplevariables to examine if there’s an
association between them
● ideal for associational claims
● don’t support causal claims
◆ experimental research designs: manipulating the independentvariable(s) to examine
its impacts on the DV
● supports causal claims
➔ tends to be trade offs between internal validity and external validity
◆ low external, low internal
● good for initial exploration and theory testing (where to look)
◆ high external, low internal
● good for describing populations and making real world predictions
◆ low external, high internal
● good for testing theories about causal patterns and exploring new treatments
◆ high external, high internal
● the “gold standard” of treatment and intervention,but rare and expensive
➔ primary source vs secondary source
◆ primary source: written by the producers of the research
● reflects original source of information about the study
◆ secondary source: written by the consumers of theresearch
● purpose
○ reviewing research and theory [literature reviews, student term papers]
○ reporting media to the general public [journalists, book authors, blog
writers, etc.]
Evaluating Frequency Claims
➔ external validity is key
➔ validity: the extent to which the evidence matchesthe claim
◆ 4 types
● external
● internal
● construct
● statistical
➔ external validity: looks at whether the sample isgeneralizable to the population of interest
(representative of the pop.)
◆ looks atsampling method
● describes how the sample was identified and acquired
◆ representative samples
● ideally generalizable to the population
◆ can be established through generalizability across
● people(population validity): the extent to whichthe results can be generalized
to a specific and/or diverse population of interest
● contexts(e
cological validity/mundane reality): theextent to which conditions of
a study generalize to the real-world settings and context
● psychological processes(experimental realism/ psychologicalrealism): extent to
which the results can be generalized to capture a meaningful psychological
processes
● studies(c onceptual replication): extent to whichthe results of the study can be
reproduced across diff samples/populations of people across diff
regions/contexts
➔ construct validity: looks at whether the variablesare appropriately operationalized in terms
of the constructs of interest
◆ operationalizationdescribes how [exactly] the variablesare measured, observed, or
manipulated
◆ the real world phenomenon, characteristic, or attribute the researcher is trying to
make conclusions about is theconstruct
➔ internal validity: looks at whether the research claimvalidates a causal claim
◆ looks into whether a cause and effect relationship was tested by the experiment
◆ aims to establishcausalitywithout the possibilityof aconfound(a third variable that
could impact the relationship between the IV and DV)
➔ statistical validity: looking at whether the conclusionis reasonable given the actual data
◆ bolder claims = need for stronger evidence
◆ aresearch claimis a conclusion that’s made basedon the results of the study
➔ 3 conditions for causality
◆ covariance- the IV must be associated with the changesin the DV
◆ temporal precedence- the IV must trigger the DV (notthe other way around)
◆ internal validity- no confounds exist
➔ key features of anexperimental design
, ◆ the IV is manipulated
● 2 or more conditions to manipulate the IV and observe the different (or not
different) impacts
○ i.e. treatment groups and control group
◆ random assignment
● randomly assigning participants to a condition purely based on chance
◆ experimental control
● everything in the study, excluding the IV, is held constant so any
differences/changes in the DV can be linked to the IV
◆ meets all 3 conditions of casualty
➔ commonresearch designs
◆ descriptive research studies: a representative sampleis used to measure key variables
for the purpose of describing the nature of the variables of interest within the
population
● ideal for frequency claims
● don’t support causal claims
◆ correlational research studies: measuring multiplevariables to examine if there’s an
association between them
● ideal for associational claims
● don’t support causal claims
◆ experimental research designs: manipulating the independentvariable(s) to examine
its impacts on the DV
● supports causal claims
➔ tends to be trade offs between internal validity and external validity
◆ low external, low internal
● good for initial exploration and theory testing (where to look)
◆ high external, low internal
● good for describing populations and making real world predictions
◆ low external, high internal
● good for testing theories about causal patterns and exploring new treatments
◆ high external, high internal
● the “gold standard” of treatment and intervention,but rare and expensive
➔ primary source vs secondary source
◆ primary source: written by the producers of the research
● reflects original source of information about the study
◆ secondary source: written by the consumers of theresearch
● purpose
○ reviewing research and theory [literature reviews, student term papers]
○ reporting media to the general public [journalists, book authors, blog
writers, etc.]
Evaluating Frequency Claims
➔ external validity is key
