Furr, & Bacharach (2014) – Psychometrics An introduction
Chapter 8 – Validity: Conceptual Basis
What is validity?
- Degree to which a test measures what it’s supposed to measure, OR better: degree to
which evidence + theory support the interpretations of test scores entailed by
proposed uses of a test.
- Measure itself is neither valid nor invalid: validity concerns interpretations + uses of
measure’s score: set of items and scores are neither valid nor invalid, however, the
author’s interpretations of scores might be.
o Validity is related to the ‘proposed uses’ of scores: e.g., do scores reflect the
psychological construct, or is this construct truly predictive of certain quality.
o Simplistic + somewhat inaccurate to draw conclusions of a score without
regard to the way in which it will be used (e.g., hammer is useful tool, but not
for cleaning).
o Some people talk about the validity of a test:
Don’t have sophisticated understanding of validity.
Used as a shortcut: shorter to say.
- Validity is a matter of degree: it’s not an ‘all-or-none’ issue interpretation as strong
vs. weak, rather than valid vs. invalid. Test should only be selected if there’s strong
enough evidence supporting the intended interpretation + use.
- Validity of test’s interpretation is based on empirical evidence + theory: not just based
on experience or opinions, but from high-quality research, which must provide
evidence for particular interpretations of test scores.
- Traditional perspective on validity: just 3 types of validity content validity, criterion
validity + construct validity (degree to which test scores can be interpreted as
reflecting a particular psychological construct).
- Construct validity depends on content of test content, internal structure of test,
psychological processes used in test responses, association among test scores and
other variables + consequences of test use = contemporary view.
The importance of validity
- Ability to interpret huge amount of research hinges on test validity: if key variable is
measured with good validity, then increased confidence in conclusion drawn
validity affects accuracy of understanding of world.
- Without test validity, decisions about societal issues could be misinformed, wasteful,
or harmful (e.g., legislation made based on non-valid research).
- Without test validity, test-based decisions about individuals could be
misinformed/harmful (e.g., placing children in specific classes where they actually
don’t belong).
Validity evidence: test content
- Match between actual content of test + content that should be included in the test;
content of test should reflect important facets of construct.
- Threats to conent validity:
o When a test includes construct-irrelevant content: content that’s extraneous to
core construct for which a test is to be interpreted + its inclusion would reduce
validity.
1
Chapter 8 – Validity: Conceptual Basis
What is validity?
- Degree to which a test measures what it’s supposed to measure, OR better: degree to
which evidence + theory support the interpretations of test scores entailed by
proposed uses of a test.
- Measure itself is neither valid nor invalid: validity concerns interpretations + uses of
measure’s score: set of items and scores are neither valid nor invalid, however, the
author’s interpretations of scores might be.
o Validity is related to the ‘proposed uses’ of scores: e.g., do scores reflect the
psychological construct, or is this construct truly predictive of certain quality.
o Simplistic + somewhat inaccurate to draw conclusions of a score without
regard to the way in which it will be used (e.g., hammer is useful tool, but not
for cleaning).
o Some people talk about the validity of a test:
Don’t have sophisticated understanding of validity.
Used as a shortcut: shorter to say.
- Validity is a matter of degree: it’s not an ‘all-or-none’ issue interpretation as strong
vs. weak, rather than valid vs. invalid. Test should only be selected if there’s strong
enough evidence supporting the intended interpretation + use.
- Validity of test’s interpretation is based on empirical evidence + theory: not just based
on experience or opinions, but from high-quality research, which must provide
evidence for particular interpretations of test scores.
- Traditional perspective on validity: just 3 types of validity content validity, criterion
validity + construct validity (degree to which test scores can be interpreted as
reflecting a particular psychological construct).
- Construct validity depends on content of test content, internal structure of test,
psychological processes used in test responses, association among test scores and
other variables + consequences of test use = contemporary view.
The importance of validity
- Ability to interpret huge amount of research hinges on test validity: if key variable is
measured with good validity, then increased confidence in conclusion drawn
validity affects accuracy of understanding of world.
- Without test validity, decisions about societal issues could be misinformed, wasteful,
or harmful (e.g., legislation made based on non-valid research).
- Without test validity, test-based decisions about individuals could be
misinformed/harmful (e.g., placing children in specific classes where they actually
don’t belong).
Validity evidence: test content
- Match between actual content of test + content that should be included in the test;
content of test should reflect important facets of construct.
- Threats to conent validity:
o When a test includes construct-irrelevant content: content that’s extraneous to
core construct for which a test is to be interpreted + its inclusion would reduce
validity.
1
