Furr, & Bacharach (2014) – Psychometrics An introduction
Chapter 10 – Response biases
Important because can harm the psychometric quality of many types of tests, scales +
inventories can diminish reliability + validity, which in turns affects interpretation of
research.
Types of response biases
- Acquiescence bias (“yea-saying + nay-saying”): occurs when an individual agrees with
statements without regard for meaning of those statements. Usually in one direction.
o Often found on personality trait inventories, attitude questionnaires, interest
inventories, clinical inventories + marketing surveys.
o Usually with Likert-scale; sum is total score having highest score on
questionnaire = having highest level of construct?
o Phrasing items important: must be all in same direction (positive/agreement or
negative/disagreement).
o Implications for test users:
If some have bias, others not: test users might not be able to detect
which respondents have high level of construct + which respondents
are simply responding with bias misinformed + misguided decisions.
Imply that there’s a correlation between 2 constructs, while there’s
actually none only look at correlations among only those
participants who responded validly (because otherwise high artificially
correlation possible).
If multiple tests are ‘contaminated’ by the bias, then the tests will be more
strongly correlated with each other than are the underlying constructs
(because acquiescent = scoring relatively high on both tests).
o Also ‘nay-saying’ bias: disagreeing with statement. Similar effects as ‘yeah-
saying’ disagree on both tests = artificially more positive correlations.
o Bias seems to occur most often when respondents don’t easily understand test
items.
- Extreme + moderate responding: responses reflecting different degrees of intensity/
endorsement/occurrence (e.g., ‘often’, or ‘almost always’. Problem is the differences
in tendency to use/avoid extreme response options (even when having same true
level of construct).
o Implications:
Extremity bias ambiguity in respondents’ scores decision makers
might make inappropriate decisions.
Inaccurate conclusions about correlations can generate artificial
differences among respondents’ test scores.
Can obscure true differences among respondents’ construct levels: e.g.,
identical test scores, but different true trait levels because one of
participants is reluctant to use more extreme response option test
score isn’t as high as trait level.
Can lead to inaccurate research conclusions.
o Use of extreme/moderate response options is itself not bias/problem, if it
reflects an individual’s true trait level (i.e., person is truly extreme).
o Problems arise when:
1
Chapter 10 – Response biases
Important because can harm the psychometric quality of many types of tests, scales +
inventories can diminish reliability + validity, which in turns affects interpretation of
research.
Types of response biases
- Acquiescence bias (“yea-saying + nay-saying”): occurs when an individual agrees with
statements without regard for meaning of those statements. Usually in one direction.
o Often found on personality trait inventories, attitude questionnaires, interest
inventories, clinical inventories + marketing surveys.
o Usually with Likert-scale; sum is total score having highest score on
questionnaire = having highest level of construct?
o Phrasing items important: must be all in same direction (positive/agreement or
negative/disagreement).
o Implications for test users:
If some have bias, others not: test users might not be able to detect
which respondents have high level of construct + which respondents
are simply responding with bias misinformed + misguided decisions.
Imply that there’s a correlation between 2 constructs, while there’s
actually none only look at correlations among only those
participants who responded validly (because otherwise high artificially
correlation possible).
If multiple tests are ‘contaminated’ by the bias, then the tests will be more
strongly correlated with each other than are the underlying constructs
(because acquiescent = scoring relatively high on both tests).
o Also ‘nay-saying’ bias: disagreeing with statement. Similar effects as ‘yeah-
saying’ disagree on both tests = artificially more positive correlations.
o Bias seems to occur most often when respondents don’t easily understand test
items.
- Extreme + moderate responding: responses reflecting different degrees of intensity/
endorsement/occurrence (e.g., ‘often’, or ‘almost always’. Problem is the differences
in tendency to use/avoid extreme response options (even when having same true
level of construct).
o Implications:
Extremity bias ambiguity in respondents’ scores decision makers
might make inappropriate decisions.
Inaccurate conclusions about correlations can generate artificial
differences among respondents’ test scores.
Can obscure true differences among respondents’ construct levels: e.g.,
identical test scores, but different true trait levels because one of
participants is reluctant to use more extreme response option test
score isn’t as high as trait level.
Can lead to inaccurate research conclusions.
o Use of extreme/moderate response options is itself not bias/problem, if it
reflects an individual’s true trait level (i.e., person is truly extreme).
o Problems arise when:
1
