SJSU Kin 175 Exam 2 Topics:
Psychometrics questions and
answers
What does it mean for data to be clean?✔✔The data reflects what it is supposed to.
What does it mean for data to be relevant?✔✔The data is applicable given the
context and purpose of data collection and is appropriate in terms of the particular
group the data is being obtained from.
What does it mean for data to be reliable?✔✔The data produced is accurate.
What does it mean for data to be valid?✔✔Data is clean and relevant.
What are logical and content validity in the context of scores?✔✔Cognitive (content)
and motor (logical) validity are measured qualitatively (through a ToS) to ensure
tests contain appropriate and relevant components for a given topic(s).
What is a Table of Specifications (TOS)?✔✔A chart used to provide qualitative
evidence of content or logical validity.
What is concurrent validity in the context of scores?✔✔The quantitative evidence of
the validity of scores through correlation of a set of scores with a criterion measure of
the same skill/knowledge.
What is predictive validity?✔✔Evidence of whether or not a given score can be a
good predictor of another score/performance.
What is construct validity?✔✔Examining construct validity ensures that measures of
intangible characteristics (stress, happiness, honesty) are clean and relevant. It is
examined through a systematic process of correlating your measure (x) of the
characteristic of interest with similar and dissimilar measures (y's). Good construct
validity is when correlation coefficients match expectations (low when correlated with
dissimilar measures and high when correlated with similar measures).
What is the definition of internal consistency in the context of scores?✔✔The
consistency of scores collected multiple times on the same day.
What does SEM stand for and what is it used for?✔✔SEM stands for Standard Error
of Measurement and it is used for quantitatively estimating measurement error.
What is the Spearman Brown Prophecy formula?✔✔The Spearman Brown Prophecy
formula is used to predict how reliability will be affected by a shortened/lengthened
test.
, What is a false negative?✔✔Incorrectly categorizing a master as a non-master.
What is objectivity used to measure?✔✔The accuracy and consistency of scores
obtained through subjective observations/scoring of performance. To examine
consistency across multiple observations, inter-rater consistency can be measured
using multiple judges on one occasion. Intra-rater consistency can be measured by
having one evaluator assess the same performance multiple times.
What is Berk's approach?✔✔A visual approach to determining an appropriate cut
score. Obtain histograms to visually find the point at which the scores would produce
the fewest improper classifications (false negatives and false positives).
What is a false positive?✔✔Incorrectly classifying a non-master as a master.
What is concurrent validity in the context of classifications?✔✔In the context of
mastery test classifications, concurrent validity is determined by examining the
extent to which a cut score results in the correct classification of participants into
mastery categories. Concurrent validity is examined through correlating the results of
your classifications (x) with a criterion measure (y) of the same skill/knowledge.
What is a cut score?✔✔The score set for a mastery test. It is the point at which
classification status (master/nonmaster) is determined.
What are logical and content validity in the context of classifications?✔✔Qualitative
evidence of validity used in both the cognitive (content) and motor (logical) domains
to ensure a test contains appropriate components for the classification of people into
master and nonmaster categories.
What is the definition of stability in the context of classifications?✔✔The consistency
of classifications over time.
What are the three questions that must be asked regarding logical and content
validity of scores and why?✔✔a. Was the emphasis within each area appropriate?
b. Was appropriate content included?
c. Was any important content excluded?
The relevance component of validity must be reviewed in the context of the broader
area of study and, more specifically, within the test itself and what it is supposed to
cover.
If data (scores) are found to be valid for one group, will it then be automatically valid
for another group? Why or why not?✔✔No. Validity in one group does not translate
to other groups. Data can only be deemed clean and relevant in/for the group you
have evaluated.
What is the difference between content validity and logical validity (of
scores)?✔✔Content validity is specifically used for measures in the cognitive domain
(e.g., from written tests). Logical validity is specifically used for measures in the
motor domain (e.g., motor skills tests).
Psychometrics questions and
answers
What does it mean for data to be clean?✔✔The data reflects what it is supposed to.
What does it mean for data to be relevant?✔✔The data is applicable given the
context and purpose of data collection and is appropriate in terms of the particular
group the data is being obtained from.
What does it mean for data to be reliable?✔✔The data produced is accurate.
What does it mean for data to be valid?✔✔Data is clean and relevant.
What are logical and content validity in the context of scores?✔✔Cognitive (content)
and motor (logical) validity are measured qualitatively (through a ToS) to ensure
tests contain appropriate and relevant components for a given topic(s).
What is a Table of Specifications (TOS)?✔✔A chart used to provide qualitative
evidence of content or logical validity.
What is concurrent validity in the context of scores?✔✔The quantitative evidence of
the validity of scores through correlation of a set of scores with a criterion measure of
the same skill/knowledge.
What is predictive validity?✔✔Evidence of whether or not a given score can be a
good predictor of another score/performance.
What is construct validity?✔✔Examining construct validity ensures that measures of
intangible characteristics (stress, happiness, honesty) are clean and relevant. It is
examined through a systematic process of correlating your measure (x) of the
characteristic of interest with similar and dissimilar measures (y's). Good construct
validity is when correlation coefficients match expectations (low when correlated with
dissimilar measures and high when correlated with similar measures).
What is the definition of internal consistency in the context of scores?✔✔The
consistency of scores collected multiple times on the same day.
What does SEM stand for and what is it used for?✔✔SEM stands for Standard Error
of Measurement and it is used for quantitatively estimating measurement error.
What is the Spearman Brown Prophecy formula?✔✔The Spearman Brown Prophecy
formula is used to predict how reliability will be affected by a shortened/lengthened
test.
, What is a false negative?✔✔Incorrectly categorizing a master as a non-master.
What is objectivity used to measure?✔✔The accuracy and consistency of scores
obtained through subjective observations/scoring of performance. To examine
consistency across multiple observations, inter-rater consistency can be measured
using multiple judges on one occasion. Intra-rater consistency can be measured by
having one evaluator assess the same performance multiple times.
What is Berk's approach?✔✔A visual approach to determining an appropriate cut
score. Obtain histograms to visually find the point at which the scores would produce
the fewest improper classifications (false negatives and false positives).
What is a false positive?✔✔Incorrectly classifying a non-master as a master.
What is concurrent validity in the context of classifications?✔✔In the context of
mastery test classifications, concurrent validity is determined by examining the
extent to which a cut score results in the correct classification of participants into
mastery categories. Concurrent validity is examined through correlating the results of
your classifications (x) with a criterion measure (y) of the same skill/knowledge.
What is a cut score?✔✔The score set for a mastery test. It is the point at which
classification status (master/nonmaster) is determined.
What are logical and content validity in the context of classifications?✔✔Qualitative
evidence of validity used in both the cognitive (content) and motor (logical) domains
to ensure a test contains appropriate components for the classification of people into
master and nonmaster categories.
What is the definition of stability in the context of classifications?✔✔The consistency
of classifications over time.
What are the three questions that must be asked regarding logical and content
validity of scores and why?✔✔a. Was the emphasis within each area appropriate?
b. Was appropriate content included?
c. Was any important content excluded?
The relevance component of validity must be reviewed in the context of the broader
area of study and, more specifically, within the test itself and what it is supposed to
cover.
If data (scores) are found to be valid for one group, will it then be automatically valid
for another group? Why or why not?✔✔No. Validity in one group does not translate
to other groups. Data can only be deemed clean and relevant in/for the group you
have evaluated.
What is the difference between content validity and logical validity (of
scores)?✔✔Content validity is specifically used for measures in the cognitive domain
(e.g., from written tests). Logical validity is specifically used for measures in the
motor domain (e.g., motor skills tests).
