Task 1: Do you know what you measure?
1. Validity
• The degree to which an instrument truly measures the construct(s) it purports to measure →
measure what you want to measure
• Three different types:
o Content validity → focuses on whether the content of the instrument corresponds with
the construct that one intends to measure, with regard to relevance and
comprehensiveness
o Criterion validity, applicable in situations in which there is a gold standard for the
construct to be measured → refers to how well the scores of the measurement
instrument agree with the scores on the gold standard
o Construct validity, applicable in situations in which there is no gold standard → refers to
whether the instrument provides the expected scores, based on existing knowledge
about the construct
• Several subtypes of validity exist within these three main types.
1.1. The concept of validity
Validation consists of testing hypotheses, which are based on theories about the construct. The
scores of the instrument should be consistent with a theoretical model of the construct.
Knowledge about the construct to be measured
• We can only assess whether a measurement instrument measures what it purports to measure if
researchers have clearly described the construct they intend to measure.
• Subsequently, we have to formulate hypotheses about what scores we expect to find on the
measurement instruments, based on our knowledge of the construct.
• Therefore, detailed knowledge of the construct and a conceptual model to hypothesize
relationships with other constructs are indispensable for a sound validation process.
Complexity of the construct
• A unidimensional construct is often easier to validate than a multidimensional construct.
• When using a multidimensional instrument, each scale or each part of the instrument that
measures a specific dimension should be validated, by formulating hypotheses for each
dimension separately.
Dependency on the situation
• A measurement instrument should be validated again if it is applied in a new situation or for
another purpose.
• A new situation can be application in another target population, another language, or another
form of administration. A well-known type of validation is cross-cultural validation, i.e. validation
when an instrument is applied in countries with a different culture and language.
Validation of scores, not measurement instruments
• Validation focuses on the scores produced by a measurement instrument, and not on the
instrument itself. This is a consequence of the previous point, i.e. that a measurement
instrument might function differently in other situations.
1. Validity
• The degree to which an instrument truly measures the construct(s) it purports to measure →
measure what you want to measure
• Three different types:
o Content validity → focuses on whether the content of the instrument corresponds with
the construct that one intends to measure, with regard to relevance and
comprehensiveness
o Criterion validity, applicable in situations in which there is a gold standard for the
construct to be measured → refers to how well the scores of the measurement
instrument agree with the scores on the gold standard
o Construct validity, applicable in situations in which there is no gold standard → refers to
whether the instrument provides the expected scores, based on existing knowledge
about the construct
• Several subtypes of validity exist within these three main types.
1.1. The concept of validity
Validation consists of testing hypotheses, which are based on theories about the construct. The
scores of the instrument should be consistent with a theoretical model of the construct.
Knowledge about the construct to be measured
• We can only assess whether a measurement instrument measures what it purports to measure if
researchers have clearly described the construct they intend to measure.
• Subsequently, we have to formulate hypotheses about what scores we expect to find on the
measurement instruments, based on our knowledge of the construct.
• Therefore, detailed knowledge of the construct and a conceptual model to hypothesize
relationships with other constructs are indispensable for a sound validation process.
Complexity of the construct
• A unidimensional construct is often easier to validate than a multidimensional construct.
• When using a multidimensional instrument, each scale or each part of the instrument that
measures a specific dimension should be validated, by formulating hypotheses for each
dimension separately.
Dependency on the situation
• A measurement instrument should be validated again if it is applied in a new situation or for
another purpose.
• A new situation can be application in another target population, another language, or another
form of administration. A well-known type of validation is cross-cultural validation, i.e. validation
when an instrument is applied in countries with a different culture and language.
Validation of scores, not measurement instruments
• Validation focuses on the scores produced by a measurement instrument, and not on the
instrument itself. This is a consequence of the previous point, i.e. that a measurement
instrument might function differently in other situations.
