Task 3 – Measuring what you want to measure
Learning goals:
1) What are the types of validity?
Validity focuses on the scores produced by a measurement instrument, and not on the instrument
itself. Therefore, a measurement instrument should be validates again if it is applied in a new
situation or for another purpose.
Types of validity:
- Content validity
Content validity is the degree to which the content of a measurement instrument is an adequate
reflection of the construct to be measures.
A first step of content validity is face validity, which is the degree to which a measurement
instrument indeed looks as though it is an adequate reflection of the construct to be measured.
Overall view, which is often a first impression, without going into too much detail
Subjective assessment, which cannot be quantified
Lack of face validity is a very strong argument for not using an instrument or to end further
validation.
When an instrument has passed the test of face validity, we have to consider its content in more
detail. The purpose of a content validation study is to assess whether the instrument adequately
represents the construct under study.
The process of content validation consists of the following steps:
1. Consider information about construct and situation
2. Consider information about content of the measurement instrument
3. Select an expert panel
4. Assess whether content of the measurement instrument corresponds with the construct
(relevant and comprehensive)
5. Use a strategy or framework to assess the correspondence between the instrument and
construct
- Criterion validity
Criterion validity is defined as the degree to which the scores of a measurement instrument are an
adequate reflection of a gold standard. This implies, that criterion validity can only be assessed when
a gold standard is available.
Criterion validity can be subdivided into:
o Concurrent validity = we consider both the score of the measurement and the score of the
gold standard at the same time
Evaluative and diagnostic purposes
o Predictive validity = we consider whether the measurement instrument predicts the gold
standard in the future.
Predictive purpose
, Task 3 – Measuring what you want to measure
The hypothesis is that the measurement instrument under study is as good as the gold standard. The
general design of criterion-related validation consists of the following steps:
1. Identify a suitable criterion and method of measurement
The gold standard is considered to represent the true state of the construct of interest. In theory, the
gold standard is a perfectly valid assessment. However, a perfect gold standard seldom exists in
practice. It is usually a measurement that has been accepted as a gold standard by experts.
Short version of an already existing questionnaire; the longer version is the gold standard
2. Identify an appropriate sample of the target population in which the measurement
instrument will ultimately be used
The validation study should be performed in the same way is in the practice it will be used.
3. Define a priori the required level of agreement between the measurement instrument and
criterion
Hypotheses = measurement instrument should be as good as the gold standard (the extent of
agreement is often not mentioned)
It is better to decide a priori which level of agreement one considers acceptable.
4. Obtain the scores or the measurement instrument and the gold standard, independently
from each other
Independent application of the measurement instrument and the gold standard is necessary. The
measurement instrument should not be part of the gold standard, or influence it in any way.
5. Determine the strength of the relationship between the instrument scores and criterion
scores
- Construct validity
In situations in which a gold standard is lacking, construct validation should be used to provide
evidence of validity. Construct validity is defined as the degree to which the scores of a
measurement instrument are consistent with hypotheses (with regard to internal relationships,
relationships with scores of other instruments or differences between relevant groups). It is based on
the assumptions that the measurement instrument validly measures the construct to be measures.
Aspects of construct validity (types of test the construct validity)
o Structural validity
The degree to which the scores of a measurement instrument are an adequate reflection of the
dimensionality of the construct to be measured. This can be assessed by factor analysis. For
validation purposes confirmatory factor analysis is more appropriate. See learning goal 7
o Hypotheses testing
The basic principle of construct validation is that hypotheses are formulated about the relationship of
scores on the instrument under study with scores on other instruments measuring similar or
dissimilar construct, or differences in the instrument scores between subgroups of patients.
Types of hypothesis testing:
Learning goals:
1) What are the types of validity?
Validity focuses on the scores produced by a measurement instrument, and not on the instrument
itself. Therefore, a measurement instrument should be validates again if it is applied in a new
situation or for another purpose.
Types of validity:
- Content validity
Content validity is the degree to which the content of a measurement instrument is an adequate
reflection of the construct to be measures.
A first step of content validity is face validity, which is the degree to which a measurement
instrument indeed looks as though it is an adequate reflection of the construct to be measured.
Overall view, which is often a first impression, without going into too much detail
Subjective assessment, which cannot be quantified
Lack of face validity is a very strong argument for not using an instrument or to end further
validation.
When an instrument has passed the test of face validity, we have to consider its content in more
detail. The purpose of a content validation study is to assess whether the instrument adequately
represents the construct under study.
The process of content validation consists of the following steps:
1. Consider information about construct and situation
2. Consider information about content of the measurement instrument
3. Select an expert panel
4. Assess whether content of the measurement instrument corresponds with the construct
(relevant and comprehensive)
5. Use a strategy or framework to assess the correspondence between the instrument and
construct
- Criterion validity
Criterion validity is defined as the degree to which the scores of a measurement instrument are an
adequate reflection of a gold standard. This implies, that criterion validity can only be assessed when
a gold standard is available.
Criterion validity can be subdivided into:
o Concurrent validity = we consider both the score of the measurement and the score of the
gold standard at the same time
Evaluative and diagnostic purposes
o Predictive validity = we consider whether the measurement instrument predicts the gold
standard in the future.
Predictive purpose
, Task 3 – Measuring what you want to measure
The hypothesis is that the measurement instrument under study is as good as the gold standard. The
general design of criterion-related validation consists of the following steps:
1. Identify a suitable criterion and method of measurement
The gold standard is considered to represent the true state of the construct of interest. In theory, the
gold standard is a perfectly valid assessment. However, a perfect gold standard seldom exists in
practice. It is usually a measurement that has been accepted as a gold standard by experts.
Short version of an already existing questionnaire; the longer version is the gold standard
2. Identify an appropriate sample of the target population in which the measurement
instrument will ultimately be used
The validation study should be performed in the same way is in the practice it will be used.
3. Define a priori the required level of agreement between the measurement instrument and
criterion
Hypotheses = measurement instrument should be as good as the gold standard (the extent of
agreement is often not mentioned)
It is better to decide a priori which level of agreement one considers acceptable.
4. Obtain the scores or the measurement instrument and the gold standard, independently
from each other
Independent application of the measurement instrument and the gold standard is necessary. The
measurement instrument should not be part of the gold standard, or influence it in any way.
5. Determine the strength of the relationship between the instrument scores and criterion
scores
- Construct validity
In situations in which a gold standard is lacking, construct validation should be used to provide
evidence of validity. Construct validity is defined as the degree to which the scores of a
measurement instrument are consistent with hypotheses (with regard to internal relationships,
relationships with scores of other instruments or differences between relevant groups). It is based on
the assumptions that the measurement instrument validly measures the construct to be measures.
Aspects of construct validity (types of test the construct validity)
o Structural validity
The degree to which the scores of a measurement instrument are an adequate reflection of the
dimensionality of the construct to be measured. This can be assessed by factor analysis. For
validation purposes confirmatory factor analysis is more appropriate. See learning goal 7
o Hypotheses testing
The basic principle of construct validation is that hypotheses are formulated about the relationship of
scores on the instrument under study with scores on other instruments measuring similar or
dissimilar construct, or differences in the instrument scores between subgroups of patients.
Types of hypothesis testing:
