Task 1 – Do you know what you measure?
Learning goals:
1. What are the measurement properties? (validity / reliability / responsiveness)
- Validity (content / criterion / construct)
The validity is the degree to which an instrument measures the construct it purports to measure
Types:
o Content validity = the degree to which the content of the instrument is an adequate
reflection of the construct to be measured
Face validity = the degree to which an instrument indeed looks as though they
are an adequate reflection of the construct to be measured first glance
o Construct validity = the degree to which the scores of an instrument are consistent with
hypotheses based on the assumption that the instrument validly measures the construct to
be measured
Structural validity = the degree to which the scores of the instrument are an
adequate reflection of the dimensionality of the construct to be measured
Hypotheses testing convergent / discriminant
Cross-cultural validity = the degree to which the performance of the items on a
translated or culturally adapted instrument are an adequate reflection of the
performance of the items of the original version of the instrument
o Criterion validity = the degree to which the scores of the instrument are an adequate
reflection of a gold standard.
Diagnostic to calculate sensitivity and specificity
, Task 1 – Do you know what you measure?
- Reliability
Reliability is the degree to which the measurement is free from measurement error or in the
extended definition of reliability the extent to which scores for patient who have not changed are
the same for repeated measurement under several conditions:
o Using different sets of items from the same measurement tool = internal consistency
used to assess the consistency of results across items within a test
E.g. asking do you smoke and are you a smoker?
How to calculate = Cronbach’s alpha
o Over time = test-retest the degree of agreement between the results of successive
measurements of the same measure, when carried out under the same conditions of
measurement. The measurements are taken by a single person or instrument on the same
item, under the same conditions, and in a short period of time.
Two identical measurements taken at different times
E.g. blood pressure measurement of the same patient taken at 5PM and at 6 PM
o By different persons on the same occasion = inter-rater the degree of agreement among
independent raters who assess the same phenomenon
E.g. two doctors measuring the same patient’s blood pressure
o By the same persons on different occasions = intra-rater the degree of agreement among
repeated measurements performed by a single rater
E.g. doctor measures a patient’s blood-pressure twice in a row
Measurement properties:
o Internal consistency = the degree of the interrelatedness among the items
o Reliability = the proportion of the total variance in the measurements which is because of
true differences among patients
o Measurement error = the systematic and random error of a patient’s score that is not
attributed to the true changes in the construct to be measured
True score – observed score = measurement error
- Responsiveness
Responsiveness is the ability of an instrument to detect change over time in the construct to be
measured.
Approaches:
- Criterion approach
- Construct approach
- Interpretability
Interpretability is the degree to which one can assign qualitative meaning to an instrument’s
quantitative scores or change in scores.
2. What are the types (characteristics) of measures?
- Diagnosis vs outcome measurements
Learning goals:
1. What are the measurement properties? (validity / reliability / responsiveness)
- Validity (content / criterion / construct)
The validity is the degree to which an instrument measures the construct it purports to measure
Types:
o Content validity = the degree to which the content of the instrument is an adequate
reflection of the construct to be measured
Face validity = the degree to which an instrument indeed looks as though they
are an adequate reflection of the construct to be measured first glance
o Construct validity = the degree to which the scores of an instrument are consistent with
hypotheses based on the assumption that the instrument validly measures the construct to
be measured
Structural validity = the degree to which the scores of the instrument are an
adequate reflection of the dimensionality of the construct to be measured
Hypotheses testing convergent / discriminant
Cross-cultural validity = the degree to which the performance of the items on a
translated or culturally adapted instrument are an adequate reflection of the
performance of the items of the original version of the instrument
o Criterion validity = the degree to which the scores of the instrument are an adequate
reflection of a gold standard.
Diagnostic to calculate sensitivity and specificity
, Task 1 – Do you know what you measure?
- Reliability
Reliability is the degree to which the measurement is free from measurement error or in the
extended definition of reliability the extent to which scores for patient who have not changed are
the same for repeated measurement under several conditions:
o Using different sets of items from the same measurement tool = internal consistency
used to assess the consistency of results across items within a test
E.g. asking do you smoke and are you a smoker?
How to calculate = Cronbach’s alpha
o Over time = test-retest the degree of agreement between the results of successive
measurements of the same measure, when carried out under the same conditions of
measurement. The measurements are taken by a single person or instrument on the same
item, under the same conditions, and in a short period of time.
Two identical measurements taken at different times
E.g. blood pressure measurement of the same patient taken at 5PM and at 6 PM
o By different persons on the same occasion = inter-rater the degree of agreement among
independent raters who assess the same phenomenon
E.g. two doctors measuring the same patient’s blood pressure
o By the same persons on different occasions = intra-rater the degree of agreement among
repeated measurements performed by a single rater
E.g. doctor measures a patient’s blood-pressure twice in a row
Measurement properties:
o Internal consistency = the degree of the interrelatedness among the items
o Reliability = the proportion of the total variance in the measurements which is because of
true differences among patients
o Measurement error = the systematic and random error of a patient’s score that is not
attributed to the true changes in the construct to be measured
True score – observed score = measurement error
- Responsiveness
Responsiveness is the ability of an instrument to detect change over time in the construct to be
measured.
Approaches:
- Criterion approach
- Construct approach
- Interpretability
Interpretability is the degree to which one can assign qualitative meaning to an instrument’s
quantitative scores or change in scores.
2. What are the types (characteristics) of measures?
- Diagnosis vs outcome measurements
