Task 2 – development of a measurement
Learning goals:
1. What should you do before developing a new measurement? Why should you develop a new
measurement scale?
Before deciding to develop a new measurement instrument, a systematic literature review of the
properties of all existing instruments intended to measure the specific characteristic or concept is
indispensable.
Such a literature review is important for three reasons:
- Searching for existing instruments prevents the development of new ones in fields where
many already exist.
- To get ideas about what a new instrument should or should not look like.
- It saves a lot of time and effort if you find a measurement instrument that can be translated
or adapted to your own specific needs
Researchers should justify their reasons for developing any new instrument.
Only if NO instrument is available, a new measurement instrument should be developed.
2. What are the six steps of developing a new measurement?
The development of a new measurement takes time because the process it iterative. The
development of a measurement instrument can be divided into six steps:
These steps are intertwined and goes back and forth between the steps in a continuous process of
evaluation and adaptation. The last steps in the development process consist of pilot-testing and
field-testing.
STEPS 1: DEFINITION AND ELABORATION OF THE CONSTRUCT INTENTED TO BE MEASURED
The most essential questions that must be answered are ‘what do we want to measure?’ in ‘which
target population?’ and for ‘which purpose?’.
- Construct
Definition of the construct starts with a decision concerning its level in the conceptual model and
considerations about potential aspects of the construct. So, in which level in the conceptual model
are we interested in?
, Task 2 – development of a measurement
If a construct has different aspects, and we want to measure all these aspects, the measurement
instrument should anticipate this multidimensionality. It is important that before actually
constructing a measurement instrument, we decide which aspects we want to include. This has to be
done in the conceptual phase, preferably based on a conceptual framework.
- Target population
The measurement instrument should be tailored to the target population, which bust me defined.
Age, gender and severity of disease determine to a large extent the content and type of instrument
that can be used.
- Purpose of measurement
Three important objectives of measurement in medicine are:
o Diagnosis = discriminative instruments that are able to discriminate between persons at a
single point in time.
o Evaluation = to evaluate the effects of treatment or other longitudinal changes we need
evaluative instruments able to measure change over time.
o Prediction = predictive measurements aim to classify individuals according to their prognosis
(future course of their disease).
Prediction models instead of measurement instruments, because they usually contain a
number of different construct and variables.
STEP 2: CHOICE OF MEASUREMENT METHOD
The type of measurement instrument should correspond closely to the construct to be measured.
When designing a PRO instrument, we next must decide whether a multi-item measurement
instrument is needed, or whether a single-item instrument will suffice. A multi-item instrument will
be more reliable than a single-item instrument.
In a multi-item measurement instrument the content of the items is often more specific, and
multidimensional instruments include all the dimensions considered to be relevant for the construct.
This makes it easier for patients to understand the items and we know that the same construct is
being measured for all patients.
In a single item we leave it to the patient it define the meaning of the construct (e.g. fatigue).
o Hampers the interpretation of the score
If more aspects are involved, multi-item instrument, in which multiple dimensions can be
distinguished are more informative, because they provide subscores for each domain.
But what do we choice?
- Complex construct are best measured with multi-item instruments
- But there might be situation in which a single-item is more preferable
STEP 3: SELECTING ITEMS
- Getting input for the items of questionnaire (literature and experts)
o Literature
Learning goals:
1. What should you do before developing a new measurement? Why should you develop a new
measurement scale?
Before deciding to develop a new measurement instrument, a systematic literature review of the
properties of all existing instruments intended to measure the specific characteristic or concept is
indispensable.
Such a literature review is important for three reasons:
- Searching for existing instruments prevents the development of new ones in fields where
many already exist.
- To get ideas about what a new instrument should or should not look like.
- It saves a lot of time and effort if you find a measurement instrument that can be translated
or adapted to your own specific needs
Researchers should justify their reasons for developing any new instrument.
Only if NO instrument is available, a new measurement instrument should be developed.
2. What are the six steps of developing a new measurement?
The development of a new measurement takes time because the process it iterative. The
development of a measurement instrument can be divided into six steps:
These steps are intertwined and goes back and forth between the steps in a continuous process of
evaluation and adaptation. The last steps in the development process consist of pilot-testing and
field-testing.
STEPS 1: DEFINITION AND ELABORATION OF THE CONSTRUCT INTENTED TO BE MEASURED
The most essential questions that must be answered are ‘what do we want to measure?’ in ‘which
target population?’ and for ‘which purpose?’.
- Construct
Definition of the construct starts with a decision concerning its level in the conceptual model and
considerations about potential aspects of the construct. So, in which level in the conceptual model
are we interested in?
, Task 2 – development of a measurement
If a construct has different aspects, and we want to measure all these aspects, the measurement
instrument should anticipate this multidimensionality. It is important that before actually
constructing a measurement instrument, we decide which aspects we want to include. This has to be
done in the conceptual phase, preferably based on a conceptual framework.
- Target population
The measurement instrument should be tailored to the target population, which bust me defined.
Age, gender and severity of disease determine to a large extent the content and type of instrument
that can be used.
- Purpose of measurement
Three important objectives of measurement in medicine are:
o Diagnosis = discriminative instruments that are able to discriminate between persons at a
single point in time.
o Evaluation = to evaluate the effects of treatment or other longitudinal changes we need
evaluative instruments able to measure change over time.
o Prediction = predictive measurements aim to classify individuals according to their prognosis
(future course of their disease).
Prediction models instead of measurement instruments, because they usually contain a
number of different construct and variables.
STEP 2: CHOICE OF MEASUREMENT METHOD
The type of measurement instrument should correspond closely to the construct to be measured.
When designing a PRO instrument, we next must decide whether a multi-item measurement
instrument is needed, or whether a single-item instrument will suffice. A multi-item instrument will
be more reliable than a single-item instrument.
In a multi-item measurement instrument the content of the items is often more specific, and
multidimensional instruments include all the dimensions considered to be relevant for the construct.
This makes it easier for patients to understand the items and we know that the same construct is
being measured for all patients.
In a single item we leave it to the patient it define the meaning of the construct (e.g. fatigue).
o Hampers the interpretation of the score
If more aspects are involved, multi-item instrument, in which multiple dimensions can be
distinguished are more informative, because they provide subscores for each domain.
But what do we choice?
- Complex construct are best measured with multi-item instruments
- But there might be situation in which a single-item is more preferable
STEP 3: SELECTING ITEMS
- Getting input for the items of questionnaire (literature and experts)
o Literature
