Research methodology
The purpose of health science/research is finding an objective, generalizable, ‘truth’.
Two pespectives:
Positivism (objectivism)
- Reality can be observed
- Observes facts of one single reality Deductive
- Value-free - Begins with hypothesis and theories
- Measures and predict - Manipulation and control
- Generalizable - Uses formal instruments
- Bijv. Noteren hoe veel dagen iemand heeft gewacht - Experimentation/survey/structured interviews
op een afspraak - Seeks for confirmation/rejection
- Deductive - Ex. Theory of planned behavior 🡪 hypothesis:
exposure to fast-food> consumption 🡪 Observation:
Interpretavism (constructivism) do the test 🡪 confirmation/rejection
- Truth and meaning are constructed by the
person/researcher (subjects) Inductive
- Researchers are inherently view through their - Thick description
frame of reference - End with hypotheses and grounded theory
- Multiple realities (are experienced), and meaning is - Emergence and portrayal
not stable - Researchers as instrument
- Constructs reality - Seeks for (contextual) theory
- Observations are value-bound - Ex. Observation: views of fastfood junkies 🡪
- To deeply understand pattern> absence of lifegoals 🡪 hypothesis> no
- Time and context bound hypothesis/generalization lifegoals > fastfood 🡪 theory >lifegoal
- Bijv. Aan de persoon vragen hoe diegene de
wachtlijst heeft ervaren
Deductive: confirm/reject theory. Look at your data with clear units of analyses. Is structured.
Inductive: open and develop theory. Analyze data for emerging patterns. Theory unstructured.
(Quasi)Experimental
- Determine causality
- Aim to generalize from experiment
- Associated with positivism/deductive approach
- Not randomly, use existing groups 🡪 Quasi
Analytical survey
- To explore and test proportions/associations/predictors between variables
- Highly deductive 🡪 je weet waar je naar op zoek bent!
Phenomological studies
- Aims for contextual description and analyses of ‘phenomena’
- Emphasize inductive logic
- Seeks the opinions and subjective accounts and interpretations of participants
- Relies on qualitative analyses of data
- Inductive
(Participatory) action research
- Research that aims to change practice in real life
- Collaborations between researches and practitioners and users (e.g. patients, community members)
- Iterative designs (herhaling)
- Mixed methods
- Understanding of perspectives in order to determine change and (often) measuring the change.
- Deductive and inductive reasoning
Iterative🡪 telkens opnieuw dingen doen tijdens onderzoek doen
1
,Research objective/question
Needs to be useful, realistic, feasible, clear and informative
- What is the big problem/issue
- What is (not) know/done about it?
- What am I going to make known about
it?
Reliability: the same result if the situation is
stable
- variability because of random
fluctuations
- no specific direction
The domain reliability refers to the degree to
which the measurement is free from
measurement error, and it contains
- internal consistency
- reliability
- measurement error
Internal consistency:
- The interrelatedness among the items
- Multi item (or multi task) instrument/
PROM
- Latent (non observable characteristics)
Reliability: test retest design
Inter rater
- different raters on the same occasion
Or better: different raters on different occasions
This reflects daily practice more adequately.
The construct should not have changed between the 2 measurements: stability!!
- Short interval between the 2 measurements
- And/or: external criteria for stability
Stability of patients by external criteria
- 7 point scale: 1 = ’completely recovered’ to 7 ‘worse than ever’; Stability defined as 3 = not changed
🡪 Statistics: ANOVA
ICC= between subject variance / (between + within subject variance)
Kappa: (p0-pe/(1-pe)
Measurement error
the systematic and random error of a patient’s score that is not attributed to true changes in the construct to be measured.
Measurement error is expressed as SEM
SEM = standard error of measurement
= standard deviation of repeated measurements in one person
SEM = var (error)
Method of bland and altman
Interval and ratio scale; measurement error is expressed
in the same units as measurement
D +/- 1,96 * SDd
D= systematic error
d= random error
2
, Validity: the result that one really aims for
- systematic deviation from the true value
- magnitude and direction
The domain validity refers to the degree to which an outcome measure measures the construct it purports to measure, and
contains the measurement properties:
● content validity, including face validity
● criterion validity
● construct validity,
Content validity
The degree to which the content of an HR-PRO instrument is an adequate reflection of the construct to be measured
Comprehensibility of items of response options; Relevance of items of response options; Comprehensiveness
Content validity is the most important measurement property 🡪 Often overlooked/neglected…
- Face validity
The degree to which (the items of) an HR-PRO instrument indeed looks as though they are an adequate reflection of
the construct to be measured. Makes it sense? - vague concept. The first view.
Criterion validity: Compare change scores to Golden Standard
Relationship between test and criterion (gold standard). Often used in diagnostic research
- Concurrent validity: test & criterion (more or less) simultaneously. E.g. range of motion and X-ray
Construct validity: Hypothesis testing regarding change scores
- Structural validity: The degree to which the scores of an HR-PRO instrument are an adequate reflection of the
dimensionality of the construct to be measured
e.g. structural validity:
Factor structure of Guy's Neurological Disability Scale
Longitudinal validity = responsiveness
Content validity: most important measurement property according to COSMIN
Criterion validity: relationship with gold standard
Construct validity: Structural validity (one dimension/factor or more?); hypotheses testing (needed in absence in golden
standard)
Responsiveness is longitudinal validity: ‘’am i measuring what i aim to measure’’
Internal validity: the extent to which its design & conduct are likely to prevent systematic errors, or bias
Critria for internal validity RCT
- Randomisation procedure
- Comparable groups at baseline
- Blinding of randomisation allocation to: patients; outcome assessor; care provider; statistician or researcher
- Compliance to the protocol by patients ‘therapie trouw’ and care providers
- Loss-to-follow-up
- Intention-to-treat analyses
3
The purpose of health science/research is finding an objective, generalizable, ‘truth’.
Two pespectives:
Positivism (objectivism)
- Reality can be observed
- Observes facts of one single reality Deductive
- Value-free - Begins with hypothesis and theories
- Measures and predict - Manipulation and control
- Generalizable - Uses formal instruments
- Bijv. Noteren hoe veel dagen iemand heeft gewacht - Experimentation/survey/structured interviews
op een afspraak - Seeks for confirmation/rejection
- Deductive - Ex. Theory of planned behavior 🡪 hypothesis:
exposure to fast-food> consumption 🡪 Observation:
Interpretavism (constructivism) do the test 🡪 confirmation/rejection
- Truth and meaning are constructed by the
person/researcher (subjects) Inductive
- Researchers are inherently view through their - Thick description
frame of reference - End with hypotheses and grounded theory
- Multiple realities (are experienced), and meaning is - Emergence and portrayal
not stable - Researchers as instrument
- Constructs reality - Seeks for (contextual) theory
- Observations are value-bound - Ex. Observation: views of fastfood junkies 🡪
- To deeply understand pattern> absence of lifegoals 🡪 hypothesis> no
- Time and context bound hypothesis/generalization lifegoals > fastfood 🡪 theory >lifegoal
- Bijv. Aan de persoon vragen hoe diegene de
wachtlijst heeft ervaren
Deductive: confirm/reject theory. Look at your data with clear units of analyses. Is structured.
Inductive: open and develop theory. Analyze data for emerging patterns. Theory unstructured.
(Quasi)Experimental
- Determine causality
- Aim to generalize from experiment
- Associated with positivism/deductive approach
- Not randomly, use existing groups 🡪 Quasi
Analytical survey
- To explore and test proportions/associations/predictors between variables
- Highly deductive 🡪 je weet waar je naar op zoek bent!
Phenomological studies
- Aims for contextual description and analyses of ‘phenomena’
- Emphasize inductive logic
- Seeks the opinions and subjective accounts and interpretations of participants
- Relies on qualitative analyses of data
- Inductive
(Participatory) action research
- Research that aims to change practice in real life
- Collaborations between researches and practitioners and users (e.g. patients, community members)
- Iterative designs (herhaling)
- Mixed methods
- Understanding of perspectives in order to determine change and (often) measuring the change.
- Deductive and inductive reasoning
Iterative🡪 telkens opnieuw dingen doen tijdens onderzoek doen
1
,Research objective/question
Needs to be useful, realistic, feasible, clear and informative
- What is the big problem/issue
- What is (not) know/done about it?
- What am I going to make known about
it?
Reliability: the same result if the situation is
stable
- variability because of random
fluctuations
- no specific direction
The domain reliability refers to the degree to
which the measurement is free from
measurement error, and it contains
- internal consistency
- reliability
- measurement error
Internal consistency:
- The interrelatedness among the items
- Multi item (or multi task) instrument/
PROM
- Latent (non observable characteristics)
Reliability: test retest design
Inter rater
- different raters on the same occasion
Or better: different raters on different occasions
This reflects daily practice more adequately.
The construct should not have changed between the 2 measurements: stability!!
- Short interval between the 2 measurements
- And/or: external criteria for stability
Stability of patients by external criteria
- 7 point scale: 1 = ’completely recovered’ to 7 ‘worse than ever’; Stability defined as 3 = not changed
🡪 Statistics: ANOVA
ICC= between subject variance / (between + within subject variance)
Kappa: (p0-pe/(1-pe)
Measurement error
the systematic and random error of a patient’s score that is not attributed to true changes in the construct to be measured.
Measurement error is expressed as SEM
SEM = standard error of measurement
= standard deviation of repeated measurements in one person
SEM = var (error)
Method of bland and altman
Interval and ratio scale; measurement error is expressed
in the same units as measurement
D +/- 1,96 * SDd
D= systematic error
d= random error
2
, Validity: the result that one really aims for
- systematic deviation from the true value
- magnitude and direction
The domain validity refers to the degree to which an outcome measure measures the construct it purports to measure, and
contains the measurement properties:
● content validity, including face validity
● criterion validity
● construct validity,
Content validity
The degree to which the content of an HR-PRO instrument is an adequate reflection of the construct to be measured
Comprehensibility of items of response options; Relevance of items of response options; Comprehensiveness
Content validity is the most important measurement property 🡪 Often overlooked/neglected…
- Face validity
The degree to which (the items of) an HR-PRO instrument indeed looks as though they are an adequate reflection of
the construct to be measured. Makes it sense? - vague concept. The first view.
Criterion validity: Compare change scores to Golden Standard
Relationship between test and criterion (gold standard). Often used in diagnostic research
- Concurrent validity: test & criterion (more or less) simultaneously. E.g. range of motion and X-ray
Construct validity: Hypothesis testing regarding change scores
- Structural validity: The degree to which the scores of an HR-PRO instrument are an adequate reflection of the
dimensionality of the construct to be measured
e.g. structural validity:
Factor structure of Guy's Neurological Disability Scale
Longitudinal validity = responsiveness
Content validity: most important measurement property according to COSMIN
Criterion validity: relationship with gold standard
Construct validity: Structural validity (one dimension/factor or more?); hypotheses testing (needed in absence in golden
standard)
Responsiveness is longitudinal validity: ‘’am i measuring what i aim to measure’’
Internal validity: the extent to which its design & conduct are likely to prevent systematic errors, or bias
Critria for internal validity RCT
- Randomisation procedure
- Comparable groups at baseline
- Blinding of randomisation allocation to: patients; outcome assessor; care provider; statistician or researcher
- Compliance to the protocol by patients ‘therapie trouw’ and care providers
- Loss-to-follow-up
- Intention-to-treat analyses
3
