Research Methods in Health Sciences HC 9
Experimental Research
Learning goals:
You know what aspects may lead to a Risk of Bias (RoB) of a RCT
You know what aspects of the study design are important to
consider when you design or judge the RoB of an experimental study
You are able to give a critical appraisal of a report of a RCT
Context
Researcher wants to no what the causal effect of a new treatment is. The
difference between observational studies and experimental studies is that
you do not manipulate exposure in observational studies.
Observational: non-experimental, there is no individual intervention
treatment and exposures occur in a non-controlled environment.
Individuals can be observed prospectively, retrospectively or currently.
Cross-sectional studies: exposure and outcome are measured at the
same time; includes representative sample of a certain population.
Case-control studies: subjects selected according to disease status
and further classified according to exposure status.
Historical cohort studies: choose cohort, determine disease
incidence determine who was exposed/ non-exposed in the past.
Prospective cohort studies: subjects selected according to exposure
and followed for occurrence of disease
Experimental studies: the investigator can control or manipulate the
exposure. It has a planned research design. RCTs are commonly
considered to be the “golden standard” of research designs the ultimate
step in testing causal hypotheses, less likely to give biased results. In a
RCT participants are randomly assigned to intervention and control groups
to prevent confounding by indication. This provides most convincing
evidence of causal relationship between exposure and effect. However,
even though it is the gold standard that does not mean that you can trust
every result in the report. When a RCT is poorly designed and conducted a
lot of things can go wrong. The study design of an RCT includes: RQ, study
population, intervention, comparator, outcome measures and instruments,
data collection, data analysis, interpretation and report.
Risk of Bias of a RCT
Definition of quality of a RCT:
, The probability that the study design does not lead to biased results
(internal validity). RoB An important step in systematic reviews of
RCTs.
The probability that the results are sufficient precise (precision)
The probability that the results can be applied in daily practice
(external validity)
Selection bias: when there are systematic differences between baseline
characteristics of control and intervention group. Randomisation prevents
selection bias, however an incorrect randomization procedure can
introduce selection bias. This can happen when generation of the
sequence is done incorrect or when the allocation sequence is not
concealed.
Performance bias: when there are systematic differences in care provided
in the groups you compare. Also when exposure to factors other than the
intervention of control is not the same for the groups. Performance bias
can also be caused when care provider and / or patients to not comply to
the protocol.
Attrition bias: when there are systematic differences in loss-at-follow-up
between groups. Reasons can be withdrawals, side effects, worse health,
or when the treatment doesn’t work.
Detection bias: when there are systematic differences in measurement of
outcomes between groups.
Reporting bias: refers to systematic differences between reported and
unreported findings.
, Criteria for internal validity: randomisation, comparable groups at
baseline, blinding of randomisation allocation, compliance to protocol,
loss-at-follow-up, and intention to treat analyses.
Criteria for external validity: relevant RQ, description of in- exclusion
criteria, description of intervention and control intervention, relevant
outcome measures, length of FU.
Criteria for precision: sample size large enough, outcome measures
presented with CI.
Aspects important to avoid bias
RQ: is the RQ relevant? Is the experiment feasible and applicable. PICO.
Applicable: recruitment realistic, payable, can effects occur in time
frame pilot studies
Feasibility: Medical Ethics Committee (MEC)
Study population:
Recruitment: is correct information provided to patients (informed
consent)
Composition: in- exclusion, homogenous, representative
Size: large enough to answers RQ, loss-at-follow-up (3 months
<20%, 12 months <30%). From loss-at-follow-up collect reasons,
numbers per group. To prevent loss-at-follow-up send reminders,
personal contact, gifts.
Allocation: comparable at baseline. Was the allocation to
intervention or control group performed by randomisation?
Important aspects of the randomization procedure is generation of
allocation sequence and concealment of allocation sequence.
o Step 1: sequence generation describes the actual method
used to generate the allocation sequence. Generation is not
adequate I the sequence is predictable. Blinding of
randomisation procedure is not adequate if not done by an
independent person or computer program that track all
procedures.
o Step 2: adequate concealment schemes are: on-site computer
system combined with allocations kept in a locked unreadable
computer file, sealed envelopes with randomization code
generated by the computer, centralized randomization, and
pre-numbered or coded identical containers which are
administered serially to participants.
Interventions: should include a control intervention which is either
placebo, no intervention or usual care, and there should be clear contrast
between interventions. This is done to assure that the effect that authors
Experimental Research
Learning goals:
You know what aspects may lead to a Risk of Bias (RoB) of a RCT
You know what aspects of the study design are important to
consider when you design or judge the RoB of an experimental study
You are able to give a critical appraisal of a report of a RCT
Context
Researcher wants to no what the causal effect of a new treatment is. The
difference between observational studies and experimental studies is that
you do not manipulate exposure in observational studies.
Observational: non-experimental, there is no individual intervention
treatment and exposures occur in a non-controlled environment.
Individuals can be observed prospectively, retrospectively or currently.
Cross-sectional studies: exposure and outcome are measured at the
same time; includes representative sample of a certain population.
Case-control studies: subjects selected according to disease status
and further classified according to exposure status.
Historical cohort studies: choose cohort, determine disease
incidence determine who was exposed/ non-exposed in the past.
Prospective cohort studies: subjects selected according to exposure
and followed for occurrence of disease
Experimental studies: the investigator can control or manipulate the
exposure. It has a planned research design. RCTs are commonly
considered to be the “golden standard” of research designs the ultimate
step in testing causal hypotheses, less likely to give biased results. In a
RCT participants are randomly assigned to intervention and control groups
to prevent confounding by indication. This provides most convincing
evidence of causal relationship between exposure and effect. However,
even though it is the gold standard that does not mean that you can trust
every result in the report. When a RCT is poorly designed and conducted a
lot of things can go wrong. The study design of an RCT includes: RQ, study
population, intervention, comparator, outcome measures and instruments,
data collection, data analysis, interpretation and report.
Risk of Bias of a RCT
Definition of quality of a RCT:
, The probability that the study design does not lead to biased results
(internal validity). RoB An important step in systematic reviews of
RCTs.
The probability that the results are sufficient precise (precision)
The probability that the results can be applied in daily practice
(external validity)
Selection bias: when there are systematic differences between baseline
characteristics of control and intervention group. Randomisation prevents
selection bias, however an incorrect randomization procedure can
introduce selection bias. This can happen when generation of the
sequence is done incorrect or when the allocation sequence is not
concealed.
Performance bias: when there are systematic differences in care provided
in the groups you compare. Also when exposure to factors other than the
intervention of control is not the same for the groups. Performance bias
can also be caused when care provider and / or patients to not comply to
the protocol.
Attrition bias: when there are systematic differences in loss-at-follow-up
between groups. Reasons can be withdrawals, side effects, worse health,
or when the treatment doesn’t work.
Detection bias: when there are systematic differences in measurement of
outcomes between groups.
Reporting bias: refers to systematic differences between reported and
unreported findings.
, Criteria for internal validity: randomisation, comparable groups at
baseline, blinding of randomisation allocation, compliance to protocol,
loss-at-follow-up, and intention to treat analyses.
Criteria for external validity: relevant RQ, description of in- exclusion
criteria, description of intervention and control intervention, relevant
outcome measures, length of FU.
Criteria for precision: sample size large enough, outcome measures
presented with CI.
Aspects important to avoid bias
RQ: is the RQ relevant? Is the experiment feasible and applicable. PICO.
Applicable: recruitment realistic, payable, can effects occur in time
frame pilot studies
Feasibility: Medical Ethics Committee (MEC)
Study population:
Recruitment: is correct information provided to patients (informed
consent)
Composition: in- exclusion, homogenous, representative
Size: large enough to answers RQ, loss-at-follow-up (3 months
<20%, 12 months <30%). From loss-at-follow-up collect reasons,
numbers per group. To prevent loss-at-follow-up send reminders,
personal contact, gifts.
Allocation: comparable at baseline. Was the allocation to
intervention or control group performed by randomisation?
Important aspects of the randomization procedure is generation of
allocation sequence and concealment of allocation sequence.
o Step 1: sequence generation describes the actual method
used to generate the allocation sequence. Generation is not
adequate I the sequence is predictable. Blinding of
randomisation procedure is not adequate if not done by an
independent person or computer program that track all
procedures.
o Step 2: adequate concealment schemes are: on-site computer
system combined with allocations kept in a locked unreadable
computer file, sealed envelopes with randomization code
generated by the computer, centralized randomization, and
pre-numbered or coded identical containers which are
administered serially to participants.
Interventions: should include a control intervention which is either
placebo, no intervention or usual care, and there should be clear contrast
between interventions. This is done to assure that the effect that authors
