Lecture 1: Course Introduction (Dr. Rineke Vasse)
Topics
• Week 1: Non-experimental designs
• Week 2: Experimental designs
• Weeks 3 & 4: Special issues
o Sample size calculation
o Responsive evaluation
CPHI: Complex Public Health Interventions
• Examples of complex public health interventions:
,Why is evaluation of CPHI so difficult?
• CPHI intervene at multiple levels
o The regular ‘gold standard’ of RCTs is not suitable
o Alternative gold standard: Cluster randomised trial (cRCT) (e.g. at a school, or
a community level) but requires more clusters/participants
o More advanced multilevel analysis
o Good alternatives: non-experimental designs
• Program evaluation
o Outcome evaluation (assessing the effects of an intervention)
o Process evaluation (assessing the intervention’s development,
implementation, and evaluation)
o Most important: Can the outcomes be attributed to the intervention?
(internal validity)
o In summary, creativity is required to assess the intervention compared to
conventional RCTs
Learning Goals
• Sensitise students for the advantages and disadvantages of different program
evaluation methods
• Enable students to match program evaluation to particular conditions
• Regarding program evaluation you will know:
o What research methods are gold standard
o What research methods are good alternatives
o How process evaluation complements outcome evaluation
o How you can study interventions’ working mechanisms
o How you can combine quantitative and qualitative methods in program
evaluation (mixed-methods)
o What checklists you can use for critical appraisal
• Regarding the special issues you will know:
o What implications a cluster design has for sample size calculation
o What implications a participatory approach has for evaluation methods
• Regarding non-experimental and experimental designs, you will be able to use
statistical data-analysis methods to test:
o Program outcomes
o How program outcomes depend on process measures
▪ = testing the intervention’s mechanisms of effects
Practical 1: Case: ‘We are the medicine ourselves’
• Program outcome: Awareness of dementia risk being modifiable
• Does the awareness of dementia risk being modifiable at post test depend on the
level of exposure to the campaign (posters, app, etc)?
Practical 2: Case: ‘Vitality@CHEM’
, • Program outcome: Vitality
• Does the improvement in vitality depend on the frequency of personal contact with
the vitality coach?
Practical 3: Several cases
• Regarding experimental designs, you will be able to apply:
o Sample size calculation methods
o RCT & cRCT
Practical 4: ‘Exhale’
• Regarding participatory designs, you will be able to apply:
o Responsive evaluation methods within participatory action research
▪ Transcripts, field notes and thematic analysis (EPH1023)
EPH2242: Year 2 Paper
• Compared to Year 1 Paper, this paper is:
o More substantial: more systematic approach and more empirical studies
o Critical appraisal of research methods employed by the studies reviewed in
the paper
Topics
• Week 1: Non-experimental designs
• Week 2: Experimental designs
• Weeks 3 & 4: Special issues
o Sample size calculation
o Responsive evaluation
CPHI: Complex Public Health Interventions
• Examples of complex public health interventions:
,Why is evaluation of CPHI so difficult?
• CPHI intervene at multiple levels
o The regular ‘gold standard’ of RCTs is not suitable
o Alternative gold standard: Cluster randomised trial (cRCT) (e.g. at a school, or
a community level) but requires more clusters/participants
o More advanced multilevel analysis
o Good alternatives: non-experimental designs
• Program evaluation
o Outcome evaluation (assessing the effects of an intervention)
o Process evaluation (assessing the intervention’s development,
implementation, and evaluation)
o Most important: Can the outcomes be attributed to the intervention?
(internal validity)
o In summary, creativity is required to assess the intervention compared to
conventional RCTs
Learning Goals
• Sensitise students for the advantages and disadvantages of different program
evaluation methods
• Enable students to match program evaluation to particular conditions
• Regarding program evaluation you will know:
o What research methods are gold standard
o What research methods are good alternatives
o How process evaluation complements outcome evaluation
o How you can study interventions’ working mechanisms
o How you can combine quantitative and qualitative methods in program
evaluation (mixed-methods)
o What checklists you can use for critical appraisal
• Regarding the special issues you will know:
o What implications a cluster design has for sample size calculation
o What implications a participatory approach has for evaluation methods
• Regarding non-experimental and experimental designs, you will be able to use
statistical data-analysis methods to test:
o Program outcomes
o How program outcomes depend on process measures
▪ = testing the intervention’s mechanisms of effects
Practical 1: Case: ‘We are the medicine ourselves’
• Program outcome: Awareness of dementia risk being modifiable
• Does the awareness of dementia risk being modifiable at post test depend on the
level of exposure to the campaign (posters, app, etc)?
Practical 2: Case: ‘Vitality@CHEM’
, • Program outcome: Vitality
• Does the improvement in vitality depend on the frequency of personal contact with
the vitality coach?
Practical 3: Several cases
• Regarding experimental designs, you will be able to apply:
o Sample size calculation methods
o RCT & cRCT
Practical 4: ‘Exhale’
• Regarding participatory designs, you will be able to apply:
o Responsive evaluation methods within participatory action research
▪ Transcripts, field notes and thematic analysis (EPH1023)
EPH2242: Year 2 Paper
• Compared to Year 1 Paper, this paper is:
o More substantial: more systematic approach and more empirical studies
o Critical appraisal of research methods employed by the studies reviewed in
the paper
