GW4003MV – Advanced Research Methods
Teacher
Sander Boxebeld ()
Vivian Reckers-Droog
Lecture: Fridays 14.00 – 15.45
Workgroup: 7
Teacher Quantitative: dr. Lucas Goossens ()
Teacher Qualitative: Jonathan Berg, MSc ()
Important!
Work group sessions:
- Application of knowledge from lectures and literature.
- Assessment of research cases.
- Practise and prepare for exam.
PC labs:
- Interpretation and assessment of results/research case using real trial data.
Knowledge videos and lectures:
- Discussion of theory and concepts from literature.
Written exam (100%):
- Assessment of research cases.
Software skills not at the core of the PC lab, but
- Focus on the interpretation of results, also in relation to Stata output (may be asked during
the exam).
- Practise with Stata (relevant for follow-up courses and thesis).
Written exam.
- 100% of your grade for this course.
- "Closed book" (no use of any course materials allowed during the exam).
- To be completed on a computer in an exam room on campus (no internet).
- Two parts: quantitative and qualitative research cases.
- In both parts, you will receive cases of existing studies and are asked to interpret their
results and evaluate their quality.
Workgroup sessions > knowledge videos > lectures > literature.
, / 2025
Lecture 1
1.2 Introduction to Causal Inference
Learning goals / after lecture 1.2 you will be able to:
1. Explain the three different reasons for examining (statistical) associations.
2. Explain the potential outcomes approach in causal inference.
3. Define ‘causal effect’.
4. Apply the concepts of consistency, positivity, and exchangeability to make a causal claim.
Why examine (statistical) associations?
1. Description: patterns X and Y.
2. Prediction: Y given X.
3. Causal inference: effect X on Y.
a. Emphasis is put on the effect of something on something else (variable that is
influencing the outcome).
b. Effect of independent variable on dependent variable.
This lecture: potential outcome approach & identifiability conditions.
Magazine advertisement:
- “Improves the quality of your skin” implies a causal effect:
o X leads to Y.
o Use of True Match Minerals powder (X) leads to a better skin (Y).
- All under dermatological control (?), but no control group!
- What would have happened had the women not used the powder?
- No information on other factors that may influence the result.
- Question we want to answer: what would have happened with Y if X was not there?
- What is the exact influence of X on Y?
In causal inference:
- We are not primarily interested in the outcome (Y) (i.e., 70% less imperfections), BUT
- We are interested in the role of the treatment (X) in achieving this outcome (i.e., without
True Match Minerals powder, would there be a difference in skin imperfections?)
Causal effect.
- “In an individual, a treatment has a causal effect if the outcome under treatment 1 would be
different from the outcome under treatment 2” (Hernàn & Robins, 2020).
, - To assess this, we need information on what would have
happened … (to Y) had … (X) not occurred.
- Difference in outcome between person being confronted
with treatment 1 and treatment 2.
- So, we need information about outcome after
treatment and outcome in absence of treatment.
To make a causal claim you need information on:
1. The outcome under treatment and
2. The outcome under absence of treatment.
In other words, you need information on all potential
outcomes to make a causal claim on the effect of X on Y. We
need information on outcome under all treatment options!
Fundamental problem in causal inference.
- Individual causal effect cannot be directly observed.
o Because we do not have information on counterfactual outcomes.
o Except under extremely strong (often unrealistic) assumptions.
- Average causal effects (i.e. in a population) cannot be determined based on individual
estimates:
o Causal inference is a problem of missing data!
- Solution: estimate an average causal effect by meeting three identifiability conditions.
Identifiability conditions.
- Average causal effect can be estimated if, and only if, all three identifiability conditions are
met:
o Positivity.
o Consistency.
o Exchangeability.
- If all three conditions are met (and an association is found in the data):
o The association between an exposure (X) and outcome (Y) is an unbiased estimate of
a causal effect.
o You can validly make a causal claim.
Positivity.
- The positivity condition requires that each individual has a positive probability of being
assigned to each treatment arm (i.e., Pr(A=a)>0 for all treatment arms, of levels of X) – so no
chance of not having any individual in a particular treatment group.
- L’Oréal example: there should be women who use the powder and women who do not use
the powder (otherwise the counterfactual is missing). Positivity condition was not met,
because all 41 women used the powder.
Consistency.
- The consistency condition requires that the treatment (or exposure X) has to be well-
defined.
- L’Oréal example: what kind of powder? How often applied? Used under what conditions
(with or without make-up?) Used for how long? Used at what time of day? How much
powder? Consistency condition was not met, because ‘powder’ is not well-defined.
Exchangeability (main part of course, using DAGs).
, - The exchangeability condition requires that the individuals assigned to the different
treatment arms are comparable.
- It does not matter who gets treatment A and who gets treatment B, the groups can be
thought of as “interchangeable”.
- This means that any difference in outcome between the groups can be attributed to the
treatment, rather than other differences between the individuals.
- L’Oréal example: women who used powder could have just as well not used it and vice
versa. Exchangeability condition was not met, because there were no two groups of women
that were comparable.
- Exchangeability condition is perfectly met if, and only if, the only difference between the
treatment groups (i.e., the case and control groups) is that one group has received the
treatment and the other has not.
Four ways to achieve exchangeability.
1. Randomized controlled trial (RCT).
a. Individuals are randomly assigned to one of the treatment arms.
b. Differences between individuals in the treatment arms are balanced out at the
group level.
c. These differences are independent of both treatment assignment and outcomes.
d. Differences are therefore random, not systematic.
e. RCTs are often considered the “gold standard” because, in principle, all identifiability
conditions are satisfied.
2. Matching.
a. For each individual with characteristics x, y, z who receives treatment A, there is an
individual with the same characteristics x, y, z who receives treatment B.
b. When perfect matching (e.g., using identical twins, triplets, …) is not possible,
statistical methods such as propensity score matching can be used to approximate
comparability between groups.
3. Stratification.
a. Randomly select individuals from different subsets (strata) of the larger population.
b. Ensures representation across key groups (e.g., age, gender, region).
c. Can be difficult to meet the positivity condition (i.e., having individuals available in
all strata for meaningful comparisons).
4. Adjustment.
a. Control for confounding factors that may bias the association between treatment
and outcome (commonly through regression analysis).
b. Assumes that individuals can, in principle, be assigned to all treatment arms across
all levels of the adjustment factors (positivity).
c. Can be combined with other designs/techniques: RCTs, stratification, matching.
d. Directed acyclic graphs (DAGs) are a useful tool to identify which factors to adjust for
(see Lectures 1.3 and 1.4).
1.3 Directed Acyclic Graphs I
Teacher
Sander Boxebeld ()
Vivian Reckers-Droog
Lecture: Fridays 14.00 – 15.45
Workgroup: 7
Teacher Quantitative: dr. Lucas Goossens ()
Teacher Qualitative: Jonathan Berg, MSc ()
Important!
Work group sessions:
- Application of knowledge from lectures and literature.
- Assessment of research cases.
- Practise and prepare for exam.
PC labs:
- Interpretation and assessment of results/research case using real trial data.
Knowledge videos and lectures:
- Discussion of theory and concepts from literature.
Written exam (100%):
- Assessment of research cases.
Software skills not at the core of the PC lab, but
- Focus on the interpretation of results, also in relation to Stata output (may be asked during
the exam).
- Practise with Stata (relevant for follow-up courses and thesis).
Written exam.
- 100% of your grade for this course.
- "Closed book" (no use of any course materials allowed during the exam).
- To be completed on a computer in an exam room on campus (no internet).
- Two parts: quantitative and qualitative research cases.
- In both parts, you will receive cases of existing studies and are asked to interpret their
results and evaluate their quality.
Workgroup sessions > knowledge videos > lectures > literature.
, / 2025
Lecture 1
1.2 Introduction to Causal Inference
Learning goals / after lecture 1.2 you will be able to:
1. Explain the three different reasons for examining (statistical) associations.
2. Explain the potential outcomes approach in causal inference.
3. Define ‘causal effect’.
4. Apply the concepts of consistency, positivity, and exchangeability to make a causal claim.
Why examine (statistical) associations?
1. Description: patterns X and Y.
2. Prediction: Y given X.
3. Causal inference: effect X on Y.
a. Emphasis is put on the effect of something on something else (variable that is
influencing the outcome).
b. Effect of independent variable on dependent variable.
This lecture: potential outcome approach & identifiability conditions.
Magazine advertisement:
- “Improves the quality of your skin” implies a causal effect:
o X leads to Y.
o Use of True Match Minerals powder (X) leads to a better skin (Y).
- All under dermatological control (?), but no control group!
- What would have happened had the women not used the powder?
- No information on other factors that may influence the result.
- Question we want to answer: what would have happened with Y if X was not there?
- What is the exact influence of X on Y?
In causal inference:
- We are not primarily interested in the outcome (Y) (i.e., 70% less imperfections), BUT
- We are interested in the role of the treatment (X) in achieving this outcome (i.e., without
True Match Minerals powder, would there be a difference in skin imperfections?)
Causal effect.
- “In an individual, a treatment has a causal effect if the outcome under treatment 1 would be
different from the outcome under treatment 2” (Hernàn & Robins, 2020).
, - To assess this, we need information on what would have
happened … (to Y) had … (X) not occurred.
- Difference in outcome between person being confronted
with treatment 1 and treatment 2.
- So, we need information about outcome after
treatment and outcome in absence of treatment.
To make a causal claim you need information on:
1. The outcome under treatment and
2. The outcome under absence of treatment.
In other words, you need information on all potential
outcomes to make a causal claim on the effect of X on Y. We
need information on outcome under all treatment options!
Fundamental problem in causal inference.
- Individual causal effect cannot be directly observed.
o Because we do not have information on counterfactual outcomes.
o Except under extremely strong (often unrealistic) assumptions.
- Average causal effects (i.e. in a population) cannot be determined based on individual
estimates:
o Causal inference is a problem of missing data!
- Solution: estimate an average causal effect by meeting three identifiability conditions.
Identifiability conditions.
- Average causal effect can be estimated if, and only if, all three identifiability conditions are
met:
o Positivity.
o Consistency.
o Exchangeability.
- If all three conditions are met (and an association is found in the data):
o The association between an exposure (X) and outcome (Y) is an unbiased estimate of
a causal effect.
o You can validly make a causal claim.
Positivity.
- The positivity condition requires that each individual has a positive probability of being
assigned to each treatment arm (i.e., Pr(A=a)>0 for all treatment arms, of levels of X) – so no
chance of not having any individual in a particular treatment group.
- L’Oréal example: there should be women who use the powder and women who do not use
the powder (otherwise the counterfactual is missing). Positivity condition was not met,
because all 41 women used the powder.
Consistency.
- The consistency condition requires that the treatment (or exposure X) has to be well-
defined.
- L’Oréal example: what kind of powder? How often applied? Used under what conditions
(with or without make-up?) Used for how long? Used at what time of day? How much
powder? Consistency condition was not met, because ‘powder’ is not well-defined.
Exchangeability (main part of course, using DAGs).
, - The exchangeability condition requires that the individuals assigned to the different
treatment arms are comparable.
- It does not matter who gets treatment A and who gets treatment B, the groups can be
thought of as “interchangeable”.
- This means that any difference in outcome between the groups can be attributed to the
treatment, rather than other differences between the individuals.
- L’Oréal example: women who used powder could have just as well not used it and vice
versa. Exchangeability condition was not met, because there were no two groups of women
that were comparable.
- Exchangeability condition is perfectly met if, and only if, the only difference between the
treatment groups (i.e., the case and control groups) is that one group has received the
treatment and the other has not.
Four ways to achieve exchangeability.
1. Randomized controlled trial (RCT).
a. Individuals are randomly assigned to one of the treatment arms.
b. Differences between individuals in the treatment arms are balanced out at the
group level.
c. These differences are independent of both treatment assignment and outcomes.
d. Differences are therefore random, not systematic.
e. RCTs are often considered the “gold standard” because, in principle, all identifiability
conditions are satisfied.
2. Matching.
a. For each individual with characteristics x, y, z who receives treatment A, there is an
individual with the same characteristics x, y, z who receives treatment B.
b. When perfect matching (e.g., using identical twins, triplets, …) is not possible,
statistical methods such as propensity score matching can be used to approximate
comparability between groups.
3. Stratification.
a. Randomly select individuals from different subsets (strata) of the larger population.
b. Ensures representation across key groups (e.g., age, gender, region).
c. Can be difficult to meet the positivity condition (i.e., having individuals available in
all strata for meaningful comparisons).
4. Adjustment.
a. Control for confounding factors that may bias the association between treatment
and outcome (commonly through regression analysis).
b. Assumes that individuals can, in principle, be assigned to all treatment arms across
all levels of the adjustment factors (positivity).
c. Can be combined with other designs/techniques: RCTs, stratification, matching.
d. Directed acyclic graphs (DAGs) are a useful tool to identify which factors to adjust for
(see Lectures 1.3 and 1.4).
1.3 Directed Acyclic Graphs I
