Advanced research Methods – GW4588M
Week 1:
Correlation/association does not imply causation.
Knowledge video 1: Directed Acyclic Graphs
DAG theory - Directed Acyclic Graphs are graphical representations of the causal structure
underlying a research question. Exposure x → Outcome y.
Are there factors that influence the exposure or/and outcome? Remove disruptive factors.
DAGs help to visualize the causal structure underlying a research question.
You need a priori theorical/subject knowledge about the causal structure to draw a DAG (previous
studies, literature, common sense… ).
Collect data on all relevant variables.
‘Simple’ rules can be applied to determine for which variables need to be corrected.
Dag terminology
1. Paths, directly related to the research question. Between Exposure x and Outcome y.
2. Causal paths and backdoor paths. Causal path follows the direction of the arrows, backdoor
path does not (arrows can go in different directions).
3. Open and closed paths, and colliders. All paths are open
unless they collide somewhere on a path. A path is closed if
arrows collide in one variable on that path.
4. Blocking open paths. Open (Causal or backdoor) paths
transmit association. The association between x and y
consists of the combination of all open paths between them.
Here: all paths except X → W Y. An open path is blocked
when we adjust for a variable (L) along the. This means that
we remove the disruptive influence of L from the association
between X and Y. How? By including variable L in the
regression analysis. Backdoor paths always need to be
closed. Causal paths need to be opened/closed depending on
RQ.
5. Opening blocked paths. Including a Collider (W) in the analysis means you open the blocked
backdoor path. This introduces bias in the association between X and Y.
Backdoor paths always need to be closed to answer the question is an unbiased manner.
If this is successful you can draw causal conclusions.
Lecture 1.2: Introduction to causal inference
Why do quantitative research?
Aim is to examine the association (or: relationship) between X and Y:
1. Description: Patterns X and Y
2. Prediction: Y given X
3. Causal inference: Effect X on Y. What is the effect on the outcome? Focus on the X variable
(different types).
,Example: Magazine advertisement → Is the make-up powder really that good?
‘Improves the quality of your skin’ implies a causal effect:
• X leads to Y
• Use of True Match Minerals powder (X) leads to a better skin (Y)
Question - Would you buy the powder?
• Is the scientific evidence convincing?
• What are arguments for and against buying the powder?
Answer
• Small sample size (n=41)
• Study performed or financed by commercial company
• All under dermatological control (?), but no control group.
– What would have happened had the women not used the powder?
– No information on other factors that may influence the result.
What do we want to know? In causal inference:
• We are not primarily interested in the outcome (Y) (i.e., 70% less imperfections), but ...
• We are interested in the role of the treatment (X) in achieving this outcome (i.e., without True
Match Minerals powder, would there be a difference in skin imperfections?)
Conclusion regarding L’Oréal study:
• We do not have that information
• No causal claim can be made based
Causal effect
Formal definition by Hernàn and Robins (2020):
‘In an individual, a treatment has a causal effect if the outcome under treatment 1 would be
different from the outcome under treatment 2.’
To assess this, we need information on:
→ What would have happened ... (to Y) had ... (X) not occured?
Assume that we have this information in relation to the L’Oréal study:
• Woman A treated with True Match Minerals powder: 2 bad spots
• Had woman A not used this powder: 5 bad spots
→ Individual treatment effect: -3 spots (or 60% less imperfections)
→ Average treatment effect: average of individual effects in a
population
To make a causal claim you need information on: (1) the outcome under treatment
and (2) the outcome under absence of
treatment. In other words, you need
information on all potential outcomes
to make a causal claim on the effect
of X on Y.
Formal notation of a causal effect:
Yia=1 =/ Yia=0
Y = Outcome
A = treatment
I = Individual
1 = yes (received treatment)
0 = no (received no treatment)
=/ does not equal
Fundamental problem in causal inference
Individual causal effect cannot be directly observed:
• Because we do not have information on counterfactual outcomes
,• Except under extremely strong (often unrealistic) assumptions
Average causal effect (i.e., in a population) cannot be determined based on individual estimates:
• Causal inference is a problem of missing data
Solution:
Estimate an average causal effect by meeting three identifiability conditions
Identifiability conditions
Average causal effect can be estimated if, and only if, three identifiability conditions are met:
1. Positivity
The positivity condition requires that:
• Each individual has a positive probability of being assigned to each treatment arm (i.e.,
Pr(A=a)>0 for all treatment arms, of levels of X).
L’Oréal example:
• There should be women who use the powder and women who do not use the powder
(otherwise the counterfactual is missing)
• Positivity condition was not met, because all 41 women used the powder
2. Consistency
The consistency condition requires that:
• The treatment (or exposure X) has to be well-defined.
L’Oréal example:
• What kind of powder? How often applied? Used under what conditions (with or without
make-up?) Used for how long? Used at what time of day? How much powder?
• Consistency condition was not met, because ‘powder’ is not well-defined
3. Exchangeability
The exchangeability condition requires that:
• The individuals assigned to the different treatment arms are comparable
• It does not matter who gets treatment A and who gets treatment B, the groups can be
thought of as “interchangeable”
• This means that any difference in outcome between the groups can be attributed to the
treatment, rather than other differences between the individuals
L’Oréal example:
• Women who used powder could have just as well not used it and vice versa
- Exchangeability condition was not met, because there were no two
groups of women that were comparable.
Exchangeability condition is perfectly met if, and only if, the only difference between the
treatment groups (i.e., the case and control groups) is that one group has received the
treatment and the other has not. Unfortunately, this is difficult to achieve in quantitative
research.
Four ways to achieve exchangeability
1. Randomized Controlled Trial (RCT)
• Individuals are randomly assigned to one of the treatment arms
• Differences between individuals in the treatment arms are balanced out at the group level
(when the group is big).
• These differences are independent of both treatment assignment and outcomes
• Differences are therefore random, not systematic
RCTs are often considered the “gold standard” because, in principle, all identifiability
conditions are satisfied.
2. Matching
• For each individual with characteristics x, y, z who receives treatment A, there is an
individual with the same characteristics x, y, z who receives treatment B
• When perfect matching (e.g., using identical twins, triplets, ...) is not possible, statistical
, methods such as propensity score matching can be used to approximate comparability
between groups.
3. Stratification
• Randomly select individuals from different subsets (strata) of the larger population
• Ensures representation across key groups (e.g., age, gender, region)
• Can be difficult to meet the positivity condition (i.e., having individuals available in all
strata for meaningful comparisons)
4. Adjustment (focus of this course)
• Control for confounding factors that may bias the association between treatment and
outcome (commonly through regression analysis)
• Assumes that individuals can, in principle, be assigned to all treatment arms across all
levels of the adjustment factors (positivity)
• Can be combined with other designs/techniques: RCTs, stratification, matching
• Directed acyclic graphs (DAGs) are a useful tool to identify which factors to adjust for.
If all three conditions are met (and an association is found in the data):
• The association between an exposure (X) and outcome (Y) is an unbiased estimate of a causal
effect.
• You can validly make a causal claim.
Lecture 1.3 Directed Acyclic Graphs l
What do we want to know?
People (including managers, policymakers, researchers in healthcare) are often interested in causal
effects, not just associations:
What is the effect of nurses’ job satisfaction on the health of ICU patients?
How does universal healthcare impact health inequalities in a country?
What is the influence of ... on ...
What is the risk of ... on ...
What would have happened to ... if ... had (not) been different
Causal claims are made everywhere, all the time ...
Important: Association ≠ Causation
Association:
• A statistical relationship between the treatment X and outcome Y
• Knowing the value of one variable may provide information on the value of another variable, but
that does not mean that one caused the other
• Knowing that Zeus died 5 days after a heart transplant does not mean the transplant caused Zeus’
death (Hernàn and Robins, 2020)
Causation:
• Difference between potential (i.e., counterfactual) outcomes
• Achievement of three identifiability conditions in research design and analysis: positivity,
consistency, and exchangeability
Achievement of exchangeability by adjustment
Exchangeability can be achieved by complete and correct adjustment.
Adjustment:
• We want to examine the effect of X on Y, so we want to adjust (or: control, or correct) for any
other factors that may influence the association between X and Y
• By definition: multivariate regression analysis
Important:
• Use DAGs to determine what to adjust for in the analysis
Week 1:
Correlation/association does not imply causation.
Knowledge video 1: Directed Acyclic Graphs
DAG theory - Directed Acyclic Graphs are graphical representations of the causal structure
underlying a research question. Exposure x → Outcome y.
Are there factors that influence the exposure or/and outcome? Remove disruptive factors.
DAGs help to visualize the causal structure underlying a research question.
You need a priori theorical/subject knowledge about the causal structure to draw a DAG (previous
studies, literature, common sense… ).
Collect data on all relevant variables.
‘Simple’ rules can be applied to determine for which variables need to be corrected.
Dag terminology
1. Paths, directly related to the research question. Between Exposure x and Outcome y.
2. Causal paths and backdoor paths. Causal path follows the direction of the arrows, backdoor
path does not (arrows can go in different directions).
3. Open and closed paths, and colliders. All paths are open
unless they collide somewhere on a path. A path is closed if
arrows collide in one variable on that path.
4. Blocking open paths. Open (Causal or backdoor) paths
transmit association. The association between x and y
consists of the combination of all open paths between them.
Here: all paths except X → W Y. An open path is blocked
when we adjust for a variable (L) along the. This means that
we remove the disruptive influence of L from the association
between X and Y. How? By including variable L in the
regression analysis. Backdoor paths always need to be
closed. Causal paths need to be opened/closed depending on
RQ.
5. Opening blocked paths. Including a Collider (W) in the analysis means you open the blocked
backdoor path. This introduces bias in the association between X and Y.
Backdoor paths always need to be closed to answer the question is an unbiased manner.
If this is successful you can draw causal conclusions.
Lecture 1.2: Introduction to causal inference
Why do quantitative research?
Aim is to examine the association (or: relationship) between X and Y:
1. Description: Patterns X and Y
2. Prediction: Y given X
3. Causal inference: Effect X on Y. What is the effect on the outcome? Focus on the X variable
(different types).
,Example: Magazine advertisement → Is the make-up powder really that good?
‘Improves the quality of your skin’ implies a causal effect:
• X leads to Y
• Use of True Match Minerals powder (X) leads to a better skin (Y)
Question - Would you buy the powder?
• Is the scientific evidence convincing?
• What are arguments for and against buying the powder?
Answer
• Small sample size (n=41)
• Study performed or financed by commercial company
• All under dermatological control (?), but no control group.
– What would have happened had the women not used the powder?
– No information on other factors that may influence the result.
What do we want to know? In causal inference:
• We are not primarily interested in the outcome (Y) (i.e., 70% less imperfections), but ...
• We are interested in the role of the treatment (X) in achieving this outcome (i.e., without True
Match Minerals powder, would there be a difference in skin imperfections?)
Conclusion regarding L’Oréal study:
• We do not have that information
• No causal claim can be made based
Causal effect
Formal definition by Hernàn and Robins (2020):
‘In an individual, a treatment has a causal effect if the outcome under treatment 1 would be
different from the outcome under treatment 2.’
To assess this, we need information on:
→ What would have happened ... (to Y) had ... (X) not occured?
Assume that we have this information in relation to the L’Oréal study:
• Woman A treated with True Match Minerals powder: 2 bad spots
• Had woman A not used this powder: 5 bad spots
→ Individual treatment effect: -3 spots (or 60% less imperfections)
→ Average treatment effect: average of individual effects in a
population
To make a causal claim you need information on: (1) the outcome under treatment
and (2) the outcome under absence of
treatment. In other words, you need
information on all potential outcomes
to make a causal claim on the effect
of X on Y.
Formal notation of a causal effect:
Yia=1 =/ Yia=0
Y = Outcome
A = treatment
I = Individual
1 = yes (received treatment)
0 = no (received no treatment)
=/ does not equal
Fundamental problem in causal inference
Individual causal effect cannot be directly observed:
• Because we do not have information on counterfactual outcomes
,• Except under extremely strong (often unrealistic) assumptions
Average causal effect (i.e., in a population) cannot be determined based on individual estimates:
• Causal inference is a problem of missing data
Solution:
Estimate an average causal effect by meeting three identifiability conditions
Identifiability conditions
Average causal effect can be estimated if, and only if, three identifiability conditions are met:
1. Positivity
The positivity condition requires that:
• Each individual has a positive probability of being assigned to each treatment arm (i.e.,
Pr(A=a)>0 for all treatment arms, of levels of X).
L’Oréal example:
• There should be women who use the powder and women who do not use the powder
(otherwise the counterfactual is missing)
• Positivity condition was not met, because all 41 women used the powder
2. Consistency
The consistency condition requires that:
• The treatment (or exposure X) has to be well-defined.
L’Oréal example:
• What kind of powder? How often applied? Used under what conditions (with or without
make-up?) Used for how long? Used at what time of day? How much powder?
• Consistency condition was not met, because ‘powder’ is not well-defined
3. Exchangeability
The exchangeability condition requires that:
• The individuals assigned to the different treatment arms are comparable
• It does not matter who gets treatment A and who gets treatment B, the groups can be
thought of as “interchangeable”
• This means that any difference in outcome between the groups can be attributed to the
treatment, rather than other differences between the individuals
L’Oréal example:
• Women who used powder could have just as well not used it and vice versa
- Exchangeability condition was not met, because there were no two
groups of women that were comparable.
Exchangeability condition is perfectly met if, and only if, the only difference between the
treatment groups (i.e., the case and control groups) is that one group has received the
treatment and the other has not. Unfortunately, this is difficult to achieve in quantitative
research.
Four ways to achieve exchangeability
1. Randomized Controlled Trial (RCT)
• Individuals are randomly assigned to one of the treatment arms
• Differences between individuals in the treatment arms are balanced out at the group level
(when the group is big).
• These differences are independent of both treatment assignment and outcomes
• Differences are therefore random, not systematic
RCTs are often considered the “gold standard” because, in principle, all identifiability
conditions are satisfied.
2. Matching
• For each individual with characteristics x, y, z who receives treatment A, there is an
individual with the same characteristics x, y, z who receives treatment B
• When perfect matching (e.g., using identical twins, triplets, ...) is not possible, statistical
, methods such as propensity score matching can be used to approximate comparability
between groups.
3. Stratification
• Randomly select individuals from different subsets (strata) of the larger population
• Ensures representation across key groups (e.g., age, gender, region)
• Can be difficult to meet the positivity condition (i.e., having individuals available in all
strata for meaningful comparisons)
4. Adjustment (focus of this course)
• Control for confounding factors that may bias the association between treatment and
outcome (commonly through regression analysis)
• Assumes that individuals can, in principle, be assigned to all treatment arms across all
levels of the adjustment factors (positivity)
• Can be combined with other designs/techniques: RCTs, stratification, matching
• Directed acyclic graphs (DAGs) are a useful tool to identify which factors to adjust for.
If all three conditions are met (and an association is found in the data):
• The association between an exposure (X) and outcome (Y) is an unbiased estimate of a causal
effect.
• You can validly make a causal claim.
Lecture 1.3 Directed Acyclic Graphs l
What do we want to know?
People (including managers, policymakers, researchers in healthcare) are often interested in causal
effects, not just associations:
What is the effect of nurses’ job satisfaction on the health of ICU patients?
How does universal healthcare impact health inequalities in a country?
What is the influence of ... on ...
What is the risk of ... on ...
What would have happened to ... if ... had (not) been different
Causal claims are made everywhere, all the time ...
Important: Association ≠ Causation
Association:
• A statistical relationship between the treatment X and outcome Y
• Knowing the value of one variable may provide information on the value of another variable, but
that does not mean that one caused the other
• Knowing that Zeus died 5 days after a heart transplant does not mean the transplant caused Zeus’
death (Hernàn and Robins, 2020)
Causation:
• Difference between potential (i.e., counterfactual) outcomes
• Achievement of three identifiability conditions in research design and analysis: positivity,
consistency, and exchangeability
Achievement of exchangeability by adjustment
Exchangeability can be achieved by complete and correct adjustment.
Adjustment:
• We want to examine the effect of X on Y, so we want to adjust (or: control, or correct) for any
other factors that may influence the association between X and Y
• By definition: multivariate regression analysis
Important:
• Use DAGs to determine what to adjust for in the analysis
