DATA1001/1901 Comprehensive Course
Notes
WEEK 1 - Study Design, Bias, and Research Methods
Confounding Variables
Definition: Confounding occurs when the control group and treatment group differ by a third lurking variable that is
difficult to notice or account for. This variable is not the treatment of interest but has an effect on the outcome, creating
spurious associations or masking true relationships.
Key Characteristics:
• The confounding variable is associated with both the treatment assignment and the outcome
• It exists outside the experimental manipulation
• It can completely alter or mask the true relationship between treatment and outcome
• Confounders often operate "in the background" and may not be immediately visible
Example Scenario: When studying whether a new medication improves patient recovery, age could be a confounder. If
older patients are more likely to receive the medication (treatment assignment related to confounder) AND older
patients naturally have longer recovery times (confounder related to outcome), the medication's true effect becomes
obscured or distorted.
Implications: Confounding is one of the most serious threats to validity in observational studies and highlights why
randomization is so important in experimental design.
Selection Bias
Definition: Selection bias occurs when some individuals are systematically more or less likely to be selected or
included in a study compared to others. This creates a non-representative sample that does not accurately reflect the
target population.
Key Characteristics:
• The mechanism of selection is not random
• Certain population subgroups are over- or under-represented
• The bias is inherent in how participants enter the study, not in treatment assignment
, • Can occur in both observational studies and poorly designed experiments
Classic Example - Surgical Selection: Healthier patients may be more likely to be chosen for or to volunteer for
surgery compared to sicker patients. This means:
• The study sample skews toward healthier individuals
• Any apparent benefit of surgery may actually reflect that healthier people naturally recover better
• The true surgical effect is confounded with baseline health status
• This is sometimes called "healthy user bias" or "allocation bias"
Other Examples:
• Online surveys attracting only internet-literate users
• Clinical trials excluding the most severely ill patients
• Labor force participation studies missing unemployed individuals
• Healthcare studies over-representing insured populations
Impact on Research: Selection bias threatens external validity (generalizability) and can create spurious associations
that do not hold in the broader population.
Randomised Control Trial (RCT)
Definition: A Randomised Control Trial is an experimental research design where participants are randomly allocated
to either a control group (receives standard treatment or placebo) or a treatment group (receives the intervention of
interest).
Core Principle: Randomization ensures that treatment assignment is independent of any participant characteristics,
both known and unknown. This is the most powerful method for establishing causal relationships.
How Randomization Works:
1. Eligible participants are enrolled
2. They are randomly assigned to treatment or control groups (using random number generators, coin flips, etc.)
3. Each participant has an equal probability of receiving any treatment condition
4. This random allocation is done BEFORE treatment begins
Key Advantages:
• Eliminates selection bias in treatment allocation
• Creates comparable groups on average (for both measured and unmeasured variables)
• Allows causal inference about the treatment effect
, • Controls for confounding through randomization rather than statistical adjustment
• The "gold standard" for causal inference
Why It Matters: Because randomization balances all baseline characteristics between groups (on average), any
systematic differences in outcomes between groups can be attributed to the treatment rather than pre-existing
differences.
Limitations:
• Not always ethically or practically feasible (e.g., cannot randomize people to smoking)
• Expensive and time-consuming
• May have lower external validity than observational studies in some cases
• Requires careful implementation to prevent bias during randomization
Observer Bias
Definition: Observer bias (also called detection bias or measurement bias) occurs when the participants, investigators,
or outcome assessors are aware of which group received the treatment versus control. This knowledge can
unconsciously influence how they behave, report, or measure outcomes.
Mechanisms of Observer Bias:
1. Participant Bias:
- Participants who know they received a "real" treatment may report better outcomes (placebo effect)
- Participants who know they received a placebo may be discouraged and report worse outcomes
- Behavioral changes occur in response to knowledge of assignment
2. Investigator Bias:
- Researchers expecting a treatment to work may unconsciously:
- Interpret ambiguous results favorably for the treatment group
- Put more effort into data collection for one group
- Be more encouraging or supportive to treatment group participants
- Ask leading questions differently to different groups
- Make different judgments in subjective outcome assessments
3. Outcome Measurement Bias:
- If assessors know the group assignment, they may measure or rate outcomes differently
, - Blood pressure readings may be recorded with different precision for different groups
- Subjective health assessments may be scored differently
How It Distorts Results: Observer bias typically inflates differences between groups or creates false differences where
none exist, ultimately undermining the validity of study conclusions.
Solutions (Blinding):
• Single Blinding: Participants don't know their assignment, but researchers do
• Double Blinding: Neither participants nor investigators know who received which treatment
• Triple Blinding: Participants, investigators, and data analysts are all blinded to treatment assignment
When Blinding is Impossible:
• Surgical interventions (participants obviously know they had surgery)
• Behavioral interventions (impossible to hide a therapy from the person receiving it)
• In these cases, researchers must be especially vigilant about objectifying outcomes
Observational Studies
Definition: An observational study is research where the investigator observes and records outcomes without randomly
assigning participants to treatment conditions. Treatment assignment occurs naturally or through participant choice,
outside the investigator's control.
Key Characteristics:
• No randomization or experimental manipulation by the researcher
• Participants self-select or are assigned to conditions based on existing circumstances
• The investigator is a passive observer, not an active manipulator
• Treatment and control groups may differ in many ways beyond the treatment itself
Types of Observational Studies:
1. Cross-Sectional Studies:
- Data collected at a single point in time
- Measures prevalence of outcomes
- Cannot establish temporal sequence (which came first?)
- Example: Survey asking current smokers about lung health
2. Case-Control Studies:
Notes
WEEK 1 - Study Design, Bias, and Research Methods
Confounding Variables
Definition: Confounding occurs when the control group and treatment group differ by a third lurking variable that is
difficult to notice or account for. This variable is not the treatment of interest but has an effect on the outcome, creating
spurious associations or masking true relationships.
Key Characteristics:
• The confounding variable is associated with both the treatment assignment and the outcome
• It exists outside the experimental manipulation
• It can completely alter or mask the true relationship between treatment and outcome
• Confounders often operate "in the background" and may not be immediately visible
Example Scenario: When studying whether a new medication improves patient recovery, age could be a confounder. If
older patients are more likely to receive the medication (treatment assignment related to confounder) AND older
patients naturally have longer recovery times (confounder related to outcome), the medication's true effect becomes
obscured or distorted.
Implications: Confounding is one of the most serious threats to validity in observational studies and highlights why
randomization is so important in experimental design.
Selection Bias
Definition: Selection bias occurs when some individuals are systematically more or less likely to be selected or
included in a study compared to others. This creates a non-representative sample that does not accurately reflect the
target population.
Key Characteristics:
• The mechanism of selection is not random
• Certain population subgroups are over- or under-represented
• The bias is inherent in how participants enter the study, not in treatment assignment
, • Can occur in both observational studies and poorly designed experiments
Classic Example - Surgical Selection: Healthier patients may be more likely to be chosen for or to volunteer for
surgery compared to sicker patients. This means:
• The study sample skews toward healthier individuals
• Any apparent benefit of surgery may actually reflect that healthier people naturally recover better
• The true surgical effect is confounded with baseline health status
• This is sometimes called "healthy user bias" or "allocation bias"
Other Examples:
• Online surveys attracting only internet-literate users
• Clinical trials excluding the most severely ill patients
• Labor force participation studies missing unemployed individuals
• Healthcare studies over-representing insured populations
Impact on Research: Selection bias threatens external validity (generalizability) and can create spurious associations
that do not hold in the broader population.
Randomised Control Trial (RCT)
Definition: A Randomised Control Trial is an experimental research design where participants are randomly allocated
to either a control group (receives standard treatment or placebo) or a treatment group (receives the intervention of
interest).
Core Principle: Randomization ensures that treatment assignment is independent of any participant characteristics,
both known and unknown. This is the most powerful method for establishing causal relationships.
How Randomization Works:
1. Eligible participants are enrolled
2. They are randomly assigned to treatment or control groups (using random number generators, coin flips, etc.)
3. Each participant has an equal probability of receiving any treatment condition
4. This random allocation is done BEFORE treatment begins
Key Advantages:
• Eliminates selection bias in treatment allocation
• Creates comparable groups on average (for both measured and unmeasured variables)
• Allows causal inference about the treatment effect
, • Controls for confounding through randomization rather than statistical adjustment
• The "gold standard" for causal inference
Why It Matters: Because randomization balances all baseline characteristics between groups (on average), any
systematic differences in outcomes between groups can be attributed to the treatment rather than pre-existing
differences.
Limitations:
• Not always ethically or practically feasible (e.g., cannot randomize people to smoking)
• Expensive and time-consuming
• May have lower external validity than observational studies in some cases
• Requires careful implementation to prevent bias during randomization
Observer Bias
Definition: Observer bias (also called detection bias or measurement bias) occurs when the participants, investigators,
or outcome assessors are aware of which group received the treatment versus control. This knowledge can
unconsciously influence how they behave, report, or measure outcomes.
Mechanisms of Observer Bias:
1. Participant Bias:
- Participants who know they received a "real" treatment may report better outcomes (placebo effect)
- Participants who know they received a placebo may be discouraged and report worse outcomes
- Behavioral changes occur in response to knowledge of assignment
2. Investigator Bias:
- Researchers expecting a treatment to work may unconsciously:
- Interpret ambiguous results favorably for the treatment group
- Put more effort into data collection for one group
- Be more encouraging or supportive to treatment group participants
- Ask leading questions differently to different groups
- Make different judgments in subjective outcome assessments
3. Outcome Measurement Bias:
- If assessors know the group assignment, they may measure or rate outcomes differently
, - Blood pressure readings may be recorded with different precision for different groups
- Subjective health assessments may be scored differently
How It Distorts Results: Observer bias typically inflates differences between groups or creates false differences where
none exist, ultimately undermining the validity of study conclusions.
Solutions (Blinding):
• Single Blinding: Participants don't know their assignment, but researchers do
• Double Blinding: Neither participants nor investigators know who received which treatment
• Triple Blinding: Participants, investigators, and data analysts are all blinded to treatment assignment
When Blinding is Impossible:
• Surgical interventions (participants obviously know they had surgery)
• Behavioral interventions (impossible to hide a therapy from the person receiving it)
• In these cases, researchers must be especially vigilant about objectifying outcomes
Observational Studies
Definition: An observational study is research where the investigator observes and records outcomes without randomly
assigning participants to treatment conditions. Treatment assignment occurs naturally or through participant choice,
outside the investigator's control.
Key Characteristics:
• No randomization or experimental manipulation by the researcher
• Participants self-select or are assigned to conditions based on existing circumstances
• The investigator is a passive observer, not an active manipulator
• Treatment and control groups may differ in many ways beyond the treatment itself
Types of Observational Studies:
1. Cross-Sectional Studies:
- Data collected at a single point in time
- Measures prevalence of outcomes
- Cannot establish temporal sequence (which came first?)
- Example: Survey asking current smokers about lung health
2. Case-Control Studies:
