INTRODUCTION TO RESEARCH METHODS
Natalia Pułtorak, 2022
MODULE 8. Chapter 8
Causality:
- Science often aims at detecting causal relationships
- Causal inference requires appropriate research design
- Conclusions strongly depend on research design
- Correlation≠causation
Treatment effect = (Person A with treatment) - (Person A without treatment)
but...person A cannot have both (at the same time)
Treatment effect = (Person A with treatment) - (Person B without treatment)
but...person B might be different from person A
3 requirements causality
- Temporal order (cause before effect)
- Association (significant relation)
- No alternative explanations
Control group comparing the results of a tested intervention with a control group is necessary
to exclude alternative explanations
Random selection Randomly selecting sampling
elements (e.g. participants) from the population for
inclusion in the study
Random assignment in experimental groups is
required to prevent confounding differences between
groups
Matching assignment participants having the same
characteristics get grouped into pairs, then within each
pair, 1 participant gets randomly assigned to either the
treatment or the control group and the other is
automatically assigned to the other group.
- isn’t used often because of its limitation: individual cases differ in thousands of ways
and you cannot decide based on which relevant characteristic you will divide them
Multiple measurements using more than one measurement allows for investigating individual
change rather besides group differences
1
,PARTS OF THE EXPERIMENT
1. Treatment or independent variable 4. Posttest
(or stimulus or manipulation) 5. Experimental group
2. Dependent variable 6. Control group
3. Pretest 7. Random assignment
1) TREATMENT you create a situation or enter into an ongoing situation and modify it.
- The treatment is the independent variable or a combination of independent
variables (e.g. physical setting, made up social situations, false records)
2) DEPENDENT VARIABLES or outcomes, are physical conditions, social behaviors,
attitudes, feelings, or beliefs of participants that change in response to a treatment.
- You can measure dependent variables by paper-and-pencil indicators,
observations, interviews, or physiological responses
3) PRETEST – measurement of the dependent variable prior to introduction of the treatment
4) POSTTEST - the measurement of the dependent variable after the treatment has been
introduced into the experimental situation
5) THE EXPERIMENTAL GROUP - receives the treatment or is the one in which the
treatment is present.
6) CONTROL GROUP - The group that does not receive the treatment is the control group
- When the independent variable has several values, you can have more than one
experimental group)
Subjects = participants
STEPS IN CONDUCTING AN EXPERIMENT
the basic steps of the research process: you decide on a topic, narrow it into a testable
research question, and then develop a hypothesis with variables. A crucial early step is to plan a
specific experimental design, how many groups to create
Control in Experiments: Control is crucial in experimental research
- you want to control all aspects of the experimental situation in order to isolate the effects
of the treatment.
- By controlling confounding variables you eliminate alternative explanations
Use of deception – intentionally misled participants through written or verbal instructions, the
actions of others, or aspects of the setting:
confederate – someone who pretends to be A cover story is a false explanation of
another research participant or bystander but the study’s purpose that you tell to
who actually works for the researcher and participants to mislead them about its
deliberately misleads participants. true purpose
2
, TYPES OF DESIGN
Classical Experimental Design: random assignment, a pretest and a posttest, an experimental
group, and a control group
Pre Experimental Designs lack random assignments and are compromises/shortcuts. You can
use them in situations in which it is difficult to use the classical design. Inferring a causal
relationship from them can be less clear than using the classical design.
Design notation is a shorthand system for symbolizing experimental design, it can express a
complex descriptions of the parts of an experiment in 5 or 6 symbols arranged in 2 lines
O = observation of dependent variable (O1 = pretests, O2= posttests)
X = treatment; independent variable (Xs = when the independent variable has more than two
levels, the s is used to distinguish them)
R = random assignment
➔ You put symbols in time order from left to right, R is first – pretest – treatment – posttest
➔ Symbols are arranged in rows, each row represents a group of participants
One shot case study
- No control group
- No random assignment
- Only one measurement (posttest)
- Example: positive case reports
MDMA therapy
One group pretest posttest
- No control group
- No random assignment
- Two measurements (pretest & posttest)
Interrupted time-series design
- No control group
- No random assignment
- Many measurements before and after
intervention
3
Natalia Pułtorak, 2022
MODULE 8. Chapter 8
Causality:
- Science often aims at detecting causal relationships
- Causal inference requires appropriate research design
- Conclusions strongly depend on research design
- Correlation≠causation
Treatment effect = (Person A with treatment) - (Person A without treatment)
but...person A cannot have both (at the same time)
Treatment effect = (Person A with treatment) - (Person B without treatment)
but...person B might be different from person A
3 requirements causality
- Temporal order (cause before effect)
- Association (significant relation)
- No alternative explanations
Control group comparing the results of a tested intervention with a control group is necessary
to exclude alternative explanations
Random selection Randomly selecting sampling
elements (e.g. participants) from the population for
inclusion in the study
Random assignment in experimental groups is
required to prevent confounding differences between
groups
Matching assignment participants having the same
characteristics get grouped into pairs, then within each
pair, 1 participant gets randomly assigned to either the
treatment or the control group and the other is
automatically assigned to the other group.
- isn’t used often because of its limitation: individual cases differ in thousands of ways
and you cannot decide based on which relevant characteristic you will divide them
Multiple measurements using more than one measurement allows for investigating individual
change rather besides group differences
1
,PARTS OF THE EXPERIMENT
1. Treatment or independent variable 4. Posttest
(or stimulus or manipulation) 5. Experimental group
2. Dependent variable 6. Control group
3. Pretest 7. Random assignment
1) TREATMENT you create a situation or enter into an ongoing situation and modify it.
- The treatment is the independent variable or a combination of independent
variables (e.g. physical setting, made up social situations, false records)
2) DEPENDENT VARIABLES or outcomes, are physical conditions, social behaviors,
attitudes, feelings, or beliefs of participants that change in response to a treatment.
- You can measure dependent variables by paper-and-pencil indicators,
observations, interviews, or physiological responses
3) PRETEST – measurement of the dependent variable prior to introduction of the treatment
4) POSTTEST - the measurement of the dependent variable after the treatment has been
introduced into the experimental situation
5) THE EXPERIMENTAL GROUP - receives the treatment or is the one in which the
treatment is present.
6) CONTROL GROUP - The group that does not receive the treatment is the control group
- When the independent variable has several values, you can have more than one
experimental group)
Subjects = participants
STEPS IN CONDUCTING AN EXPERIMENT
the basic steps of the research process: you decide on a topic, narrow it into a testable
research question, and then develop a hypothesis with variables. A crucial early step is to plan a
specific experimental design, how many groups to create
Control in Experiments: Control is crucial in experimental research
- you want to control all aspects of the experimental situation in order to isolate the effects
of the treatment.
- By controlling confounding variables you eliminate alternative explanations
Use of deception – intentionally misled participants through written or verbal instructions, the
actions of others, or aspects of the setting:
confederate – someone who pretends to be A cover story is a false explanation of
another research participant or bystander but the study’s purpose that you tell to
who actually works for the researcher and participants to mislead them about its
deliberately misleads participants. true purpose
2
, TYPES OF DESIGN
Classical Experimental Design: random assignment, a pretest and a posttest, an experimental
group, and a control group
Pre Experimental Designs lack random assignments and are compromises/shortcuts. You can
use them in situations in which it is difficult to use the classical design. Inferring a causal
relationship from them can be less clear than using the classical design.
Design notation is a shorthand system for symbolizing experimental design, it can express a
complex descriptions of the parts of an experiment in 5 or 6 symbols arranged in 2 lines
O = observation of dependent variable (O1 = pretests, O2= posttests)
X = treatment; independent variable (Xs = when the independent variable has more than two
levels, the s is used to distinguish them)
R = random assignment
➔ You put symbols in time order from left to right, R is first – pretest – treatment – posttest
➔ Symbols are arranged in rows, each row represents a group of participants
One shot case study
- No control group
- No random assignment
- Only one measurement (posttest)
- Example: positive case reports
MDMA therapy
One group pretest posttest
- No control group
- No random assignment
- Two measurements (pretest & posttest)
Interrupted time-series design
- No control group
- No random assignment
- Many measurements before and after
intervention
3
