Chapter 5: Introduction to
Experimental Research
Research uses 2 methods:
● method of agreement → if X then Y → accomplished via experimental group
● method of disagreement → if not X then not Y → accomplished via control group
both criteria are never fully met in research → joint method should be the standard
Experiment: systematic research in which the investigator directly varies some
factor(s), holds all others constant, and observes the results of the variation
- Independent variables: controlled by experimenter (subject of interest)
- Subject variables: preexisting participant characteristics (e.g. gender)
- Extraneous variables: held constant
- Dependent variables: measured behaviours
1. Manipulated Independent Variables
Minimum of 2 levels (a.k.a., conditions). Types of independent variables:
- Situational variables: can be encountered in natural environments
- Task variables: tasks to be executed by participants
- Instructional variables: same task executed in different ways by diff participants
- Control/experimental group: placed in treatment or comparison group
2. Subject Variables
With manipulated variables, conclusions about the causes of behaviour can be made
with some degree of confidence; with subject variables, causal conclusions cannot be
drawn. The reason has to do with the experimenter’s amount of control. With
manipulated variables, the experiment can meet the criteria for demonstrating causality.
3. Extraneous Variables
Confounder: uncontrolled extraneous variable that covaries with the independent
variable and could provide an alternative explanation for the results.
4. Dependent Variables
Ceiling effect: when the average group scores are so high that no difference can be
determined between conditions
,Floor effect: when all the scores are extremely low and no difference can be
determined between conditions (usually because the task is too difficult for everyone)
The Validity of Experimental Research
Statistical conclusion validity extent to which the researcher uses statistics
properly and draws the appropriate conclusions
from them
Construct validity the adequacy of operational definitions for both
the independent and the dependent variables
Convergent validity A measure displays theoretically expected
correlations with other measures
External validity whether research findings generalise beyond
the experimental context
Ecological validity Whether research findings generalise from
artificial to natural environments
Internal validity degree to which an experiment is
methodologically sound and confound-free
Threats to internal validity
- Comparisons between nonequivalent groups
- Absence of control groups (sometimes, depends on context)
- History: significant events occurring between pre- and post-test
- Maturation: natural human changes between pre- and post-test
- Regression to the mean: having an initial extreme score and a second less
extreme score probabilistically
- Testing/practice effect: pre-test has sensitising effect on post-test scores
- Instrumentation: measurement instrument changes between pre- and post-test
- Subject selection effect: respondent sample or analysis is biassed toward a
specific subset of a target population
- Attrition: subjects dropping out before experiment completion
, Chapter 6: Methodological Control in
Experimental Research
Between-subjects design: participants are in group A or B
Within-subjects (repeated measures) design: participants are in groups A and B
Between subjects
➢ Participants are naive to the research hypothesis
➢ Large numbers of people may need to be recruited, tested, and debriefed
➢ Differences between conditions might be due to the independent variables or
differences between the individuals (solved by making equivalent groups)
Creating equivalent groups
1. Random assignment → once selected for the study, every participant has an
equal chance of being placed in any of the groups
- blocked random assignment → ensures each condition has an equal
amount of randomly assigned participants
2. Matching → participants are grouped together based on a subject variable
(matching variable) and are then randomly distributed to conditions. Good for
small participant numbers.
- must be confident that matching and dependent variables are correlated
- must be possible to assess participants on the matching variable
Within subjects
➢ Fewer people need to be recruited
➢ Eliminates the equivalent groups problem
➢ Order effect: type of sequencing effect that is a consequence of the order in
which participants are administered the experimental conditions
- Progressive effects → performance changes progressively from trial to trial
- Carryover effect → some sequences produce different effects from others
Experimental Research
Research uses 2 methods:
● method of agreement → if X then Y → accomplished via experimental group
● method of disagreement → if not X then not Y → accomplished via control group
both criteria are never fully met in research → joint method should be the standard
Experiment: systematic research in which the investigator directly varies some
factor(s), holds all others constant, and observes the results of the variation
- Independent variables: controlled by experimenter (subject of interest)
- Subject variables: preexisting participant characteristics (e.g. gender)
- Extraneous variables: held constant
- Dependent variables: measured behaviours
1. Manipulated Independent Variables
Minimum of 2 levels (a.k.a., conditions). Types of independent variables:
- Situational variables: can be encountered in natural environments
- Task variables: tasks to be executed by participants
- Instructional variables: same task executed in different ways by diff participants
- Control/experimental group: placed in treatment or comparison group
2. Subject Variables
With manipulated variables, conclusions about the causes of behaviour can be made
with some degree of confidence; with subject variables, causal conclusions cannot be
drawn. The reason has to do with the experimenter’s amount of control. With
manipulated variables, the experiment can meet the criteria for demonstrating causality.
3. Extraneous Variables
Confounder: uncontrolled extraneous variable that covaries with the independent
variable and could provide an alternative explanation for the results.
4. Dependent Variables
Ceiling effect: when the average group scores are so high that no difference can be
determined between conditions
,Floor effect: when all the scores are extremely low and no difference can be
determined between conditions (usually because the task is too difficult for everyone)
The Validity of Experimental Research
Statistical conclusion validity extent to which the researcher uses statistics
properly and draws the appropriate conclusions
from them
Construct validity the adequacy of operational definitions for both
the independent and the dependent variables
Convergent validity A measure displays theoretically expected
correlations with other measures
External validity whether research findings generalise beyond
the experimental context
Ecological validity Whether research findings generalise from
artificial to natural environments
Internal validity degree to which an experiment is
methodologically sound and confound-free
Threats to internal validity
- Comparisons between nonequivalent groups
- Absence of control groups (sometimes, depends on context)
- History: significant events occurring between pre- and post-test
- Maturation: natural human changes between pre- and post-test
- Regression to the mean: having an initial extreme score and a second less
extreme score probabilistically
- Testing/practice effect: pre-test has sensitising effect on post-test scores
- Instrumentation: measurement instrument changes between pre- and post-test
- Subject selection effect: respondent sample or analysis is biassed toward a
specific subset of a target population
- Attrition: subjects dropping out before experiment completion
, Chapter 6: Methodological Control in
Experimental Research
Between-subjects design: participants are in group A or B
Within-subjects (repeated measures) design: participants are in groups A and B
Between subjects
➢ Participants are naive to the research hypothesis
➢ Large numbers of people may need to be recruited, tested, and debriefed
➢ Differences between conditions might be due to the independent variables or
differences between the individuals (solved by making equivalent groups)
Creating equivalent groups
1. Random assignment → once selected for the study, every participant has an
equal chance of being placed in any of the groups
- blocked random assignment → ensures each condition has an equal
amount of randomly assigned participants
2. Matching → participants are grouped together based on a subject variable
(matching variable) and are then randomly distributed to conditions. Good for
small participant numbers.
- must be confident that matching and dependent variables are correlated
- must be possible to assess participants on the matching variable
Within subjects
➢ Fewer people need to be recruited
➢ Eliminates the equivalent groups problem
➢ Order effect: type of sequencing effect that is a consequence of the order in
which participants are administered the experimental conditions
- Progressive effects → performance changes progressively from trial to trial
- Carryover effect → some sequences produce different effects from others
