APPLIED RESEARCH METHODS HOORCOLLEGES
HOORCOLLEGE 1
TYPES OF SCIENTIFIC RESEARCH
Observations: Finding phenomena
Correlations and quasi-experiments: Finding relationships
Experiments: Finding causal explanations
All of them: Developing and testing theories of experience and
behaviour
How do you tell a good theory from a bad one? Precision, parsimony,
testability and falsifiability. Parsimony means that a theory explains a
phenomena with the amount of assumptions. How fewer assumptions a
theory make, the better the theory is
TYPES OF VALIDITY
Internal validity: Did the intervention rather than a confounded
variable cause the results?
External validity: How far can the results be generalized?
Construct validity: Which aspect of the intervention caused the
results?
Statistical validity: Are the statistical conclusions correct?
CORRELATIONAL RESEARCH
Correlational research questions:
How closely are two variables related?
o Correlation: direction and size
How can I predict one variable if I know the other?
o Regression: prediction. This is based on real predictions in
the future.
Correlation and regression cannot tell you how a effect is caused, by
which mechanisms the effect is caused. This is called the causality
problem.
If there is causality, then there is also correlation but it is not the other
way around: when there is a correlation, this does not mean there is
always causality.
Temporal orders not prove causality either; if A is the cause of B, A must
happen before B. But this does not mean that if A happens before B, A is
the cause of B. Even if two variables are both correlated and temporally
,ordered, the earlier does not have to be the cause of the later one. It may
be, but it does not have to be. Correlation is a necessary, but not a
sufficient precondition for causation.
VARIABLES IN EXPERIMENTS
Independent variables: Manipulated by experimenter.
Dependent variables: Measured by experimenter. Beware of floor
effects and ceiling effects. Avoid situations where every participant
scores low (floor effect) or situations where every participant scores
high (ceiling effect).
Control variables: Controlled by experimenter. Hold the control
variables constant, you decrease the variety in your participants.
You can also turn them into independent variables.
BETWEEN-SUBJECTS VERSUS WITHIN-SUBJECTS DESIGN
Between-subjects designs: Independent groups. Every subject
experiences only one level of the independent variable: Random
assignment.
Within-subjects designs: Repeated measures. Every subject
experiences every level of the independent variable.
PROBLEMS OF EXPERIMENTAL DESIGNS
These problems are particularly critical in clinical psychology:
Quasi-experiments instead of random assignment. This is especially
seen in clinical psychology: the patients are compared to healthy
controls. There is no random assignment of the participants.
However, a quasi-experiment is not much better than finding a
correlation.
External validity:
o Laboratory versus everyday life
o Patients versus analogue populations
Low sample size, leading to a low statistical power.
TWO TYPES OF ERRORS
Problems in generalizing from the small experimental sample to the
population.
,Psychologists usually learn to avoid the alpha-error (false positive error),
but does not learn much about the beta-error (false negative error). The
beta-error is the same as power. The power is usually 80%.
EFFECT SIZE AND STATISTICAL POWER
Effect size: How large is a difference / correlation / relationship?
Statistical power: What is the probability that this effect will be
statistically significant in an experiment?
Situations in which you use these two:
Experiment in preparation: Determine necessary sample size
Experiment completed: Determine power of the experiment
Evaluation of published studies: Are the effects for real?
COHEN’S D AS AN EXAMPLE
One example of an effect size is Cohen’s d. d is always used in a t-test. d
is the difference between the two means standardised by the standard
deviation. d is always positive, the minimum is 0. A d of 0.2 is a small
effect, 0.5 is a medium effect, and 0.8 is a large effect. But d can go to
3.0, this is not likely but it is possible.
, WHAT AFFECTS POWER?
Effect size: Larger effects are easier to find.
Sample size: Effects are easier to find with many participants.
Sample size is changeable.
Alpha error: Increasing the alpha error reduces the beta error. This
is not something you can really manipulate yourself.
EXAMPLE: DETERMINING SAMPLE SIZE
HOORCOLLEGE 1
TYPES OF SCIENTIFIC RESEARCH
Observations: Finding phenomena
Correlations and quasi-experiments: Finding relationships
Experiments: Finding causal explanations
All of them: Developing and testing theories of experience and
behaviour
How do you tell a good theory from a bad one? Precision, parsimony,
testability and falsifiability. Parsimony means that a theory explains a
phenomena with the amount of assumptions. How fewer assumptions a
theory make, the better the theory is
TYPES OF VALIDITY
Internal validity: Did the intervention rather than a confounded
variable cause the results?
External validity: How far can the results be generalized?
Construct validity: Which aspect of the intervention caused the
results?
Statistical validity: Are the statistical conclusions correct?
CORRELATIONAL RESEARCH
Correlational research questions:
How closely are two variables related?
o Correlation: direction and size
How can I predict one variable if I know the other?
o Regression: prediction. This is based on real predictions in
the future.
Correlation and regression cannot tell you how a effect is caused, by
which mechanisms the effect is caused. This is called the causality
problem.
If there is causality, then there is also correlation but it is not the other
way around: when there is a correlation, this does not mean there is
always causality.
Temporal orders not prove causality either; if A is the cause of B, A must
happen before B. But this does not mean that if A happens before B, A is
the cause of B. Even if two variables are both correlated and temporally
,ordered, the earlier does not have to be the cause of the later one. It may
be, but it does not have to be. Correlation is a necessary, but not a
sufficient precondition for causation.
VARIABLES IN EXPERIMENTS
Independent variables: Manipulated by experimenter.
Dependent variables: Measured by experimenter. Beware of floor
effects and ceiling effects. Avoid situations where every participant
scores low (floor effect) or situations where every participant scores
high (ceiling effect).
Control variables: Controlled by experimenter. Hold the control
variables constant, you decrease the variety in your participants.
You can also turn them into independent variables.
BETWEEN-SUBJECTS VERSUS WITHIN-SUBJECTS DESIGN
Between-subjects designs: Independent groups. Every subject
experiences only one level of the independent variable: Random
assignment.
Within-subjects designs: Repeated measures. Every subject
experiences every level of the independent variable.
PROBLEMS OF EXPERIMENTAL DESIGNS
These problems are particularly critical in clinical psychology:
Quasi-experiments instead of random assignment. This is especially
seen in clinical psychology: the patients are compared to healthy
controls. There is no random assignment of the participants.
However, a quasi-experiment is not much better than finding a
correlation.
External validity:
o Laboratory versus everyday life
o Patients versus analogue populations
Low sample size, leading to a low statistical power.
TWO TYPES OF ERRORS
Problems in generalizing from the small experimental sample to the
population.
,Psychologists usually learn to avoid the alpha-error (false positive error),
but does not learn much about the beta-error (false negative error). The
beta-error is the same as power. The power is usually 80%.
EFFECT SIZE AND STATISTICAL POWER
Effect size: How large is a difference / correlation / relationship?
Statistical power: What is the probability that this effect will be
statistically significant in an experiment?
Situations in which you use these two:
Experiment in preparation: Determine necessary sample size
Experiment completed: Determine power of the experiment
Evaluation of published studies: Are the effects for real?
COHEN’S D AS AN EXAMPLE
One example of an effect size is Cohen’s d. d is always used in a t-test. d
is the difference between the two means standardised by the standard
deviation. d is always positive, the minimum is 0. A d of 0.2 is a small
effect, 0.5 is a medium effect, and 0.8 is a large effect. But d can go to
3.0, this is not likely but it is possible.
, WHAT AFFECTS POWER?
Effect size: Larger effects are easier to find.
Sample size: Effects are easier to find with many participants.
Sample size is changeable.
Alpha error: Increasing the alpha error reduces the beta error. This
is not something you can really manipulate yourself.
EXAMPLE: DETERMINING SAMPLE SIZE
