Study questions
Applied Research Methods - D&H
Lecture 1
• Which types of research can be conducted with observations, correlations, and
experiments, respectively?
Observations: finding phenomena
Correlations: relationships
Experiments: causal explanations
• What is meant by the precision of a theory?
The theory that predicts more precise than the other is better.
When a theory is precise it is easier to be wrong, but it tells us more if it is right.
• What is meant by the parsimony of a theory?
A theory that makes just a few assumptions to predict something is better than a very
complicated theory that makes a lot of assumptions to predict a phenomena. Counts if both
theories have the same precision.
A theory that makes just a few assumptions to predict something is better than a very
complicated theory that makes a lot of assumptions to predict a phenomena. Counts if both
theories have the same precision.
A theory that makes just a few assumptions to predict something is better than a very
complicated theory that makes a lot of assumptions to predict a phenomena. Counts if both
theories have the same precision.
Easier theories are better than complicated ones.
• Why are testability and falsifiability considered important features of a theory?
Testability: theory can only be good, when it can be tested.
Falsifiability: theory can only be good, when it can be wrong.
Theories are useless if they could explain everything. Theory must make predictions
that can be wrong.
Otherwise, we cannot distinguish science from pseudoscience, and we cannot compare
studies based on their degree of falsifiability.
• What is the internal validity of a study?
Did the intervention cause the results? (and not a confounded variable)
• What is the external validity of a study?
1
,How far can the results be generalized?
• What is the construct validity of a study?
Which aspect of the intervention caused the results?
Manipulation. Which aspects are important?
The extent to which variables measure what they are supposed to measure.
• What is the statistical validity of a study?
Are the statistical conclusions correct?
• How can correlations be used and interpreted?
Correlation: direction and size (how closely are 2 variables related)
Regression: prediction (how can I predict one variable if I know the other.
• How can correlations not be interpreted?
Not as an explanation of ‘why’ or causality.
The direction of the relation is unclear. There can be some third variables.
Beware of causal interpretations: correlation is necessary, but not sufficient prediction
for causation.
When one event is before another, you still not can say that this is causation. You need
an experiment for this.
• Does correlation imply causality? If yes, why? If not, why not?
No, correlation between 2 variables simply indicates that a relationship exist. Whereas
causation is more specific and says that one event actually causes the other. Correlation does
not necessary mean that the one effect the other, there is only a connection.
If causality, then correlation
But not: if correlation, then causality
• How does the temporal order of two variables help to establish a causal relation
between them?
Even if 2 variables are both correlated and temporally ordered, the earlier one does not have
to be the cause of the later one.
There is an experiment needed.
2
, In an experiment: the independent variable must occur first in time to influence the dependent
variable.
• What do you have to do to test whether two variables are causally related?
Experiment -> (randomization, manipulation of the independent variable)
Temporal order, alternative explanations.
• What are independent, dependent, and control variables of experiments?
- Independent: this is manipulated.
- Dependent: this is measured.
- Control variables: when not controlled, this are confounding variables.
• What does it mean if an experimental independent variable is a between-subjects
variable?
- Different groups, get different level of variable.
- Every subject experiences only one level of the independent variable.
- Randomization to groups.
• What does it mean if an experimental independent variable is a within-subjects
variable?
Every subject experiences every level of the independent variable.
• What are advantages and disadvantages of between-subjects and within-subjects
experimental designs?
Within:
+ requires fewer participants.
+ Easier to find effects.
- But there can be an order-effect.
Between:
+ minimize the learning effect, lead to shorter sessions.
+ Also no confounding variables.
- But you need a big sample to have good randomization.
• What is random assignment, and why is it so very important?
- 1 control group and 1 or more experimental groups.
- Ensure that the groups are comparable.
- Avoiding confounding variables.
3
Applied Research Methods - D&H
Lecture 1
• Which types of research can be conducted with observations, correlations, and
experiments, respectively?
Observations: finding phenomena
Correlations: relationships
Experiments: causal explanations
• What is meant by the precision of a theory?
The theory that predicts more precise than the other is better.
When a theory is precise it is easier to be wrong, but it tells us more if it is right.
• What is meant by the parsimony of a theory?
A theory that makes just a few assumptions to predict something is better than a very
complicated theory that makes a lot of assumptions to predict a phenomena. Counts if both
theories have the same precision.
A theory that makes just a few assumptions to predict something is better than a very
complicated theory that makes a lot of assumptions to predict a phenomena. Counts if both
theories have the same precision.
A theory that makes just a few assumptions to predict something is better than a very
complicated theory that makes a lot of assumptions to predict a phenomena. Counts if both
theories have the same precision.
Easier theories are better than complicated ones.
• Why are testability and falsifiability considered important features of a theory?
Testability: theory can only be good, when it can be tested.
Falsifiability: theory can only be good, when it can be wrong.
Theories are useless if they could explain everything. Theory must make predictions
that can be wrong.
Otherwise, we cannot distinguish science from pseudoscience, and we cannot compare
studies based on their degree of falsifiability.
• What is the internal validity of a study?
Did the intervention cause the results? (and not a confounded variable)
• What is the external validity of a study?
1
,How far can the results be generalized?
• What is the construct validity of a study?
Which aspect of the intervention caused the results?
Manipulation. Which aspects are important?
The extent to which variables measure what they are supposed to measure.
• What is the statistical validity of a study?
Are the statistical conclusions correct?
• How can correlations be used and interpreted?
Correlation: direction and size (how closely are 2 variables related)
Regression: prediction (how can I predict one variable if I know the other.
• How can correlations not be interpreted?
Not as an explanation of ‘why’ or causality.
The direction of the relation is unclear. There can be some third variables.
Beware of causal interpretations: correlation is necessary, but not sufficient prediction
for causation.
When one event is before another, you still not can say that this is causation. You need
an experiment for this.
• Does correlation imply causality? If yes, why? If not, why not?
No, correlation between 2 variables simply indicates that a relationship exist. Whereas
causation is more specific and says that one event actually causes the other. Correlation does
not necessary mean that the one effect the other, there is only a connection.
If causality, then correlation
But not: if correlation, then causality
• How does the temporal order of two variables help to establish a causal relation
between them?
Even if 2 variables are both correlated and temporally ordered, the earlier one does not have
to be the cause of the later one.
There is an experiment needed.
2
, In an experiment: the independent variable must occur first in time to influence the dependent
variable.
• What do you have to do to test whether two variables are causally related?
Experiment -> (randomization, manipulation of the independent variable)
Temporal order, alternative explanations.
• What are independent, dependent, and control variables of experiments?
- Independent: this is manipulated.
- Dependent: this is measured.
- Control variables: when not controlled, this are confounding variables.
• What does it mean if an experimental independent variable is a between-subjects
variable?
- Different groups, get different level of variable.
- Every subject experiences only one level of the independent variable.
- Randomization to groups.
• What does it mean if an experimental independent variable is a within-subjects
variable?
Every subject experiences every level of the independent variable.
• What are advantages and disadvantages of between-subjects and within-subjects
experimental designs?
Within:
+ requires fewer participants.
+ Easier to find effects.
- But there can be an order-effect.
Between:
+ minimize the learning effect, lead to shorter sessions.
+ Also no confounding variables.
- But you need a big sample to have good randomization.
• What is random assignment, and why is it so very important?
- 1 control group and 1 or more experimental groups.
- Ensure that the groups are comparable.
- Avoiding confounding variables.
3
