Study Questions – Applied Research
Methods
Week 1
General Topics
- Which types of research can be conducted with observations,
correlations, and experiments, respectively?
Observation: finding phenomena and seeing if they are worth researching
Correlations: finding relationships between variables
Experiments: finding causal explanations
- What is meant by the precision of a theory?
The more precise a theory predicts a phenome correctly, the better. High
precision theories clearly define which observations/ measurements are
predicted by the theory and which observations and measurements falsify
it.
- What is meant by the parsimony of a theory?
Trying to explain the theory with as few assumptions as possible, the
fewer you need the better
- Why are testability and falsifiability considered important features
of a theory?
When you are able to test a theory, you are able to falsify it, a theory
needs to be able to be wrong. This way you can distinguish science from
pseudoscience.
- What is the internal validity of a study?
Did the intervention, rather than a confounded variable, cause the
results?
- What is the external validity of a study?
How far can results be generalized?
- What is the construct validity of a study?
Which aspects of the intervention caused the results?
- What is the statistical validity of a study?
Are the statistical conclusions that were made, correct?
- How can correlations be used and interpreted?
You can look at their direction and size (strength) and interpret how
closely two variables are related
- How can correlations not be interpreted?
As an indication of causality
- Does correlation imply causality? If yes, why? If not, why not?
1
,No, even if two variables are correlated and temporally ordered, the
earlier one does not have to be the cause of the later one
- Does causality imply correlation? If yes, why? If not, why not?
Yes, correlation is a necessary precondition for causation
- How does the temporal order of two variables help to establish a
causal relation between them?
It can disprove causality, what happened later cannot be the cause of
what happened later
- What do you have to do to test whether two variables are causally
related?
Conduct an experiment
- What are independent, dependent, and control variables of
experiments?
Independent: manipulated by the researcher
Dependent: measured by the researcher
Control: controlled by the researcher, you hold them constant, so you
don’t mix them up with your other variables
- What does it mean if an experimental independent variable is a
between-subjects variable?
Every subject experiences only one level of the independent variable
- What does it mean if an experimental independent variable is a
within-subjects variable?
Every subject experiences every level of the independent variable
- What are advantages and disadvantages of between-subjects and
within-subjects experimental designs?
Advantage of between subject design: less time consuming
Disadvantage of between subject design: you need large samples
Advantage of within subject design: difference between groups is easier
to find
Disadvantage of within subject design: order effects
- What is random assignment, and why is it so very important?
People randomly get condition A or B, makes sure the differences
between groups are equally distributed
- What is the difference between a quasi-experiment and a real
experiment?
In quasi experiments there isn’t random assignment
Statistical Power
- In statistical testing, what is the alpha error?
The false positives, type I error, DETECTING effect that IS NOT there ->
alpha is often .05
2
, - In statistical testing, what is the beta error?
False negatives, type II error, NOT DETECTING effect that IS there
- What does the term "effect size" mean?
How large a difference/correlation/relationship actually is
- What does the term "statistical power" mean?
The probability that an effect will be statistically significant in an
experiment
- What does it mean when the statistical power of a study is small or
large, respectively?
Low power = test only has a small chance of detecting a true effect or
that the results are likely to be distorted by random and systematic error
High power = indicates a large chance of a test detecting a true effect
- In which situations is it important to consider the power of studies?
In preparing an experiment, that way you can determine the necessary
sample size, in a completed experiment, to determine the power of this
experiment, in evaluation of published studies, are the found effects for
real?
- How is Cohen's effect size value d computed?
By using the means and standard deviation
- What are the conventional values of d for small, medium, and large
effects, respectively?
0.2 is small, 0.5 is medium, and 0.8 is large
- Which factors affect the statistical power of a study?
Effects size, large effects are easier to find, sample size, effects are easier
to find with many participants, alpha error, increasing the false positives
reduces the false negatives
- How can the power of a study be increased?
By increasing the effect size
- Which effect size values are usually used together with t-tests,
ANOVAs, and correlations?
t-test: d, ANOVA: f or partial eta2, correlation: r
- Why is the correlation a particularly simple and useful effect size
value?
Everyone knows it
- What are the two main disadvantages of small-sample studies?
Random fluctuation of effects in sample, smaller samples means more
fluctuation and publication bias favours significant effects
3
Methods
Week 1
General Topics
- Which types of research can be conducted with observations,
correlations, and experiments, respectively?
Observation: finding phenomena and seeing if they are worth researching
Correlations: finding relationships between variables
Experiments: finding causal explanations
- What is meant by the precision of a theory?
The more precise a theory predicts a phenome correctly, the better. High
precision theories clearly define which observations/ measurements are
predicted by the theory and which observations and measurements falsify
it.
- What is meant by the parsimony of a theory?
Trying to explain the theory with as few assumptions as possible, the
fewer you need the better
- Why are testability and falsifiability considered important features
of a theory?
When you are able to test a theory, you are able to falsify it, a theory
needs to be able to be wrong. This way you can distinguish science from
pseudoscience.
- What is the internal validity of a study?
Did the intervention, rather than a confounded variable, cause the
results?
- What is the external validity of a study?
How far can results be generalized?
- What is the construct validity of a study?
Which aspects of the intervention caused the results?
- What is the statistical validity of a study?
Are the statistical conclusions that were made, correct?
- How can correlations be used and interpreted?
You can look at their direction and size (strength) and interpret how
closely two variables are related
- How can correlations not be interpreted?
As an indication of causality
- Does correlation imply causality? If yes, why? If not, why not?
1
,No, even if two variables are correlated and temporally ordered, the
earlier one does not have to be the cause of the later one
- Does causality imply correlation? If yes, why? If not, why not?
Yes, correlation is a necessary precondition for causation
- How does the temporal order of two variables help to establish a
causal relation between them?
It can disprove causality, what happened later cannot be the cause of
what happened later
- What do you have to do to test whether two variables are causally
related?
Conduct an experiment
- What are independent, dependent, and control variables of
experiments?
Independent: manipulated by the researcher
Dependent: measured by the researcher
Control: controlled by the researcher, you hold them constant, so you
don’t mix them up with your other variables
- What does it mean if an experimental independent variable is a
between-subjects variable?
Every subject experiences only one level of the independent variable
- What does it mean if an experimental independent variable is a
within-subjects variable?
Every subject experiences every level of the independent variable
- What are advantages and disadvantages of between-subjects and
within-subjects experimental designs?
Advantage of between subject design: less time consuming
Disadvantage of between subject design: you need large samples
Advantage of within subject design: difference between groups is easier
to find
Disadvantage of within subject design: order effects
- What is random assignment, and why is it so very important?
People randomly get condition A or B, makes sure the differences
between groups are equally distributed
- What is the difference between a quasi-experiment and a real
experiment?
In quasi experiments there isn’t random assignment
Statistical Power
- In statistical testing, what is the alpha error?
The false positives, type I error, DETECTING effect that IS NOT there ->
alpha is often .05
2
, - In statistical testing, what is the beta error?
False negatives, type II error, NOT DETECTING effect that IS there
- What does the term "effect size" mean?
How large a difference/correlation/relationship actually is
- What does the term "statistical power" mean?
The probability that an effect will be statistically significant in an
experiment
- What does it mean when the statistical power of a study is small or
large, respectively?
Low power = test only has a small chance of detecting a true effect or
that the results are likely to be distorted by random and systematic error
High power = indicates a large chance of a test detecting a true effect
- In which situations is it important to consider the power of studies?
In preparing an experiment, that way you can determine the necessary
sample size, in a completed experiment, to determine the power of this
experiment, in evaluation of published studies, are the found effects for
real?
- How is Cohen's effect size value d computed?
By using the means and standard deviation
- What are the conventional values of d for small, medium, and large
effects, respectively?
0.2 is small, 0.5 is medium, and 0.8 is large
- Which factors affect the statistical power of a study?
Effects size, large effects are easier to find, sample size, effects are easier
to find with many participants, alpha error, increasing the false positives
reduces the false negatives
- How can the power of a study be increased?
By increasing the effect size
- Which effect size values are usually used together with t-tests,
ANOVAs, and correlations?
t-test: d, ANOVA: f or partial eta2, correlation: r
- Why is the correlation a particularly simple and useful effect size
value?
Everyone knows it
- What are the two main disadvantages of small-sample studies?
Random fluctuation of effects in sample, smaller samples means more
fluctuation and publication bias favours significant effects
3
