General Topics
• Which types of research can be conducted with observations, correlations, and experiments, respectively?
- experimental; randomised and non-randomised controlled trial, observational; analytical (cohort, case-
control and cross-sectional study) and descriptive study
- scientific research
• What is meant by the precision of a theory?
- theoretical concepts, which are often hard to measure, must be defined with such precision that others can
use those definitions to measure those concepts and test that theory
- how exact the predictions are
• What is meant by the parsimony of a theory?
- when there are multiple explanations of a phenomenon, scientists must always accept the simplest or
logically most economical explanation
- a theory based on only a few assumptions is better than a complicated theory that makes predictions on
many assumptions > how many assumptions the theory is based on
• Why are testability and falsifiability considered important features of a theory?
- a theory must be stated in a way that it can be disproven. Theories that cannot be tested or falsified are not
scientific theories and any such knowledge is not scientific knowledge
• What is the internal validity of a study?
- did the intervention rather than a confounded variable cause the results?
• What is the external validity of a study?
- how far can the results be generalized?
• What is the construct validity of a study?
- which aspect of the intervention caused the results?
• What is the statistical validity of a study?
- are the statistical conslusions correct?
• How can correlations be used and interpreted?
- direction and size (how strong they are and in which direction)
• How can correlations not be interpreted?
- as causal interpretation
• Does correlation imply causality? If yes, why? If not, why not?
- No, a correlation shows that there is a relation but you don’t know what kind of relation so you can’t say
that one causes the other
• Does causality imply correlation? If yes, why? If not, why not?
- yes, correlation is necessary for causality to exist, but is not sufficient (voldoende)
• How does the temporal order of two variables help to establish a causal relation between them?
- It doesn’t.
- if A is the cause of B, A must happen before B (right)
- if A happens before B and there’s an effect, A is the cause of B (wrong)
• What do you have to do to test whether two variables are causally related?
- Conduct an experiment
• What are independent, dependent, and control variables of experiments?
- independent; manipulated by the experimenter
- dependent; the variable you’re measuring
- control; the ones that are controlled by the experimenter, this one is held constant and are independent
variables
• What does it mean if an experimental independent variable is a between-subjects variable?
- Independent groups
- every subject experiences only one level of the independent variable (random assignment)
• What does it mean if an experimental independent variable is a within-subjects variable?
- every subject experiences every level of the independent variable (order effets)
• What are advantages and disadvantages of between-subjects and within-subjects experimental designs?
- Advantages; within: less time and money, more people at once, geen rekening houden met random
assignment
- Disadvantages; within: counterbalancing, order effects
- Advantages; between: causal relations if randomly assigned groups
- Disadvantages; between: random assignment
• What is random assignment, and why is it so very important?
, - Randomly assign people to a group to get equal groups and make sure there are less confounding variables
• What is the difference between a quasi-experiment and a real experiment?
- A real experiment is with random assignment and a quasi-experiment is a study with an independent
variable that is manipulated but with no randomization to a group
Statistical Power
• In statistical testing, what is the alpha error?
- type I error; reject a true null hypothesis (false positive). Dat je zegt dat iets significant is terwijl het niet in de
populatie bestaat (aannemen Ha)
- p-value is small
• In statistical testing, what is the beta error?
- type II error; failure to reject a false null hypothesis (false negative). Dat je zegt dat iets niet significant is in
het sample, maar het wel in de populatie bestaat (aannemen H0)
- p-value is large
• What does the term "effect size" mean?
- how large is a difference/correlation/relationship? How strong the effect is
- calculated using a specific statistical measure, such as Pearson’s correlation coefficient for the relationship
between variables or Cohen’s d for the difference between groups.
• What does the term "statistical power" mean?
- The probability of accepting the alternative hypothesis if it is true. Opposite of the beta error (type II error).
So an effect that is in the population and is significant
- The higher the statistical power for a given experiment, the lower the probability of making a Type II (false
negative) error. That is the higher the probability of detecting an effect when there is an effect. In fact, the
power is precisely the inverse of the probability of a Type II error > power = 1 – type II error.
• What does it mean when the statistical power of a study is small or large, respectively?
- Low Statistical Power: Large risk of committing Type II errors, e.g. a false negative.
- High Statistical Power: Small risk of committing Type II errors.
• In which situations is it important to consider the power of studies?
- to determine the necessary number of subjets needed for your study
• How is Cohen's effect size value d computed?
- for the independent samples T-test, Cohen's d is determined by calculating the mean difference between
your two groups, and then dividing the result by the pooled standard deviation
- (mean1 – mean2) / SD
• What are the conventional values of d for small, medium, and large effects, respectively?
- 0.2, 0.5, 0.8
• Which factors affect the statistical power of a study?
- effect size
- sample size
- alpha error (significance level = 0.05)
• How can the power of a study be increased?
- Increase effect size or sample size
• Which effect size values are usually used together with t-tests, ANOVAs, and correlations?
- T-test: d
- Anova: f (f=d/2); partial eta^2 (percentage of explained variance)
- Correlation: r
• Why is the correlation a particularly simple and useful effect size value?
- because we know that an r close to 1 or -1 indicates a strong relation and so indicates a strong effect and
value close to 0 indicates a small effect. So it already gives you the effect size, you don’t have to compute it.
If every value was computed into r then everybody could understand it easily
• What are the two main disadvantages of small-sample studies?
- Random fluctuation of effects in the samples; the smaller the sample the more the found results in the
sample will be very different from reality (less generalizability)
• Which types of research can be conducted with observations, correlations, and experiments, respectively?
- experimental; randomised and non-randomised controlled trial, observational; analytical (cohort, case-
control and cross-sectional study) and descriptive study
- scientific research
• What is meant by the precision of a theory?
- theoretical concepts, which are often hard to measure, must be defined with such precision that others can
use those definitions to measure those concepts and test that theory
- how exact the predictions are
• What is meant by the parsimony of a theory?
- when there are multiple explanations of a phenomenon, scientists must always accept the simplest or
logically most economical explanation
- a theory based on only a few assumptions is better than a complicated theory that makes predictions on
many assumptions > how many assumptions the theory is based on
• Why are testability and falsifiability considered important features of a theory?
- a theory must be stated in a way that it can be disproven. Theories that cannot be tested or falsified are not
scientific theories and any such knowledge is not scientific knowledge
• What is the internal validity of a study?
- did the intervention rather than a confounded variable cause the results?
• What is the external validity of a study?
- how far can the results be generalized?
• What is the construct validity of a study?
- which aspect of the intervention caused the results?
• What is the statistical validity of a study?
- are the statistical conslusions correct?
• How can correlations be used and interpreted?
- direction and size (how strong they are and in which direction)
• How can correlations not be interpreted?
- as causal interpretation
• Does correlation imply causality? If yes, why? If not, why not?
- No, a correlation shows that there is a relation but you don’t know what kind of relation so you can’t say
that one causes the other
• Does causality imply correlation? If yes, why? If not, why not?
- yes, correlation is necessary for causality to exist, but is not sufficient (voldoende)
• How does the temporal order of two variables help to establish a causal relation between them?
- It doesn’t.
- if A is the cause of B, A must happen before B (right)
- if A happens before B and there’s an effect, A is the cause of B (wrong)
• What do you have to do to test whether two variables are causally related?
- Conduct an experiment
• What are independent, dependent, and control variables of experiments?
- independent; manipulated by the experimenter
- dependent; the variable you’re measuring
- control; the ones that are controlled by the experimenter, this one is held constant and are independent
variables
• What does it mean if an experimental independent variable is a between-subjects variable?
- Independent groups
- every subject experiences only one level of the independent variable (random assignment)
• What does it mean if an experimental independent variable is a within-subjects variable?
- every subject experiences every level of the independent variable (order effets)
• What are advantages and disadvantages of between-subjects and within-subjects experimental designs?
- Advantages; within: less time and money, more people at once, geen rekening houden met random
assignment
- Disadvantages; within: counterbalancing, order effects
- Advantages; between: causal relations if randomly assigned groups
- Disadvantages; between: random assignment
• What is random assignment, and why is it so very important?
, - Randomly assign people to a group to get equal groups and make sure there are less confounding variables
• What is the difference between a quasi-experiment and a real experiment?
- A real experiment is with random assignment and a quasi-experiment is a study with an independent
variable that is manipulated but with no randomization to a group
Statistical Power
• In statistical testing, what is the alpha error?
- type I error; reject a true null hypothesis (false positive). Dat je zegt dat iets significant is terwijl het niet in de
populatie bestaat (aannemen Ha)
- p-value is small
• In statistical testing, what is the beta error?
- type II error; failure to reject a false null hypothesis (false negative). Dat je zegt dat iets niet significant is in
het sample, maar het wel in de populatie bestaat (aannemen H0)
- p-value is large
• What does the term "effect size" mean?
- how large is a difference/correlation/relationship? How strong the effect is
- calculated using a specific statistical measure, such as Pearson’s correlation coefficient for the relationship
between variables or Cohen’s d for the difference between groups.
• What does the term "statistical power" mean?
- The probability of accepting the alternative hypothesis if it is true. Opposite of the beta error (type II error).
So an effect that is in the population and is significant
- The higher the statistical power for a given experiment, the lower the probability of making a Type II (false
negative) error. That is the higher the probability of detecting an effect when there is an effect. In fact, the
power is precisely the inverse of the probability of a Type II error > power = 1 – type II error.
• What does it mean when the statistical power of a study is small or large, respectively?
- Low Statistical Power: Large risk of committing Type II errors, e.g. a false negative.
- High Statistical Power: Small risk of committing Type II errors.
• In which situations is it important to consider the power of studies?
- to determine the necessary number of subjets needed for your study
• How is Cohen's effect size value d computed?
- for the independent samples T-test, Cohen's d is determined by calculating the mean difference between
your two groups, and then dividing the result by the pooled standard deviation
- (mean1 – mean2) / SD
• What are the conventional values of d for small, medium, and large effects, respectively?
- 0.2, 0.5, 0.8
• Which factors affect the statistical power of a study?
- effect size
- sample size
- alpha error (significance level = 0.05)
• How can the power of a study be increased?
- Increase effect size or sample size
• Which effect size values are usually used together with t-tests, ANOVAs, and correlations?
- T-test: d
- Anova: f (f=d/2); partial eta^2 (percentage of explained variance)
- Correlation: r
• Why is the correlation a particularly simple and useful effect size value?
- because we know that an r close to 1 or -1 indicates a strong relation and so indicates a strong effect and
value close to 0 indicates a small effect. So it already gives you the effect size, you don’t have to compute it.
If every value was computed into r then everybody could understand it easily
• What are the two main disadvantages of small-sample studies?
- Random fluctuation of effects in the samples; the smaller the sample the more the found results in the
sample will be very different from reality (less generalizability)
