RM Exam Prep
30 Multiple Choice Questions (70% of exam grade)
- Four response options, one correct answer
- Grading with guess correction
2 (out of 3) Open Questions (30% of exam grade)
- Usually discussion of advantages & disadvantages of different designs and techniques (esp. in terms of
internal & external validity, reliability, ethics)
- Make sure to (1) define terms/concepts & (2) discuss.
Research Design
■ Internal Validity: the extent to which we are confident that a causal relationship really exists
between two variables.
■ External Validity: the extent to which findings of your research can be generalised to other
similar situations.
Common threats to (mostly) internal validity
★ Internal validity:
1. History (context effect)
2. Maturation (subject effect)
3. Testing- performance (sensitization)
4. instability/ instrumentation
Common threats to (mostly) external validity
★ External validity:
1. Selection bias
2. Statistical regression
a. Regression towards the mean
b. Ceiling-floor effects
3. Morality (attrition)
1
, Measurement
★ Measurement Reliability: tests how consistently a method measures something. Generally,
measurement reliability is categorised into three groups: stability over time, internal consistency,
and intercoder reliability.
a. Test-retest: stability over time
b. Internal consistency: consistency between the different indicators
c. Intercoder reliability: different researchers finding similar outcomes
★ Measurement Validity: indicates whether the research accurately measures what you want to
measure. Therefore, measurement validity checks if the research is free of systematic errors, using
different criteria such as face, content, and construct validity.
a. Face validity: does the indicator intuitively seem like a good concept?
b. Content validity: does the indicator cover the full range of aspects?
c. Construct/ criterion validity: does the measure conform to the theoretical expectation? Is
it associated with theoretically relevant factors?
● Concurrent: is there another criteria that’s correlated?
● Predictive: is there another criteria that’s correlated (in the future)?
● Convergent: is there a correlation with existing measures?
● Discriminant: is there an overlap with other concepts? Does the concept allow
differentiation?
Research Ethics (3 Basic Principles)
I. Do no harm
II. Voluntary participation & informed consent
● Topic and nature of questions, purpose, use of information, freedom to stop, permission to use
data, risks involved
● Can marginalised and vulnerable populations give consent?
III. Protection of privacy and confidentiality
● Public information, confidential, anonymous data- don’t make promises you can’t keep.
Conceptual Goodness (Gerring)
1. Familiarity
2. Resonance
3. Field utility
4. Theoretical utility
5. Parsimony
6. Depth
7. Differentiation
8. Consistency
2
30 Multiple Choice Questions (70% of exam grade)
- Four response options, one correct answer
- Grading with guess correction
2 (out of 3) Open Questions (30% of exam grade)
- Usually discussion of advantages & disadvantages of different designs and techniques (esp. in terms of
internal & external validity, reliability, ethics)
- Make sure to (1) define terms/concepts & (2) discuss.
Research Design
■ Internal Validity: the extent to which we are confident that a causal relationship really exists
between two variables.
■ External Validity: the extent to which findings of your research can be generalised to other
similar situations.
Common threats to (mostly) internal validity
★ Internal validity:
1. History (context effect)
2. Maturation (subject effect)
3. Testing- performance (sensitization)
4. instability/ instrumentation
Common threats to (mostly) external validity
★ External validity:
1. Selection bias
2. Statistical regression
a. Regression towards the mean
b. Ceiling-floor effects
3. Morality (attrition)
1
, Measurement
★ Measurement Reliability: tests how consistently a method measures something. Generally,
measurement reliability is categorised into three groups: stability over time, internal consistency,
and intercoder reliability.
a. Test-retest: stability over time
b. Internal consistency: consistency between the different indicators
c. Intercoder reliability: different researchers finding similar outcomes
★ Measurement Validity: indicates whether the research accurately measures what you want to
measure. Therefore, measurement validity checks if the research is free of systematic errors, using
different criteria such as face, content, and construct validity.
a. Face validity: does the indicator intuitively seem like a good concept?
b. Content validity: does the indicator cover the full range of aspects?
c. Construct/ criterion validity: does the measure conform to the theoretical expectation? Is
it associated with theoretically relevant factors?
● Concurrent: is there another criteria that’s correlated?
● Predictive: is there another criteria that’s correlated (in the future)?
● Convergent: is there a correlation with existing measures?
● Discriminant: is there an overlap with other concepts? Does the concept allow
differentiation?
Research Ethics (3 Basic Principles)
I. Do no harm
II. Voluntary participation & informed consent
● Topic and nature of questions, purpose, use of information, freedom to stop, permission to use
data, risks involved
● Can marginalised and vulnerable populations give consent?
III. Protection of privacy and confidentiality
● Public information, confidential, anonymous data- don’t make promises you can’t keep.
Conceptual Goodness (Gerring)
1. Familiarity
2. Resonance
3. Field utility
4. Theoretical utility
5. Parsimony
6. Depth
7. Differentiation
8. Consistency
2
