PSYC4050: Psychological Research Methods Exam
Questions With Accurate Answers
What is selection bias? Give an example - ANSWER - selection bias occurs when the
variables of interest in the study both cause the attribute you're selecting on
- e.g., want to know the correlation between physical attractiveness and acting talent
- we find that the more attractive you are, the worse at acting you tend to be
- BUT turns out we only sampled participants from a pool of successful actors, who
themselves tend to be selected on the basis of attractiveness and acting talent
- this does not mean the trend is not real, it is just not generalisable
- it might not exist in the general population
What are some examples of spurious effects due to selection bias? - ANSWER 1. Among
US college students, academic ability is inversely related to sporting ability
- does not mean that in general, people that are smart are therefore terrible at sport
- reflects the fact that to get into college in the first place, you probably were either
really athletic or had a great GPA
- here, the trend we see is just reflective of the attributes of people within our sample
2. Among people who have had heart attacks, smokers have better subsequent health
than non-smokers
3. Among low birthweight infants, those whose mothers smoked during pregnancy are
,less likely to die than those whose mothers did not smoke
Let's say you sampled participants from the UQ first year participant pool. How might
your data be vulnerable to selection bias? - ANSWER - academic achievement (thus IQ
and conscientiousness)
- age
- WEIRD samples
What is the difference between selection bias and missing data? - ANSWER -
conceptually similar problems
- selection bias refers to WHO participates in the study, and arises from the experimental
design
- missing data refers to what data is either provided or not provided by your participants
What are the 3 sources of missing data? - ANSWER 1. Missing data from some
participants
- are people who did not provide data somehow different from those who did provide
data?
2. Missing data from some variables
- why would people not provide data here?
- is the study affected?
- e.g., sensitive topics
,3. Missing data from a subset of people/measures
- why would only some people withhold a response to some items?
When is missing data probably benign? - ANSWER - very little of it (less than 5%)
- unsystematic (no pattern)
When can missing data be problematic? - ANSWER 1. If there is a substantial amount of
missing data but it is unsystematic, this can be problematic because it can reduce your
sample size and thus reduce your power and precision
- BUT costs of this can be minimised, and it is not necessarily the end of the world
2. If there is missing data in a systematic way, it can cause problems by biasing the
estimates of parameters of interest
What are the checks we use to diagnose missing data problems? - ANSWER 1.
Inadequately sampled participants?
2. Inadequately assessed variables?
3. Missing completely at random (MCAR)?
4. Missing at random (MAR)?
5. Missing not at random (MNAR)?
- the last 3 indicate how systematic the missing data is
, What are the missing data mechanisms? - ANSWER - MNAR
- MAR
- MCAR
What does missing completely at random mean? Give an example - ANSWER - whether a
data point is missing or not does NOT depend on the true value of the missing data
point or any other variable in the dataset
- e.g., a programming error occurs such that participants randomly miss individual items
on a survey
- no particular item was always missing
- participants had missing items randomly (some did, some didn't)
- no other variable is driving the missingness of the data
- thus, this data is missing completely at random (MCAR)
- this is typically not problematic and does not bias population parameters
What does missing at random mean? Give an example - ANSWER - whether a data point
is missing or not does not depend on the true value of the missing data point, although
it may be impacted by other variables in the study
- e.g., What if older people are less likely to answer questions about their number of sex
partners, but the number of sex partners they have had does not affect whether they
answer the sex partner question?
- this data is missing at random, impacted by a variable in the study (age) but not the
true value of the missing data point itself (no of sex partners)
- usually not problematic either, no bias in parameter estimates
Questions With Accurate Answers
What is selection bias? Give an example - ANSWER - selection bias occurs when the
variables of interest in the study both cause the attribute you're selecting on
- e.g., want to know the correlation between physical attractiveness and acting talent
- we find that the more attractive you are, the worse at acting you tend to be
- BUT turns out we only sampled participants from a pool of successful actors, who
themselves tend to be selected on the basis of attractiveness and acting talent
- this does not mean the trend is not real, it is just not generalisable
- it might not exist in the general population
What are some examples of spurious effects due to selection bias? - ANSWER 1. Among
US college students, academic ability is inversely related to sporting ability
- does not mean that in general, people that are smart are therefore terrible at sport
- reflects the fact that to get into college in the first place, you probably were either
really athletic or had a great GPA
- here, the trend we see is just reflective of the attributes of people within our sample
2. Among people who have had heart attacks, smokers have better subsequent health
than non-smokers
3. Among low birthweight infants, those whose mothers smoked during pregnancy are
,less likely to die than those whose mothers did not smoke
Let's say you sampled participants from the UQ first year participant pool. How might
your data be vulnerable to selection bias? - ANSWER - academic achievement (thus IQ
and conscientiousness)
- age
- WEIRD samples
What is the difference between selection bias and missing data? - ANSWER -
conceptually similar problems
- selection bias refers to WHO participates in the study, and arises from the experimental
design
- missing data refers to what data is either provided or not provided by your participants
What are the 3 sources of missing data? - ANSWER 1. Missing data from some
participants
- are people who did not provide data somehow different from those who did provide
data?
2. Missing data from some variables
- why would people not provide data here?
- is the study affected?
- e.g., sensitive topics
,3. Missing data from a subset of people/measures
- why would only some people withhold a response to some items?
When is missing data probably benign? - ANSWER - very little of it (less than 5%)
- unsystematic (no pattern)
When can missing data be problematic? - ANSWER 1. If there is a substantial amount of
missing data but it is unsystematic, this can be problematic because it can reduce your
sample size and thus reduce your power and precision
- BUT costs of this can be minimised, and it is not necessarily the end of the world
2. If there is missing data in a systematic way, it can cause problems by biasing the
estimates of parameters of interest
What are the checks we use to diagnose missing data problems? - ANSWER 1.
Inadequately sampled participants?
2. Inadequately assessed variables?
3. Missing completely at random (MCAR)?
4. Missing at random (MAR)?
5. Missing not at random (MNAR)?
- the last 3 indicate how systematic the missing data is
, What are the missing data mechanisms? - ANSWER - MNAR
- MAR
- MCAR
What does missing completely at random mean? Give an example - ANSWER - whether a
data point is missing or not does NOT depend on the true value of the missing data
point or any other variable in the dataset
- e.g., a programming error occurs such that participants randomly miss individual items
on a survey
- no particular item was always missing
- participants had missing items randomly (some did, some didn't)
- no other variable is driving the missingness of the data
- thus, this data is missing completely at random (MCAR)
- this is typically not problematic and does not bias population parameters
What does missing at random mean? Give an example - ANSWER - whether a data point
is missing or not does not depend on the true value of the missing data point, although
it may be impacted by other variables in the study
- e.g., What if older people are less likely to answer questions about their number of sex
partners, but the number of sex partners they have had does not affect whether they
answer the sex partner question?
- this data is missing at random, impacted by a variable in the study (age) but not the
true value of the missing data point itself (no of sex partners)
- usually not problematic either, no bias in parameter estimates
