Survey week 1
Errors in survey
Random
- Sampling error: it happens
- Questions invalidity: honest mistake
Biased (systematic)
- Sample bias: first row students?
- Questions bias: did you read the literature?
What can be wrong in a question
- Double barreled: asking two different things but only give one answer scale
- Leading question (biased)
- Poorly worded
- The wau the question is asked
- Misunderstanding
- Memory problems
- The way the info is recorded
- The way the info is processed
- Do respondents have the requisite knowledge?
- If you just want a yes/no answer, have you given more possibilities?
- Have you thought whether you should include ‘don’t know’ option?
- Are you relying too much on the respondent’s memory?
Who doesn’t know?
You can’t just assume that the people who say don’t know will be randomly distributed
over the other factors. Saying ‘I don’t know’ may be culturally sensitive. Saying ‘I don’t
know’ varies over e.g. level of education.
Midpoints, the other don’t know
You can’t just assume that the people in the middle will evenly split to two questions.
Should you use midpoint/don’t know option
- If it’s offered people will use it
- If you want people to make a choice, leave out
- If they really can: include
Answer options
- Measurements level: nominal, ordinal, interval and ratio
- Try to get interval/ratio information
- Sometimes answers are more specific than others
- If you get answers you have to take social desirability into account, you can make
undesirable answers less threatening.
Balanced questions
1
, You want balanced questions, meaning if there is a positive side also mention negative
sides.
Structure of your question
Is it A, or is it B?
- Primacy: first answer category are more likely to be picked (visual)
- Recency: last answer category are more likely to be picked (aural)
Acquiescence, you tend to agree
No answer categories
- No/less bias due to response options
- Increased validity
- Tiresome for respondent
- Explorative questions
- unanticipated responses
- Very important to clarify the question
- Length in cm
- Transformations after data collection
- Sometimes answer categories are preferred
Open ended questions
+ Respondents answer in their own terms
+ Allow for new, unexpected answers
+ Exploratory, generate fixed answer questions
- Time-consuming for both
- Difficult to code
- Interviewer variation in recoding answers
Question order
First general questions, then specific questions
Survey week 2
Reaching your audience
Survey modes
When we talk about survey modes we mean how are we contacting our respondents.
- Phone survey
- Face-to-face interviews
- Mail survey
- Internet survey
Using an interviewer
+ Help with complex questions
+ Use complex questionnaires
+ Increase response rates
+ Fewer missing answers
+ Control over conversation
- Costs: interviewers are expensive
- Privacy
2
Errors in survey
Random
- Sampling error: it happens
- Questions invalidity: honest mistake
Biased (systematic)
- Sample bias: first row students?
- Questions bias: did you read the literature?
What can be wrong in a question
- Double barreled: asking two different things but only give one answer scale
- Leading question (biased)
- Poorly worded
- The wau the question is asked
- Misunderstanding
- Memory problems
- The way the info is recorded
- The way the info is processed
- Do respondents have the requisite knowledge?
- If you just want a yes/no answer, have you given more possibilities?
- Have you thought whether you should include ‘don’t know’ option?
- Are you relying too much on the respondent’s memory?
Who doesn’t know?
You can’t just assume that the people who say don’t know will be randomly distributed
over the other factors. Saying ‘I don’t know’ may be culturally sensitive. Saying ‘I don’t
know’ varies over e.g. level of education.
Midpoints, the other don’t know
You can’t just assume that the people in the middle will evenly split to two questions.
Should you use midpoint/don’t know option
- If it’s offered people will use it
- If you want people to make a choice, leave out
- If they really can: include
Answer options
- Measurements level: nominal, ordinal, interval and ratio
- Try to get interval/ratio information
- Sometimes answers are more specific than others
- If you get answers you have to take social desirability into account, you can make
undesirable answers less threatening.
Balanced questions
1
, You want balanced questions, meaning if there is a positive side also mention negative
sides.
Structure of your question
Is it A, or is it B?
- Primacy: first answer category are more likely to be picked (visual)
- Recency: last answer category are more likely to be picked (aural)
Acquiescence, you tend to agree
No answer categories
- No/less bias due to response options
- Increased validity
- Tiresome for respondent
- Explorative questions
- unanticipated responses
- Very important to clarify the question
- Length in cm
- Transformations after data collection
- Sometimes answer categories are preferred
Open ended questions
+ Respondents answer in their own terms
+ Allow for new, unexpected answers
+ Exploratory, generate fixed answer questions
- Time-consuming for both
- Difficult to code
- Interviewer variation in recoding answers
Question order
First general questions, then specific questions
Survey week 2
Reaching your audience
Survey modes
When we talk about survey modes we mean how are we contacting our respondents.
- Phone survey
- Face-to-face interviews
- Mail survey
- Internet survey
Using an interviewer
+ Help with complex questions
+ Use complex questionnaires
+ Increase response rates
+ Fewer missing answers
+ Control over conversation
- Costs: interviewers are expensive
- Privacy
2
