survey (OP 1) hoorcolleges
week 1 hoorcollege 1
★ what is survey research about?
○ produce statistics / draw inferences about a target population.
★ advantages:
○ relatively low costs
○ relatively fast delivery
○ rapid data processing
○ can research large populations
○ various modes: phone, (e)mail, internet, face-to-f@ace
★ disadvantages:
○ generally limited to scaled or check-list questions
○ no control over response rate
○ provides information more than understanding
○ increasing public resistance
○ difficult to explore in depth
★ two inferences:
1. how well answers measure characteristics to be described.
2. how closely sample responding mirrors population.
Characteristics of
population
Sample of
population members
who answer
questions
Answers
respondents give
★ common sources of error in surveys:
○ a poorly worded question
○ deviations from the script by the interviewer
○ misunderstanding on the part of the interviewee
○ memory problems on the part of the interviewee
○ the way the information is recorded by the interviewer
, ○ the way the information is processed, either when answers are coded
or when data are entered into the computer
★ two types of error:
○ random error
■ sample: sampling error
● it happens
■ questions: invalidity
● honest mistake
○ systematic error or bias
■ sample bias
● only front row students?
■ question bias
● did you read the literature today?
★ designing questions: general rules
○ remember your research question
○ decide exactly what you want to find out
○ consider your audience
○ imagine yourself as a respondent
■ would you answer the questions?
■ how would you answer the questions?
■ identify any vague or misleading questions
★ common mistakes:
○ ambiguous terms
■ for example: often, regularly, frequently
○ long questions
○ double-barreled questions
○ very general questions
■ for example: how happy are you?
○ leading questions
○ double negatives
○ technical terms
★ a question:
○ do respondents have the knowledge to answer the question?
○ do you just want a yes / no answer, have you given more possibilities?
○ have you thought whether you should include ‘don’t know’ options?
○ are you relying too much on the respondent’s memory?
★ use midpoint or don’t know?
○ if you include don’t know option, people will use it
■ if you need people to choose one or the other: do not include
don’t know
■ if it is possible that people lack the knowledge required to
answer a particular question: including don’t know can be
sensible
■ some will spend less time thinking about difficult questions
, ○ same holds for middle category
○ beware:
■ leaving out DK / middle option may cause irritation
■ and sometimes middle ground / DK is a possibility
★ being specific:
○ how detailed should the answer be?
○ more precise is more burdensome
○ very precise?
■ rounding off answers
★ getting answers: social desirability
○ make undesirable answer less threatening
○ desirability varies between groups
★ primacy and recency:
○ presentation of answers
○ primacy: first answer category
○ recency: last answer category
○ rotate answers
■ bias at individual level
★ open ended questions, solution?
○ no / less bias due to response options
■ increased validity
○ exhausting for a respondent
○ explorative questions
■ unanticipated responses
○ very important to clarify the question
■ height in centimeters
■ age in years
○ transformation after data collection / lots of different responses
○ sometimes answer categories are preferred
■ less detailed, but less exhausting
★ opened ended-questions:
○ advantages:
■ respondents answer in their own terms
■ allow for new, unexpected responses
■ exploratory - generate fixed answer categories
○ disadvantages:
■ time-consuming for (interviewer and) respondent
■ difficult to code
● combine positive and negative wording
● middle category, don’t know option
■ all items
● ask yourself: is a respondent capable and willing to
respond?
● social desirability?
week 1 hoorcollege 1
★ what is survey research about?
○ produce statistics / draw inferences about a target population.
★ advantages:
○ relatively low costs
○ relatively fast delivery
○ rapid data processing
○ can research large populations
○ various modes: phone, (e)mail, internet, face-to-f@ace
★ disadvantages:
○ generally limited to scaled or check-list questions
○ no control over response rate
○ provides information more than understanding
○ increasing public resistance
○ difficult to explore in depth
★ two inferences:
1. how well answers measure characteristics to be described.
2. how closely sample responding mirrors population.
Characteristics of
population
Sample of
population members
who answer
questions
Answers
respondents give
★ common sources of error in surveys:
○ a poorly worded question
○ deviations from the script by the interviewer
○ misunderstanding on the part of the interviewee
○ memory problems on the part of the interviewee
○ the way the information is recorded by the interviewer
, ○ the way the information is processed, either when answers are coded
or when data are entered into the computer
★ two types of error:
○ random error
■ sample: sampling error
● it happens
■ questions: invalidity
● honest mistake
○ systematic error or bias
■ sample bias
● only front row students?
■ question bias
● did you read the literature today?
★ designing questions: general rules
○ remember your research question
○ decide exactly what you want to find out
○ consider your audience
○ imagine yourself as a respondent
■ would you answer the questions?
■ how would you answer the questions?
■ identify any vague or misleading questions
★ common mistakes:
○ ambiguous terms
■ for example: often, regularly, frequently
○ long questions
○ double-barreled questions
○ very general questions
■ for example: how happy are you?
○ leading questions
○ double negatives
○ technical terms
★ a question:
○ do respondents have the knowledge to answer the question?
○ do you just want a yes / no answer, have you given more possibilities?
○ have you thought whether you should include ‘don’t know’ options?
○ are you relying too much on the respondent’s memory?
★ use midpoint or don’t know?
○ if you include don’t know option, people will use it
■ if you need people to choose one or the other: do not include
don’t know
■ if it is possible that people lack the knowledge required to
answer a particular question: including don’t know can be
sensible
■ some will spend less time thinking about difficult questions
, ○ same holds for middle category
○ beware:
■ leaving out DK / middle option may cause irritation
■ and sometimes middle ground / DK is a possibility
★ being specific:
○ how detailed should the answer be?
○ more precise is more burdensome
○ very precise?
■ rounding off answers
★ getting answers: social desirability
○ make undesirable answer less threatening
○ desirability varies between groups
★ primacy and recency:
○ presentation of answers
○ primacy: first answer category
○ recency: last answer category
○ rotate answers
■ bias at individual level
★ open ended questions, solution?
○ no / less bias due to response options
■ increased validity
○ exhausting for a respondent
○ explorative questions
■ unanticipated responses
○ very important to clarify the question
■ height in centimeters
■ age in years
○ transformation after data collection / lots of different responses
○ sometimes answer categories are preferred
■ less detailed, but less exhausting
★ opened ended-questions:
○ advantages:
■ respondents answer in their own terms
■ allow for new, unexpected responses
■ exploratory - generate fixed answer categories
○ disadvantages:
■ time-consuming for (interviewer and) respondent
■ difficult to code
● combine positive and negative wording
● middle category, don’t know option
■ all items
● ask yourself: is a respondent capable and willing to
respond?
● social desirability?
