February 10th, 2021
SOCI2151H Week 5: Quantitative Survey Design
Reading: Chapter 8
Part One: preparing a questionnaire
Best method for collecting data on large populations is to observe directly
● Must first have probability sampling!
○ If you are using quantitative methods, then you have 3 sampling
techniques to choose from
Guidelines for asking questions
● The key is to frame the questions that will get quality responses, since SRM’s
attempt to understand attitudes/behaviour
● Utilize the “likert” scale (usually from 1 to 5 - sometimes from 1 to 10)
Two ways to pose questions:
1. Open-ended questions
a. Provide space for the respondent to write their own answers
b. But not open to greater researcher bias
i. Not recommended for your proposal
ii. Can be useful at the end of the survey (‘other’ category)
c. Example:
i. What do you think about smoking?
ii. What does abuse mean to you?
2. Closed-ended questions
a. Respondents select an answer from a list
b. Most commonly used
c. Example:
i. Do you smoke? Check Yes or No
ii. Use of likert scale
d. Most popular and effective because of greater uniformity and easily
processed
e. Improved clarity, but still room for ambiguity because difficulty in
foreseeing responses
i. Use of ‘other’ category to fix
Closed-ended questions requirements
1. Response categories should be mutually exhaustive
2. Answer categories must be mutually exclusive
Asking Good Questions - Mutually Exclusive
, 2
BAD
● How often do you date?
Good!!
● In the last month, how many nights have you been on a date?
Asking Good Questions - Mutually Exhaustive
BAD
● What is your marital status? Married or single?
Good!!
● What is your marital status? Married, Single or Other
Avoid ‘double barrelled’ questions
● Poor: when i call for an appointment, the clerk answers promptly and is
courteous and efficiency
● Better:
○ when i call for an appointment the clerk answers promptly (agree or
disagree).
○ When I call for an appointment the clerk is courteous (agree or disagree).
○ When I call for an appointment the clerk is efficient (agree or disagree).
● AVOID THE USE OF AND!!!!!!!!!
● When measuring sensitive attitudes, provide an option, such as “don’t know/no
opinion/undecided”
Avoid leading or leaning questions
● Example: the majority of physicians feel that smoking is harmful. Do you agree?
○ Fix: do you agree or disagree that smoking is harmful
● Example: you don’t smoke, do you?
○ Fix: during the past week, did you smoke at least one cigarette?
Avoid jargon, slang and abbreviations
● Jargon: should the government spend even more money on assisting people with
disabilities than the millions they have already spent helping them?
ALWAYS pretest your questions on family, friends or colleagues
Part Two: survey design
Use contingency questions:
SOCI2151H Week 5: Quantitative Survey Design
Reading: Chapter 8
Part One: preparing a questionnaire
Best method for collecting data on large populations is to observe directly
● Must first have probability sampling!
○ If you are using quantitative methods, then you have 3 sampling
techniques to choose from
Guidelines for asking questions
● The key is to frame the questions that will get quality responses, since SRM’s
attempt to understand attitudes/behaviour
● Utilize the “likert” scale (usually from 1 to 5 - sometimes from 1 to 10)
Two ways to pose questions:
1. Open-ended questions
a. Provide space for the respondent to write their own answers
b. But not open to greater researcher bias
i. Not recommended for your proposal
ii. Can be useful at the end of the survey (‘other’ category)
c. Example:
i. What do you think about smoking?
ii. What does abuse mean to you?
2. Closed-ended questions
a. Respondents select an answer from a list
b. Most commonly used
c. Example:
i. Do you smoke? Check Yes or No
ii. Use of likert scale
d. Most popular and effective because of greater uniformity and easily
processed
e. Improved clarity, but still room for ambiguity because difficulty in
foreseeing responses
i. Use of ‘other’ category to fix
Closed-ended questions requirements
1. Response categories should be mutually exhaustive
2. Answer categories must be mutually exclusive
Asking Good Questions - Mutually Exclusive
, 2
BAD
● How often do you date?
Good!!
● In the last month, how many nights have you been on a date?
Asking Good Questions - Mutually Exhaustive
BAD
● What is your marital status? Married or single?
Good!!
● What is your marital status? Married, Single or Other
Avoid ‘double barrelled’ questions
● Poor: when i call for an appointment, the clerk answers promptly and is
courteous and efficiency
● Better:
○ when i call for an appointment the clerk answers promptly (agree or
disagree).
○ When I call for an appointment the clerk is courteous (agree or disagree).
○ When I call for an appointment the clerk is efficient (agree or disagree).
● AVOID THE USE OF AND!!!!!!!!!
● When measuring sensitive attitudes, provide an option, such as “don’t know/no
opinion/undecided”
Avoid leading or leaning questions
● Example: the majority of physicians feel that smoking is harmful. Do you agree?
○ Fix: do you agree or disagree that smoking is harmful
● Example: you don’t smoke, do you?
○ Fix: during the past week, did you smoke at least one cigarette?
Avoid jargon, slang and abbreviations
● Jargon: should the government spend even more money on assisting people with
disabilities than the millions they have already spent helping them?
ALWAYS pretest your questions on family, friends or colleagues
Part Two: survey design
Use contingency questions:
