WEEK 1- VALID ANSWERS? PROPER
QUESTIONS!
Chapter Summary: Surveys are designed to produce statistics about a target population. The process
by which this is done rests on inferring the characteristics of the target population from the answers
provided by a sample of respondents. This chapter outlines the two kinds of inferences that are
required. It also describes the two types of error, bias and variability, that can limit the accuracy of
those inferences.
What is a survey research process
o Produce statistics about a target population
Purpose of a Survey:
o Goal: Provide statistical estimates of the characteristics of a target population
Fundamental premises of survey research process:
1. By describing the sample of people who respond, we can describe the target population aka ->
sample reflects population aimed
2. Answers people give can be used to accurately describe characteristics of the respondents aka -
> answers given by respondents measure the desired concepts
,Survey methodology Addresses:
o To determine how well answers collected represent what they intended to measure
o How similar is the sample from the population
o Factors that affect Error:
o Design of survey
o Data collection methods
Two types of error: random & systematic:
1. Random
o sampling error: variations between the sample statistics and the population parameters
o invalidity
2. Systematic (aka biased)
, o social desirability (sometimes): can be because of the vagueness of the wording (question) so
the answers are different as the intended ones
o Un/over-represented people: 65+ are less likely to answer to telephone surveys
Social desirability -> have to make undesirable questions less threatening
-> doesn’t have to seem to personal (doesn’t have to contain specific details
about a music band that the respondent might like or not, for e.g.) or with
potential socially undesirable answers, and pay attention to avoid double-
barreled questions and avoid making too long questions
Social desirability bias is when responders answer differently due to taboo (like is the case of
mental illness reporting or smoking)
How to change it:
Review the wording- less judgy
From “did you ever committed a crime in your life?” to “Everybody
makes mistakes, what about you? Did you ever committed any
serious crime that you may or may not regret?” (make it less
threatening)
Adopt multiple choices questions (to “hide” the question)
Anonimity and confidentiality
Self administered data entry
Social desirability varies between groups: how much money do you earn? / how often
do you get drunk?
Common sources of random error in a survey research:
o Ambiguous wording of questions (undefined concepts, not clear enough, frustrating…)
o Deviations from the script by the interviewer
o Misunderstanding on the part of the interviewee
o Memory problems on the part of the interviewee (questions that need flashbacks)
Survey methodology in practice:
1. Research objective
o The aim of the study is to investigate to what extent demographic characteristics are associated
to how people describe the taste of cilantro
o -> associate demographics of people and their answers, compare subgroups of a population
studied
2. List of variables
o Demographics (sex, country of residence, age) 🡪 nominal and ratio data
o Taste of cilantro -> binary
o There is always an IV and a DV
QUESTIONS!
Chapter Summary: Surveys are designed to produce statistics about a target population. The process
by which this is done rests on inferring the characteristics of the target population from the answers
provided by a sample of respondents. This chapter outlines the two kinds of inferences that are
required. It also describes the two types of error, bias and variability, that can limit the accuracy of
those inferences.
What is a survey research process
o Produce statistics about a target population
Purpose of a Survey:
o Goal: Provide statistical estimates of the characteristics of a target population
Fundamental premises of survey research process:
1. By describing the sample of people who respond, we can describe the target population aka ->
sample reflects population aimed
2. Answers people give can be used to accurately describe characteristics of the respondents aka -
> answers given by respondents measure the desired concepts
,Survey methodology Addresses:
o To determine how well answers collected represent what they intended to measure
o How similar is the sample from the population
o Factors that affect Error:
o Design of survey
o Data collection methods
Two types of error: random & systematic:
1. Random
o sampling error: variations between the sample statistics and the population parameters
o invalidity
2. Systematic (aka biased)
, o social desirability (sometimes): can be because of the vagueness of the wording (question) so
the answers are different as the intended ones
o Un/over-represented people: 65+ are less likely to answer to telephone surveys
Social desirability -> have to make undesirable questions less threatening
-> doesn’t have to seem to personal (doesn’t have to contain specific details
about a music band that the respondent might like or not, for e.g.) or with
potential socially undesirable answers, and pay attention to avoid double-
barreled questions and avoid making too long questions
Social desirability bias is when responders answer differently due to taboo (like is the case of
mental illness reporting or smoking)
How to change it:
Review the wording- less judgy
From “did you ever committed a crime in your life?” to “Everybody
makes mistakes, what about you? Did you ever committed any
serious crime that you may or may not regret?” (make it less
threatening)
Adopt multiple choices questions (to “hide” the question)
Anonimity and confidentiality
Self administered data entry
Social desirability varies between groups: how much money do you earn? / how often
do you get drunk?
Common sources of random error in a survey research:
o Ambiguous wording of questions (undefined concepts, not clear enough, frustrating…)
o Deviations from the script by the interviewer
o Misunderstanding on the part of the interviewee
o Memory problems on the part of the interviewee (questions that need flashbacks)
Survey methodology in practice:
1. Research objective
o The aim of the study is to investigate to what extent demographic characteristics are associated
to how people describe the taste of cilantro
o -> associate demographics of people and their answers, compare subgroups of a population
studied
2. List of variables
o Demographics (sex, country of residence, age) 🡪 nominal and ratio data
o Taste of cilantro -> binary
o There is always an IV and a DV
