Goal of survey research is to draw inferences about a target population
Advantage Disadvantages
- Relatively low costs - Generally limited to scaled or check-list
- Relatively fast delivery questions
- Rapid data processing - No control over response rate
- Can research large populations - Provides information more than
- Various modes understanding
o Phone email internet face-to-face - Increasing public resistance
- Difficult to explore issues in depth
Goal is to provide info about population (target audience) make 2 inferences
- from sample to population = how closely the sample reflect the population
o Sampling error
- From given answers to sample = how well answers measures characteristics
o Invalidity
Errors that are present in surveys
Common sources
- Poorly worded question
o Invalid finding
- Deviation form script by interviewer (F2F and telephone)
- Misunderstanding on the part of the interviewee
o Unclear questions
o How many drinks do have in the weekend?
What is a drink?
- Memory problems on the part of the interviewee
o A lot of attention is asked
- The way the information is recorded by the interviewer
- The way the information is processed, either when answers are coded or when data are
entered into the computer
TYPES OF ERROR
- Random error
o possible error that stems solely from the fact that data are collected form a sample
rather than from every single member of the population influences reliability
e.g., the sample does not match the population, this can always happen the
respondent does not understand the question, honest mistake
- Systematic error or bias
o in some systematic way, the people responding to a survey are different from the
target population as a whole influences validity
e.g., only Dutch students in sample, sample does not reflect population have
you read the literature? Many will say yes
Bias = in a systematic way people respond differently like the target population
Three steps in data collection from the sample which can cause bias:
- Sample frame (those who have a chance of being selected): there are people in the
population who have no chance of participating
- Process of selection: if the process of selection is non-random
- Not possible to obtain answers from everyone: e.g., health problems or language delay
Validity = to what extent do the answers measure what we want to measure
, DESIGNING QUESTIONS
There are two types of questions:
- Objective facts = respondents can fill out right away, because they are facts (height, weight)
- Subjective states = respondents must think carefully about these (feelings)
General rules
- Remember RQ
- Decide exactly what you want to find out
- Consider your audience
- Imagine yourself as a respondent
Common in research questions
- Ambiguous terms Often, regularly, frequently
- Long questions Relying on memory – make into separate questions
- double-barreled fixed by making two
- Very general questions
- Leading questions/biased often problematic unless it is a sensitive question (do you also
believe ….)
- Negative terms “not”, “never” and double negatives is even worse
- Technical terms: people need to be able to fully understand the question
- Poorly phrased questions
- Respondent has not enough knowledge to answer the question
To ensure reliability:
- Each respondent is asked the same set of questions
- Answers to these questions are recorded
Use midpoint or don’t know
If you include don’t know or middle option, people will use it
- If you need people to choose one or the other, do not include
- If it is possible that people lack the knowledge required to answer a particular question,
include don’t know can be sensible
- If used, some people will spend less time thinking about difficult questions
Be aware:
- Leaving out “don’t know” or middle option may cause irritation
- And sometimes “don’t know” or middle option is a possibility
MEASUREMENTS LEVELS
Ratio is often convenient for the researcher, but not for the respondent
People often round off to whole numbers
Decide for yourself:
- How detailed should the question be?
- The more precise the more stressful for the respondent
Totally agree – agree – disagree – totally disagree
Ordinal, but combined with multiple items interval
Agree X X X X X X X X X (more than 7) disagree
Ordinal treat as interval
Advantage Disadvantages
- Relatively low costs - Generally limited to scaled or check-list
- Relatively fast delivery questions
- Rapid data processing - No control over response rate
- Can research large populations - Provides information more than
- Various modes understanding
o Phone email internet face-to-face - Increasing public resistance
- Difficult to explore issues in depth
Goal is to provide info about population (target audience) make 2 inferences
- from sample to population = how closely the sample reflect the population
o Sampling error
- From given answers to sample = how well answers measures characteristics
o Invalidity
Errors that are present in surveys
Common sources
- Poorly worded question
o Invalid finding
- Deviation form script by interviewer (F2F and telephone)
- Misunderstanding on the part of the interviewee
o Unclear questions
o How many drinks do have in the weekend?
What is a drink?
- Memory problems on the part of the interviewee
o A lot of attention is asked
- The way the information is recorded by the interviewer
- The way the information is processed, either when answers are coded or when data are
entered into the computer
TYPES OF ERROR
- Random error
o possible error that stems solely from the fact that data are collected form a sample
rather than from every single member of the population influences reliability
e.g., the sample does not match the population, this can always happen the
respondent does not understand the question, honest mistake
- Systematic error or bias
o in some systematic way, the people responding to a survey are different from the
target population as a whole influences validity
e.g., only Dutch students in sample, sample does not reflect population have
you read the literature? Many will say yes
Bias = in a systematic way people respond differently like the target population
Three steps in data collection from the sample which can cause bias:
- Sample frame (those who have a chance of being selected): there are people in the
population who have no chance of participating
- Process of selection: if the process of selection is non-random
- Not possible to obtain answers from everyone: e.g., health problems or language delay
Validity = to what extent do the answers measure what we want to measure
, DESIGNING QUESTIONS
There are two types of questions:
- Objective facts = respondents can fill out right away, because they are facts (height, weight)
- Subjective states = respondents must think carefully about these (feelings)
General rules
- Remember RQ
- Decide exactly what you want to find out
- Consider your audience
- Imagine yourself as a respondent
Common in research questions
- Ambiguous terms Often, regularly, frequently
- Long questions Relying on memory – make into separate questions
- double-barreled fixed by making two
- Very general questions
- Leading questions/biased often problematic unless it is a sensitive question (do you also
believe ….)
- Negative terms “not”, “never” and double negatives is even worse
- Technical terms: people need to be able to fully understand the question
- Poorly phrased questions
- Respondent has not enough knowledge to answer the question
To ensure reliability:
- Each respondent is asked the same set of questions
- Answers to these questions are recorded
Use midpoint or don’t know
If you include don’t know or middle option, people will use it
- If you need people to choose one or the other, do not include
- If it is possible that people lack the knowledge required to answer a particular question,
include don’t know can be sensible
- If used, some people will spend less time thinking about difficult questions
Be aware:
- Leaving out “don’t know” or middle option may cause irritation
- And sometimes “don’t know” or middle option is a possibility
MEASUREMENTS LEVELS
Ratio is often convenient for the researcher, but not for the respondent
People often round off to whole numbers
Decide for yourself:
- How detailed should the question be?
- The more precise the more stressful for the respondent
Totally agree – agree – disagree – totally disagree
Ordinal, but combined with multiple items interval
Agree X X X X X X X X X (more than 7) disagree
Ordinal treat as interval
