Summary Research Workshop: Survey; Complete exam preparation

RESEARCH WORKSHOP – SURVEY COMPLETE LECTURE NOTES


WEEK 1
Fowler (2014): chapter 2, 6 and7

Lecture 1
Valid answers? Proper questions!

GOALS RESEARCH WORKSHOP: SURVEY

 You can justify the use of survey methods for research
 You get practical experience with the empirical cycle
 You get to apply quantitative research methods and techniques related to survey
research
 You practice with writing a research plan + report
 You can operationalize and measure constructs within the context of a survey study

What is survey research?
-> Produce statistics about target population

Why survey research?




TWO INFERENCES: An inference is an idea
I. The answers given by your participants measure the desired concepts or conclusion that's
II. Your sample reflects the population you aim to study drawn from evidence

TWO TYPES OF ERRORS:
1. Random

COMMON SOURCES OF RANDOM ERROR IN SURVEY RESEARCH:
 Ambiguous wording of questions
 Deviations from the script by the interviewer
 Misunderstanding on the part of the interviewee

,  Memory problems on the part of the interviewee

2. Systematic
 Social Desirability




GETTING ANSWERS: SOCIAL DESIRABILITY
-> Make undesirable answer less threatening:
 Everybody makes mistakes in their lives. What about you? Did you commit any serious
crimes in the past that vou may or may not regret?
 Have you ever made any of these transgressions? (check all that apply):
 Wolf-whistling
 Ignoring a homeless person begging for attention
 Hitting a person
 Stealing
 Murder
-> Social desirability varies between groups
 How much money do you earn?
 How often do you get completely drunk?
 How many sexual partners have you had?

"The quality of data will be no better than the most error-prone feature of the survey design"
(Fowler, 2014, p.6)

SURVEY METHODOLOGY IN PRACTICE:

1. Research objective
 the aim of the study is to investigate to what extent demographic characteristics are
associated to how people describe the taste of cilantro
2. List of variables
 Taste of cilantro
 Demographic characteristics (age [ratio], sex [nominal], ancestry[nominal] )
3. Analysis plan

DEVELOPING AND EVALUATING QUESTIONNAIRES

four phases by Fowler (2004):
I. Preliminary question design steps
II. Presurvey evaluation

, III. Design, format, and layout
IV. Field pre-test

EXAMPLE:
I. Preliminary question design steps:
1. Focus groups (e.g., terminology: cilantro or coriander?)
2. Drafting questions

COMMON MISTAKES
 Ambiguous terms: 'often', 'regularly', frequently
 Long questions
 Double-barrelled questions: may be different answers to each part
 Very general questions: because they lack a frame of reference
 Leading questions: hinting at a preferred response
 Negative terms: not', 'never' - especially double negatives
 Technical terms: (jargon and acronyms)

OPEN ENDED QUESTIONS
 No/less bias from response options
 Increased validity
 Tiresome for a respondent
 Explorative questions
 Unanticipated responses
 Very important to clarify the question
 Length in centimeters
 Age in years
 Transformations after data collection/ lot of different responses
 Sometimes answer categories are preferred
 Less detailed, but less tiresome



THE FALLIBLE BRAIN
 How often in the last two months did you visit a bar?
 How often in the last two months did you take an exam?
 How often in the last two months did you go online to find health information?

 Over - and under-reporting
 Media use: often over-reported
 Salient behaviours more accurately reported

INCLUDING "DON'T KNOW" AS A RESPONSE OPTION
-> If you include it, people use it
 If you need people to choose one or the other: do not include don't know
 If it is possible that people lack the knowledge required to answer a particular question:
including don't know can be sensible

Is there an alternative solution?


 Routing - filter questions
o Some questions are not relevant to every interviewee