RS: Survey Lectures
6 steps on how to make a survey
Step 1: what do you want to measure
a) Manifest variables: can be directly observed. E.g. height, hair colour
b) Latent variables: can only be observed indirectly. E.g. wealth, intelligence. E.g. attitudes as
- Multiple questions because:
o The concepts are multi-faceted
o When you ask multiple questions about the same construct you will at the very
least be able to establish that you have measured one underlying thing
o You can detect/decrease the influence of unsystematic errors (people providing
the wrong answer)
- Self-report measures: Measurement that represent a set of indicators of the latent
construct If you score high on Y, then this should reflect in A,B,C,..
Validity: Whether the results really do represent what they are supposed to measure)
Reliability: Whether the results can be reproduced under the same conditions (internally consistent)
Step 2: From theory to questions and answers
- Option 1: existing scales E.g. need for cognition, privacy concerns
o Advantages: the scales are validated
o Disadvantages: language/translations, not all scientists are survey methodologists.
- Option 2: when developing your own items
o Internal method (inductive): statistical approach - Many items are used and
through statistical grouping techniques it is decided which ones were relevant.
o Facet method (deductive): Instrument should fully represent each dimension of
the construct that is intended to be measured. explain why you came up with a
specific item (theoretically driven)
Step 3: Phrase specific items ~ Krosnick & Presser, 2010
Conventional wisdom:
- Use simple familiar words (avoid technical terms, jargon and slang)
- Use simple syntax
- Avoid words with ambiguous meanings, i.e., aim for wording that all respondents will
interpret in the same way
- Strive for wording that is specific and concrete (as opposed to general and abstract)
- Make response options exhaustive and mutually exclusive
- Avoid leading or loaded questions that push respondents toward an answer
- Ask about one thing at a time (Avoid double-barrelled questions)
- Avoid questions with single or double negations
CASM: cognitive aspects of survey methodology
Stages of the question-answering process ~Tourangeau et al., 2000
Stage 0 (Encoding): Turning thoughts into communication The mere fact that someone has lived
through an event, does not necessarily mean that he or she absorbed much information about it
E.g. “what did you have for breakfast?” not very distinctive, people might not remember, so a lot of
discrepancies
6 steps on how to make a survey
Step 1: what do you want to measure
a) Manifest variables: can be directly observed. E.g. height, hair colour
b) Latent variables: can only be observed indirectly. E.g. wealth, intelligence. E.g. attitudes as
- Multiple questions because:
o The concepts are multi-faceted
o When you ask multiple questions about the same construct you will at the very
least be able to establish that you have measured one underlying thing
o You can detect/decrease the influence of unsystematic errors (people providing
the wrong answer)
- Self-report measures: Measurement that represent a set of indicators of the latent
construct If you score high on Y, then this should reflect in A,B,C,..
Validity: Whether the results really do represent what they are supposed to measure)
Reliability: Whether the results can be reproduced under the same conditions (internally consistent)
Step 2: From theory to questions and answers
- Option 1: existing scales E.g. need for cognition, privacy concerns
o Advantages: the scales are validated
o Disadvantages: language/translations, not all scientists are survey methodologists.
- Option 2: when developing your own items
o Internal method (inductive): statistical approach - Many items are used and
through statistical grouping techniques it is decided which ones were relevant.
o Facet method (deductive): Instrument should fully represent each dimension of
the construct that is intended to be measured. explain why you came up with a
specific item (theoretically driven)
Step 3: Phrase specific items ~ Krosnick & Presser, 2010
Conventional wisdom:
- Use simple familiar words (avoid technical terms, jargon and slang)
- Use simple syntax
- Avoid words with ambiguous meanings, i.e., aim for wording that all respondents will
interpret in the same way
- Strive for wording that is specific and concrete (as opposed to general and abstract)
- Make response options exhaustive and mutually exclusive
- Avoid leading or loaded questions that push respondents toward an answer
- Ask about one thing at a time (Avoid double-barrelled questions)
- Avoid questions with single or double negations
CASM: cognitive aspects of survey methodology
Stages of the question-answering process ~Tourangeau et al., 2000
Stage 0 (Encoding): Turning thoughts into communication The mere fact that someone has lived
through an event, does not necessarily mean that he or she absorbed much information about it
E.g. “what did you have for breakfast?” not very distinctive, people might not remember, so a lot of
discrepancies
