Survey Lectures
Week 1: Validity and Designing Questions
• A series of formatted questions (e.g. checklists and scales).
• Delivered to a de ned sample of people.
• Seeks to produce statistics about a target population.
• With the expectation of responses immediately or within a few days.
• Common method for capturing public opinion.
• Amongst di erent generations or students.
• Modes: Use of phone, mail, e-mail, website or personal interview to deliver.
• Must be carefully worded and pre-tested.
• Cannot assess causal relationships between variables (only provide indication in speci c
cases).
• Good question and instrument evaluation prior to actually doing a survey is one of the least
expensive ways of to reduce error in survey estimates.
• Requires careful question design and pretesting.
Advantages Disadvantages
Relatively low cost Di cult to give in-depth answer “why” question or
(implementation) give explanations.
(no determine cause-e ect)
Fast delivery No control over response rate
Rapid data processing Increasing public resistance - survey fatigue and
unwillingness to participate
Reach large populations Di cult to explore issues in depth
Multiple methods (phone, internet, e-mail) Provides information over understanding
Can make generalizations to population (with Limited to scaled or check-list questions.
proper sampling)
1. Types of Surveys
1. Cross-sectional
2. Longitudinal
1. Trend studies
2. Cohort
3. Panel
4. Cross-lagged
ffi
ffi ff fi ff fi
, A. Trend – di erent people over time
B. Cohort – same type of people over time
C. Panel Studies – same individuals over time
D. Cross-lagged - measures a IV and DV at 2 points in time.
#A
Trend Studies
Di erent people over time
• Examines changes in trend opinion across time (in population).
• A di erent set of people asked each time.
Each study collects data from di erent individuals
• Aim: Find changes in trend of opinions over time.
How attitude in society changes over time
The di erence in opinions is the trend.
A: Size remains even if people drop out the research.
D: Not clear why the change occurs.
It is not collecting the info from the same individual
#A.1
Example
In 2007, 1,000 people are asked for their support of same-sex marriage. In 2017, a different set
of 1,000 people are asked for their support of same-sex marriage.
The di erence in opinions is the trend in support for gay marriage.
#B
Cohort Studies
Same type of people over time
• Examines changes in a cohort across time.
Cohort: Share a similar characteristic (e.g. age, year of study).
• E.g. People who are born in the same year.
• Aim: Find changes in cohort.
• How: Ask multiple questions over time.
• A di erent set of individuals from the same cohort asked each time.
Ask the same cohort BUT di erent individuals
Advantage: Provides more detail than trend
Disadvantage: Determining cause and e ect is a problem.
• Why? Because the same people are not studied over time.
ffffff ff ff ff ff ff
, #B.1
Example
To study the relationship between nutrition and health in children, a cohort of newborns born
in 2012, was created. Each following year a random sample of the cohort was surveyed
(parents).
The cohort has the same characteristic: born in 2012.
#C
Panel Studies
Same individuals over time
• Examines changes in same individuals across time.
• Each study collects data for the same individuals.
• Divided into waves (di erent points in time)
A: More detail than a cohort + Provides opportunity for more in-depth analysis
D: Expensive + Attrition (drop-out) + cannot determine cause-e ect relationships.
• One must account for attrition (people drop o the study) – each wave of data is collected from
the panel of subjects.
#C.1
Example
Study aimed at examining the relationship between risky sexual behaviour and perceptions
of peer involvement.
In 2008, Dutch adolescents between the ages of 12 and 17 participated in an online panel study. In
the first wave (May 2008) 1,765 adolescents participated. In the second wave (November 2008)
1,445 adolescents who participated in Wave 1, completed the questionnaire.
There are 2 waves.
2. Steps for Survey Research
In order to carry out survey research:
1. RQ
2. Create a questionnaire
De ne variables and concepts
Create questions for variables and demographics
Merging questions
Introduction of survey
3. Enter survey
4. Administer survey
fi ff ff ff
, 5. Analyse data
6. Report results
7. Present results
3. From research questions (RQ) to operationalization
• De ne which is the subject that will be investigated.
• 1. Research objective → 2. List of variables → 3. Analysis plan
• Topics range from:
• Socia media + self-concept
• Watching TV + attitude towards ethic minorities
• Grati cations + social media
• Crime series + sense of safety on streets
• Product placement + purchase intention
• Smoking behaviour in lms + attitude towards smoking.
• News + perception of fake news
• Type of newspaper + worldviews.
• Politicians on Twitter + trust in politics.
• Companies + sustainability.
• Diversity in workplace + organization identi cation.
#A
Creating Research Questions
• Requirements for a good RQ:
1. Not too broad
2. Contain variables de ned
3. Relevant to comm. science
4. No yes/no question
• To create a research question, we need to look closer at the concepts in the research question
in order to transform them into clear variables.
• Try to describe the variables of the research question and think of how to
operationalize it.
• Sub-questions can be made too.
• Since doing a survey —> talk about relation rather than cause/e ect.
• Keep in mind the target audience.
• Operationalization of concepts:
• You have at least 1 latent variable and 5 questions to measure it.
• Create at least 10 survey questions in total.
• Theoretical de nitions of the variables.
• Items measure what they have to measure
fi fi
fi fi fi fi ff
Week 1: Validity and Designing Questions
• A series of formatted questions (e.g. checklists and scales).
• Delivered to a de ned sample of people.
• Seeks to produce statistics about a target population.
• With the expectation of responses immediately or within a few days.
• Common method for capturing public opinion.
• Amongst di erent generations or students.
• Modes: Use of phone, mail, e-mail, website or personal interview to deliver.
• Must be carefully worded and pre-tested.
• Cannot assess causal relationships between variables (only provide indication in speci c
cases).
• Good question and instrument evaluation prior to actually doing a survey is one of the least
expensive ways of to reduce error in survey estimates.
• Requires careful question design and pretesting.
Advantages Disadvantages
Relatively low cost Di cult to give in-depth answer “why” question or
(implementation) give explanations.
(no determine cause-e ect)
Fast delivery No control over response rate
Rapid data processing Increasing public resistance - survey fatigue and
unwillingness to participate
Reach large populations Di cult to explore issues in depth
Multiple methods (phone, internet, e-mail) Provides information over understanding
Can make generalizations to population (with Limited to scaled or check-list questions.
proper sampling)
1. Types of Surveys
1. Cross-sectional
2. Longitudinal
1. Trend studies
2. Cohort
3. Panel
4. Cross-lagged
ffi
ffi ff fi ff fi
, A. Trend – di erent people over time
B. Cohort – same type of people over time
C. Panel Studies – same individuals over time
D. Cross-lagged - measures a IV and DV at 2 points in time.
#A
Trend Studies
Di erent people over time
• Examines changes in trend opinion across time (in population).
• A di erent set of people asked each time.
Each study collects data from di erent individuals
• Aim: Find changes in trend of opinions over time.
How attitude in society changes over time
The di erence in opinions is the trend.
A: Size remains even if people drop out the research.
D: Not clear why the change occurs.
It is not collecting the info from the same individual
#A.1
Example
In 2007, 1,000 people are asked for their support of same-sex marriage. In 2017, a different set
of 1,000 people are asked for their support of same-sex marriage.
The di erence in opinions is the trend in support for gay marriage.
#B
Cohort Studies
Same type of people over time
• Examines changes in a cohort across time.
Cohort: Share a similar characteristic (e.g. age, year of study).
• E.g. People who are born in the same year.
• Aim: Find changes in cohort.
• How: Ask multiple questions over time.
• A di erent set of individuals from the same cohort asked each time.
Ask the same cohort BUT di erent individuals
Advantage: Provides more detail than trend
Disadvantage: Determining cause and e ect is a problem.
• Why? Because the same people are not studied over time.
ffffff ff ff ff ff ff
, #B.1
Example
To study the relationship between nutrition and health in children, a cohort of newborns born
in 2012, was created. Each following year a random sample of the cohort was surveyed
(parents).
The cohort has the same characteristic: born in 2012.
#C
Panel Studies
Same individuals over time
• Examines changes in same individuals across time.
• Each study collects data for the same individuals.
• Divided into waves (di erent points in time)
A: More detail than a cohort + Provides opportunity for more in-depth analysis
D: Expensive + Attrition (drop-out) + cannot determine cause-e ect relationships.
• One must account for attrition (people drop o the study) – each wave of data is collected from
the panel of subjects.
#C.1
Example
Study aimed at examining the relationship between risky sexual behaviour and perceptions
of peer involvement.
In 2008, Dutch adolescents between the ages of 12 and 17 participated in an online panel study. In
the first wave (May 2008) 1,765 adolescents participated. In the second wave (November 2008)
1,445 adolescents who participated in Wave 1, completed the questionnaire.
There are 2 waves.
2. Steps for Survey Research
In order to carry out survey research:
1. RQ
2. Create a questionnaire
De ne variables and concepts
Create questions for variables and demographics
Merging questions
Introduction of survey
3. Enter survey
4. Administer survey
fi ff ff ff
, 5. Analyse data
6. Report results
7. Present results
3. From research questions (RQ) to operationalization
• De ne which is the subject that will be investigated.
• 1. Research objective → 2. List of variables → 3. Analysis plan
• Topics range from:
• Socia media + self-concept
• Watching TV + attitude towards ethic minorities
• Grati cations + social media
• Crime series + sense of safety on streets
• Product placement + purchase intention
• Smoking behaviour in lms + attitude towards smoking.
• News + perception of fake news
• Type of newspaper + worldviews.
• Politicians on Twitter + trust in politics.
• Companies + sustainability.
• Diversity in workplace + organization identi cation.
#A
Creating Research Questions
• Requirements for a good RQ:
1. Not too broad
2. Contain variables de ned
3. Relevant to comm. science
4. No yes/no question
• To create a research question, we need to look closer at the concepts in the research question
in order to transform them into clear variables.
• Try to describe the variables of the research question and think of how to
operationalize it.
• Sub-questions can be made too.
• Since doing a survey —> talk about relation rather than cause/e ect.
• Keep in mind the target audience.
• Operationalization of concepts:
• You have at least 1 latent variable and 5 questions to measure it.
• Create at least 10 survey questions in total.
• Theoretical de nitions of the variables.
• Items measure what they have to measure
fi fi
fi fi fi fi ff
