Introduction to Research Methods
Part III: Tools for Evaluating Frequency Claims
Chapter 6: Surveys and Observations: Describing What People Do
LEARNING OBJECTIVES:
Explain how carefully prepared questions improve the construct validity of a poll or
survey.
Describe how researchers can make observations with good construct validity.
Surveys, polls, and observational methods are used to support frequency claims, they also
measure variables for associations and causal claims. When interrogating a claim based on
data from survey or an observational study, we ask about the construct validity of the
measurement.
Construct Validity of Surveys and Polls
Survey and Poll: both mean the same thing, a method of posing questions to people
on the phone, in personal interviews, on written questionnaires, or online.
Survey often used when people are asked about a consumer product.
Poll used when people are asked about their social or political opinions.
How much can you learn about a phenomenon just by asking people questions?
Choosing Question Formats
Survey questions can follow many formats.
Open-Ended Questions: allow correspondents to answer any way they like.
Advantage: provide researchers with spontaneous, rich information.
Disadvantage: responses must be coded and categorized, a process that can
often be difficult and time consuming.
Psychologists therefore, in the interest of efficiency, restrict the answers people can give.
Forced-Choice Format: people give their opinion by picking the best of two or more
options.
, Frequently used in political polls or for asking for opinions on current issues
or preference between two choices.
Likert Scale: when people are presented with a statement and are asked to use a
rating scale to indicate their degree of agreement. The scale contains more than one
item and is anchored by the terms strongly agree, agree, neither agree nor disagree,
etc.
Semantic Differential Format: instead of a degree of agreement, respondents might
be asked to rate a target object using a numerical scale that is anchored with
adjectives.
Writing Well-Worded Questions
The way a question is worded and presented in a survey can make a tremendous difference
in how people answer. Creators of surveys work to ensure that the wording and order of
questions wont influence respondents’ answers.
Leading Questions
Leading Question: there is a difference in meaning because of the wording of a
question.
Might not capture true opinions.
Double-Barreled Questions
Double-Barreled Question: asks two questions in one. Has poor construct validity,
because people might be responding to the first half of the question, the second half
of the question, or both.
Difficult to answer.
Negative Wording
Negatively Worded Question: can also make survey items unnecessarily
complicated. When a question is cognitively difficult for people, it can cause
confusion and thus reduce the construct validity of a survey.
Question Order
Part III: Tools for Evaluating Frequency Claims
Chapter 6: Surveys and Observations: Describing What People Do
LEARNING OBJECTIVES:
Explain how carefully prepared questions improve the construct validity of a poll or
survey.
Describe how researchers can make observations with good construct validity.
Surveys, polls, and observational methods are used to support frequency claims, they also
measure variables for associations and causal claims. When interrogating a claim based on
data from survey or an observational study, we ask about the construct validity of the
measurement.
Construct Validity of Surveys and Polls
Survey and Poll: both mean the same thing, a method of posing questions to people
on the phone, in personal interviews, on written questionnaires, or online.
Survey often used when people are asked about a consumer product.
Poll used when people are asked about their social or political opinions.
How much can you learn about a phenomenon just by asking people questions?
Choosing Question Formats
Survey questions can follow many formats.
Open-Ended Questions: allow correspondents to answer any way they like.
Advantage: provide researchers with spontaneous, rich information.
Disadvantage: responses must be coded and categorized, a process that can
often be difficult and time consuming.
Psychologists therefore, in the interest of efficiency, restrict the answers people can give.
Forced-Choice Format: people give their opinion by picking the best of two or more
options.
, Frequently used in political polls or for asking for opinions on current issues
or preference between two choices.
Likert Scale: when people are presented with a statement and are asked to use a
rating scale to indicate their degree of agreement. The scale contains more than one
item and is anchored by the terms strongly agree, agree, neither agree nor disagree,
etc.
Semantic Differential Format: instead of a degree of agreement, respondents might
be asked to rate a target object using a numerical scale that is anchored with
adjectives.
Writing Well-Worded Questions
The way a question is worded and presented in a survey can make a tremendous difference
in how people answer. Creators of surveys work to ensure that the wording and order of
questions wont influence respondents’ answers.
Leading Questions
Leading Question: there is a difference in meaning because of the wording of a
question.
Might not capture true opinions.
Double-Barreled Questions
Double-Barreled Question: asks two questions in one. Has poor construct validity,
because people might be responding to the first half of the question, the second half
of the question, or both.
Difficult to answer.
Negative Wording
Negatively Worded Question: can also make survey items unnecessarily
complicated. When a question is cognitively difficult for people, it can cause
confusion and thus reduce the construct validity of a survey.
Question Order
