Notes book final exam
Chapter 7: Survey Research
Survey can be administered in two keyways:
1. The questionnaire self-administered
Questionnaires can also be with open-ended questions.
Mail survey Paper-and-pencil survey sent through the mail
Group-administered questionnaire Administered to respondents in a group
setting
Household drop-off survey Paper-and-pencil survey that is administered by
dropping it off at the respondent’s household and, either picking it up at a later time,
or having the respondent return it directly.
Point-of-experience survey Delivered at or immediately after the experience that
the respondent is being asked about. (right after a training event etc.)
Electronic survey Administered via computer program (email or website)
Email survey/web survey Distributed via email
Dual-media surveys Distributed simultaneously in two ways. For instance, if you
distribute a survey to participants as an attachment or they can complete directly on
the web as a web form.
2. The interview Far more personal than questionnaires
Personal interview Interviewer works directly with the respondent; the
interviewer has the opportunity to ask follow-up questions.
Group interview In a group setting
Focus group The input is on one or more focus topics and is collected from
participants in a small group setting.
Telephone interview A personal interview that is conducted over the telephone
,Selecting the survey method:
Population issues:
- Can the population be identified?
- Is the population literate?
- Are there language issues?
- Will the population cooperate?
- What are the geographic restrictions?
Sampling issues:
- What data is available?
- Can respondents be found?
- Who is the respondent?
- Can all members of the population be sampled?
- Are response rates likely to be a problem?
- Will incentives for participation help?
Question issues:
- What types of questions can you ask?
- Will filter questions be needed? (A question to determine if your respondent is
qualified to answer the questions of interest)
- Can question sequence be controlled?
- Will lengthy questions be asked?
- Will long response scales be used?
Content issues:
- Can the respondents be expected to know about the issue?
- Will the respondent need to consult records?
Bias issues:
- Can social desirability be avoided? (people want to look good in the eyes of others)
- Can interviewer distortion and subversion be controlled?
- Can false respondents be avoided?
Administrative issues:
- Costs
- Facilities
- Time
- Personnel
Clearly, there are lots of issues to consider when you are selecting which type of survey to
use in your study.
Practice questions: 12/15 first attempt
, Chapter 8: Introduction to design
You must meet 3 criteria before you can say that you have evidence for a causal relationship:
1. Temporal precedence time order, cause must occur before effect
2. Covariation of the cause and effect if X than Y, if not X than not Y. If that is the
case, you proved that there is a relationship
3. No plausible alternative explanations For example, shoe size and reading ability
for children
Continuous relationship If more of X than more of Y, if less of X than less of Y.
Correlation does not mean causation!
Third variable/missing variable problem Other variable or factor is causing the outcome
At the heart of internal validity
A simple way of ruling out the third variable/missing variable problem is by using a control
group.
Internal validity is the approximate truth about inferences regarding cause-effect or causal
relationships
Establishing internal validity you have ruled out plausible alternative explanations and
have thus shown that indeed a causal relationship likely exists between your program and
the intended outcome
All that internal validity means is that you have sufficient evidence that what you did in your
immediate study (the program) caused what you observed (the outcome) to happen
Threats to internal validity Any factor that leads you to draw an incorrect conclusion that
your treatment or program causes the outcome
Single-group threats occurs in a study that uses only a single group (no control group)
Multiple-group threats occurs in studies that use multiple groups for instance, a program
and a comparison group
Social threats to internal validity Threats to internal validity that arise because social
research is conducted in real-world human contexts where people will react to not only what
affects them, but also to what is happening to others around them
Chapter 7: Survey Research
Survey can be administered in two keyways:
1. The questionnaire self-administered
Questionnaires can also be with open-ended questions.
Mail survey Paper-and-pencil survey sent through the mail
Group-administered questionnaire Administered to respondents in a group
setting
Household drop-off survey Paper-and-pencil survey that is administered by
dropping it off at the respondent’s household and, either picking it up at a later time,
or having the respondent return it directly.
Point-of-experience survey Delivered at or immediately after the experience that
the respondent is being asked about. (right after a training event etc.)
Electronic survey Administered via computer program (email or website)
Email survey/web survey Distributed via email
Dual-media surveys Distributed simultaneously in two ways. For instance, if you
distribute a survey to participants as an attachment or they can complete directly on
the web as a web form.
2. The interview Far more personal than questionnaires
Personal interview Interviewer works directly with the respondent; the
interviewer has the opportunity to ask follow-up questions.
Group interview In a group setting
Focus group The input is on one or more focus topics and is collected from
participants in a small group setting.
Telephone interview A personal interview that is conducted over the telephone
,Selecting the survey method:
Population issues:
- Can the population be identified?
- Is the population literate?
- Are there language issues?
- Will the population cooperate?
- What are the geographic restrictions?
Sampling issues:
- What data is available?
- Can respondents be found?
- Who is the respondent?
- Can all members of the population be sampled?
- Are response rates likely to be a problem?
- Will incentives for participation help?
Question issues:
- What types of questions can you ask?
- Will filter questions be needed? (A question to determine if your respondent is
qualified to answer the questions of interest)
- Can question sequence be controlled?
- Will lengthy questions be asked?
- Will long response scales be used?
Content issues:
- Can the respondents be expected to know about the issue?
- Will the respondent need to consult records?
Bias issues:
- Can social desirability be avoided? (people want to look good in the eyes of others)
- Can interviewer distortion and subversion be controlled?
- Can false respondents be avoided?
Administrative issues:
- Costs
- Facilities
- Time
- Personnel
Clearly, there are lots of issues to consider when you are selecting which type of survey to
use in your study.
Practice questions: 12/15 first attempt
, Chapter 8: Introduction to design
You must meet 3 criteria before you can say that you have evidence for a causal relationship:
1. Temporal precedence time order, cause must occur before effect
2. Covariation of the cause and effect if X than Y, if not X than not Y. If that is the
case, you proved that there is a relationship
3. No plausible alternative explanations For example, shoe size and reading ability
for children
Continuous relationship If more of X than more of Y, if less of X than less of Y.
Correlation does not mean causation!
Third variable/missing variable problem Other variable or factor is causing the outcome
At the heart of internal validity
A simple way of ruling out the third variable/missing variable problem is by using a control
group.
Internal validity is the approximate truth about inferences regarding cause-effect or causal
relationships
Establishing internal validity you have ruled out plausible alternative explanations and
have thus shown that indeed a causal relationship likely exists between your program and
the intended outcome
All that internal validity means is that you have sufficient evidence that what you did in your
immediate study (the program) caused what you observed (the outcome) to happen
Threats to internal validity Any factor that leads you to draw an incorrect conclusion that
your treatment or program causes the outcome
Single-group threats occurs in a study that uses only a single group (no control group)
Multiple-group threats occurs in studies that use multiple groups for instance, a program
and a comparison group
Social threats to internal validity Threats to internal validity that arise because social
research is conducted in real-world human contexts where people will react to not only what
affects them, but also to what is happening to others around them
