Research design
o Experimental designs
o Cross-sectional and longitudinal designs
o Comparative designs
o Historical designs
o = strategy for investigating RQ/hypothesis in coherent and logical way, incl what kind
of data is needed, how the data is collected and what methods of analysis will be used
Methods of data collection
o Questionnaires/surveys
o Interviews/focus groups
o Participant observation/ethnographic research
o Textual/discourse/content analysis
o = requires the def of clear boundaries (time, place, units, factors, variables)
6 main ethical issues
o 1. Voluntary participation
o 2. Informed consent
o 3. privacy
o 4. Harm
o 5. Exploitation
o 6. Consequences for future research
Internal validity
o Causal inferences
o Extent to which we can be confident that IV (causal) produced observed effect
o biggest threats to validity: if you do not design your research in way that enables you
to rule out other, alternative explanations, the internal validity of study = threatened
External validity
o Generalizability, extent to which results from a study can be generalized beyond the
particular study
Reliability
o Refers to the repeatability or consistency of your findings
o Research design is reliable if other researchers can perform exactly same procedures
and come up with same results
o Is a necessary condition for validity
Measurement reliability
o Consistency and precision
o Types: test-retests; internal consistency; intercoder/inter-rater reliability
Measurement validity
o Accuracy
, o Types: face, content, criterion (aka construct validity): concurrent, predictive,
convergent, discriminant
Research design
Experimental design
Experimental designs
o Randomized intervention
o Thought to most closely resemble the true scientific methods
o Most effective design for testing whether or not two variables are casually related
Types
o Laboratory
o Field
o Natural
o Survey
Ethical issues
o Treatment usually concerns concealment and/or deception
o Field studies: real world effects, debriefing not possible
o Tend to be more pronounced in experimental research
o Deception
o Issues of professional conduct
o Publication bias
o Harm: may cause the participants distress
o Must make a risk/gain assessment
Advantages Disadvantages
Ability to derive causal inferences Artificial environment
Experimental control Unrepresentative subject pools
Precise measurement External validity
Ability to explore the details of process Experimenter bias
Relative economy
Most scientific research design
Survey experiments
o Cross sectional
o Longitudinal
Advantages Disadvantages
Cohort study: you can look at age, period Possibility of Measurement validity
and cohort effects (using APC models)
Panel study: you can do cross (time) lagged Limited internal validity
causal analysis All (exc. Cross lagged) are correlational
causal inference: internal validity designs and only statistical control
o Experimental designs
o Cross-sectional and longitudinal designs
o Comparative designs
o Historical designs
o = strategy for investigating RQ/hypothesis in coherent and logical way, incl what kind
of data is needed, how the data is collected and what methods of analysis will be used
Methods of data collection
o Questionnaires/surveys
o Interviews/focus groups
o Participant observation/ethnographic research
o Textual/discourse/content analysis
o = requires the def of clear boundaries (time, place, units, factors, variables)
6 main ethical issues
o 1. Voluntary participation
o 2. Informed consent
o 3. privacy
o 4. Harm
o 5. Exploitation
o 6. Consequences for future research
Internal validity
o Causal inferences
o Extent to which we can be confident that IV (causal) produced observed effect
o biggest threats to validity: if you do not design your research in way that enables you
to rule out other, alternative explanations, the internal validity of study = threatened
External validity
o Generalizability, extent to which results from a study can be generalized beyond the
particular study
Reliability
o Refers to the repeatability or consistency of your findings
o Research design is reliable if other researchers can perform exactly same procedures
and come up with same results
o Is a necessary condition for validity
Measurement reliability
o Consistency and precision
o Types: test-retests; internal consistency; intercoder/inter-rater reliability
Measurement validity
o Accuracy
, o Types: face, content, criterion (aka construct validity): concurrent, predictive,
convergent, discriminant
Research design
Experimental design
Experimental designs
o Randomized intervention
o Thought to most closely resemble the true scientific methods
o Most effective design for testing whether or not two variables are casually related
Types
o Laboratory
o Field
o Natural
o Survey
Ethical issues
o Treatment usually concerns concealment and/or deception
o Field studies: real world effects, debriefing not possible
o Tend to be more pronounced in experimental research
o Deception
o Issues of professional conduct
o Publication bias
o Harm: may cause the participants distress
o Must make a risk/gain assessment
Advantages Disadvantages
Ability to derive causal inferences Artificial environment
Experimental control Unrepresentative subject pools
Precise measurement External validity
Ability to explore the details of process Experimenter bias
Relative economy
Most scientific research design
Survey experiments
o Cross sectional
o Longitudinal
Advantages Disadvantages
Cohort study: you can look at age, period Possibility of Measurement validity
and cohort effects (using APC models)
Panel study: you can do cross (time) lagged Limited internal validity
causal analysis All (exc. Cross lagged) are correlational
causal inference: internal validity designs and only statistical control
