measurement/data collection - Answers - Data must be measured accurately and consistently
- KEY to overall quality of the evidence
- KEY to our level of confidence in the findings
- IMPORTANT ELEMENT IN "internal validity" of the study (confidence that findings are primarily
r/t the impact of the IV on DV)
- GOAL: Unbiased, accurate, consistent data
- Measurement Error
*Random error is always present
*Systematic error can be guarded against
primary data - Answers - Collected by the researcher directly
- Allows for more control of procedures, quality
secondary data - Answers - Data extracted from that previously collected
- Convenient
- No control over quality (only how extracted, used)
internal validity - Answers - Can I have confidence in the study results?
- Do results accurately represent the impact of the independent variable on the dependent - AS
FREE FROM ERROR AS POSSIBLE ?
external validity - Answers - How "generalizable" are the results?
- To what population beyond subjects do the results apply?
common threats to internal validity - Answers - History (impact of events/ time): extraneous
variable
- Maturation (subjects change)
- Instrumentation (quality of data collected)
- Hawthorne Effect (testing effect on subjects): not "real"
- Mortality (loss of subjects): decreases representativeness
, - Sampling (i.e. "selection bias")
*Sampling BIAS occurs with (nonprobablity) sampling AND/OR NO random assignment to
groups
**All increase the potential for error
how to support internal validity - Answers - Research design:
*Use design with greatest control over error (Experimental)
*Control Groups
*Randomization to groups (double blind even better)
- Sampling:
*Use homogeneous sampling (subjects similar to each other)
*Use probability sampling techniques (increases randomization)
- Use of data collection procedures that are:
*Uniform
*Consistent
*Replicable
*Also: Document alternative explanations for findings not a real threat (anticipate critics
skepticism)
- Statistical analysis:
*Tests the NULL HYPOTHESIS
*Determine the probability of a TYPE I error
threats to external validity - Answers - Dependent on the nature of the study question/variables
(sometimes important questions only impact a specific group of people)
- Nonprobability Sampling (esp. convenience):
*No assurance of representativeness related to larger population of interest; greatly limits
external validity
- Homogeneity of sample:
*Increases internal validity - BUT - level of external validity (generalizability) is lessened