ECP Kazdin Book Exam Summary

EXPERIMENTAL CLINICAL PSYCHOLOGY KAZDIN BOOK
EXAM SUMMARY

Week 1 – Validity
Chapter 2: Internal & External Validity

- Better design  Competing explanations become less plausible

INTERNAL VALIDITY: Does the intervention account for the results?
- To what extend are other possible explanations for the results ruled out?
- Threats (Table 2.2, p.39)
o HISTORY: Events that happen independent of the study, to all participants
 Protect: Control group & Randomization
o MATURATION: Changes resulting from processes within the subjects
 The longer the study, the bigger the impact
 Protect: Control group & Randomization
o TESTING: Change due to repeated assessment
 Protect: Control group & Randomization
o INSTRUMENTATION: Changes in measurement instrument or procedure
 Bigger problem if human observer
 RESPONSE SHIFT: Participants’ internal standards of measurement change (they
alter their definition of what a problem is over time)
 Protect: Standardized, automated assessments
o STATISTICAL REGRESSION: Extreme scores regressing towards the mean at the next
assessment
 Protect: Control group & Randomization; Measurements w/ high validity &
reliability; two pre-tests & select only those who scored extreme both times
o SELECTION BIAS: Differences between groups before the start of the intervention
 Protect: Randomization
o ATTRITION: Loss of subjects changing random composition of the groups
 Prevent drop out: Rewards, close contact
 Statistical methods to control for attrition
o DIFFUSION OF TREATMENT: Participants receiving treatment of the other condition
 SPECIAL TREATMENT: Control group receiving some special treatment (e.g. extra
money because they were in the control group).
 Protect: Monitor conditions

EXTERNAL VALIDITY: Can the results be generalized?
- Threats (Table 2.3, p.53)
o SAMPLE CHARACTERISTICS: Results restricted to sample with specific characteristics
 WEIRD subjects

,  Convenience sample
 Inclusion criteria (e.g. specific disorder)
 Protect: Give reasons for choosing particular sample & select consistent subjects
o NARROW STIMULUS SAMPLING: Results restricted to small range of stimulus
characteristics
 The more specific the stimulus the more factors might change your results
(stimulus could be a picture that’s shown, prayer, or a therapist that’s
administering the intervention)
 Protect: Present at least 2 stimuli that vary slightly in their characteristics
o REACTIVITY OF EXPERIMENTAL ARRANGEMENTS: Results not generalizable to subjects
that are unaware that they’re participating in a study
 Protect: Make the purpose of the study less clear
o REACTIVITY OF ASSESSMENT: Results not generalizable to subjects that are unaware
that they’re being assessed
 Obtrusive assessment: Subjects are aware that their performance is being
assessed
 Reactive measures: Awareness causes different responses
 Protect: Measures where purpose is less clear; test implicit attitudes; biological
measures; different types of assessment (e.g. self-report & direct observations)
o TEST SENSITIZATION: Results not generalizable to subjects that were not previously
assessed
 Pre-test measurement sensitizes subjects to experimental manipulation 
Changes how they react to the manipulation
 Protect: Measures that are less likely to sensitize subjects
o MULTIPLE TREATMENT INTERFERENCE: Results restricted to subjects who have
experienced the same treatments in the same order
 If subjects are exposed to multiple treatments or performed multiple tasks
 Protect: Counterbalancing, cross-over designs
o NOVELTY EFFECTS: Results restricted to new context
 Effects of intervention are due to novelty/innovation
o GENERALITY ACROSS MEASURES, SETTING & TIME: Results restricted to measures,
settings & assessment conditions of the study
 Effects do not transfer to other settings
 Cohort effects: Group at a particular time point, can have different experiences
- External validity not important for proof of concept
- Less important than internal validity!

Chapter 3: Construct & Data Evaluation Validity

CONSTRUCT VALIDITY: Is the presumed cause the actual cause? Interpretive validity
- Confound = third variable that influences the outcome
- Threats (Table 3.1, p.73)

, o ATTENTION & CONTACT: Increase of attention for/contact with the client explains
observed effect
 Protect: Control group also treatment
o SINGLE OPERATIONS/NARROW STIMULUS SAMPLING: Features that are thought to be
irrelevant (set of stimuli or experimenter) could cause the observed effect
 Protect: More than 1 stimulus, experimenter, etc.
o EXPERIMENTER EXPECTANCIES: Expectancies of the experimenter influence their
behavior which in turn influences the subjects’ responses and so cause the observed
effect
 Protect: Experimenters blinded; provide all experimenters with standard
statement of expectations; measure their expectations with a questionnaire &
correlate results with outcome
o DEMAND CHARACTERISTICS: Secondary incidental cues of the experiment can promote
certain behavior of the subjects and so cause the observed effect
 E.g. information conveyed to prospective subjects, instructions, procedures
 Protect (Table 3.2, p.81):
 Post experimental inquiry: ask for ideas about purpose of the study in
the end  If ideas in line with expectations = problem
 Pre-inquiry: tell subjects about procedures and see how they would
perform  If performance in line with expectations = problem
 Simulators: Subject act like they already received procedures & try to
deceive experimenters  If able to deceive (experimenters think they
actually received the treatment) = problem

DATA EVALUATION VALIDITY: Do the data & analyses demonstrate what is stated?
- Concepts (Table 3.3, p.84):
o ALPHA: Type I error, false negative, rejecting null-hypothesis when true
o BETA: Type II error, false positive, accepting null-hypothesis when false
o POWER: 1- beta, true negative, rejecting null-hypothesis when false
o EFFECT SIZE: Difference between conditions
o STANDARD DEVIATION: Variability about a mean
- Threats (Table 3.4, p.86)
o LOW POWER: = harder to detect differences
 Protect: Larger sample size & effect size; Compute power in advance
o SUBJECT HETEROGENEIETY: Increased variability  harder to detect differences
 Protect:
 Specify characteristics, inclusion & exclusion criteria for subjects
 Statistically evaluate characteristics that may have an effect
o VARIABILITY IN PROCEDURES: Inconsistent procedures increase variability
 Protect:
 Explicit & standardized procedures
 Train all experimenters together
 Encourage experimenters to report when they deviate
o UNRELIABILITY OF MEASURES: Errors in measurement increase variability