Week 2:
Reading 1: Goodwin chapter 5 (2007) – Introduction to experimental research: (p. 193-230)
-> Essential features of an experiment (independent + extraneous + dependent variables)
-> How the validity of a study can be affected by how well it is designed
• ESSENTIAL FEATURES OF EXPERIMENTAL RESEARCH:
o Experiment = independent variables (under control) + extraneous
(constant) + dependent (behaviour measured)
o John Stuart Mill -> inductive logic rules
§ Method of agreement = ‘if X, then Y’ (= experimental group)
• E.g. child watching violent TV -> aggressive
§ Method of differences = ‘if not X, then not Y’ (= control group)
• E.g. child not watching violent TV -> not aggressive
§ -> Will never know for sure but can to experiments with a certain
amount of confidence
• Establishing independent variables:
o Independent variable = the one being studied (will it influence behaviour?)
§ Minimum of 2 levels (i.e. compare 2 situations)
o Field experiment = taken in the field
§ Field research = broader – any research outside the lab (experimental or
not)
o Varieties of independent variables:
§ Situational = features in the environment participants might
encounter
• E.g. how many bystanders affect helping someone or not
§ Task = can vary the types of task performed
• E.g. different optical illusions
§ Instructional = ask different groups to perform a task in different
ways
• E.g. different instructions on how to memorize a list
§ à Possible to combine different types
o Control groups:
§ Experimental group = those who undergo the experiment
§ Control group = don’t do the experiment
• Not necessary in all research
§ “Joint Method” = simple experiment with both groups
• Controlling extraneous variables:
o Extraneous variable = any variables that are not of interest but might
influence the behaviour being studied
§ Failure to control this = can influence measure systematically
§ E.g. only use males in all conditions
o Confound = uncontrolled extraneous variable that ‘covaries’ with the
independent
§ It changes at the same time as the indep (= can’t separate the two)
§ E.g. use only males in one condition & only females in another
1
,• Measuring dependent variables:
o Dependent variable = behaviour being influenced by independent variable
§ Needs to be defined precisely (otherwise replication impossible)
§ Best to use dependent variables that have been used in prior studies
• Shown to be reliable & valid
o Needing to develop a new measure -> should start with pilot study to avoid:
§ Ceiling effect = when average scores for different groups are so high
that no difference can be determined
• Happens when dependent variable is so easy that everyone
gets high score
• Can look at response time
§ Floor effect = when all scores are extremely low -> can’t differentiate
between the groups – happens when test is too difficult
• Can’t do much about it
o Keep control!!
o à Constructs (e.g. anxiety) can be any of these 3 types of variable
• MANIPULATED VS SUBJECT VARIABLES:
o Subject variable = already existing characteristics in participants
§ E.g. sex, age, SES, mental disorder…
§ Don’t manipulate this -> select people from both groups (have &
don’t)
o “True” experiment = with manipulated variables
§ ≠ Quasi experiment/ex post facto/ natural group studies = subject
variable
o à Studies often combine both
• Drawing conclusions when using subject variables:
o (!!!) Can’t draw causal conclusions with subject variables
o Little control -> other aspects of each participant may differ (e.g. self-
confidence)
o Can’t say difference between groups was caused by subject variables
§ Can only say the 2 groups performed differently
o Causal conclusions can be drawn in manipulated samples
• Example of the Bobo doll:
o Independent =
§ Manipulated -> seeing aggression (cartoon, person, control)
§ Non manipulated -> sex; sex congruent vs incongruent with
experimenter
o Extraneous =
§ Avoided size of doll effect (smaller doll for kids)
§ All mildly frustrated (toys taken away from them) – same emotional
arousal
o Dependent =
§ Aggressive behaviour (imitative or not)
2
,• THE VALIDITY OF EXPERIMENTAL RESEARCH:
• Statistical conclusion validity:
o Statistical conclusion validity = extent to which the researcher uses
statistics properly – draws appropriate conclusions from them
• Construct validity:
o Does that test measure what it intends to
o Construct validity (in experimental research) = adequacy of definition of
both independent & dependent variables
• External validity:
o External validity = degree to which findings generalize to larger population,
environment, time (e.g. not only undergraduates)
o Other populations:
§ 75% of published research in 1980 was done on undergraduates
• Often kinda forced into participating
§ WEIRD population
§ Culture -> should generalize to all cultures when only studied
Western
§ Gender -> lot of studies only use males
• E.g. Kohlberg’s theory of moral development
§ Guidelines for recruiting students:
o Other environment:
§ Do results apply to real life settings
§ Ecological validity = research with relevance for everyday cognitive
activities
§ Should make more studies out of lab
o Other times:
§ Historical events can impact results of past research
3
, • E.g. probably for anxiety in war times or economic crises
(covid)
o à External validity of some research can increase with new research
confirming it (e.g. finding out about false memory – now applied to
eyewitness testimony)
• Internal validity:
o Internal validity = degree to which an experiment is methodologically
sound & confound free
o Confidence results are measured by dependent variable + directly associated
with independent variable + not result of other uncontrolled factor
• THREATS TO INTERNAL VALIDITY:
o Uncontrolled extraneous factors
§ Especially when no control group
o Often happens in longitudinal studies when several measures are taken (pre
& post test)
• Studies extending over a period of time:
o Pretest & posttest -> evaluate participants before & after experience
§ Pretest -> treatment (or control) -> posttest
o History & maturation:
§ History = when an event occurs between pre & posttest that impacts
results
• E.g. change in grading policy can lead to great decreased anxiety
§ Maturation = getting used to what is being tested
• E.g. students get used to college life (reduced test anxiety)
§ à Use of control group can confirm effective results (if same pre &
posttest)
o Regression: (p.221 – 195 of book)
§ Regression to the mean = more likely to pick scores close to the
mean than outliers
§ Need control group of same extremity as testing group
• E.g. testing test anxiety (should keep high anxiety control
group)
• “The improvement could be a regression effect” -> if only 1
measure
• Might get more extreme result the 1st time à 2nd time will be
closer to average
o Testing & instrumentation:
§ Testing = when taking a pretest has an effect on posttest
• E.g. remember some items of pretest
§ Instrumentation = problem when there are changes in
measurements
• E.g. if posttest is easier than pretest
§ à Control group can help detect this
4
Reading 1: Goodwin chapter 5 (2007) – Introduction to experimental research: (p. 193-230)
-> Essential features of an experiment (independent + extraneous + dependent variables)
-> How the validity of a study can be affected by how well it is designed
• ESSENTIAL FEATURES OF EXPERIMENTAL RESEARCH:
o Experiment = independent variables (under control) + extraneous
(constant) + dependent (behaviour measured)
o John Stuart Mill -> inductive logic rules
§ Method of agreement = ‘if X, then Y’ (= experimental group)
• E.g. child watching violent TV -> aggressive
§ Method of differences = ‘if not X, then not Y’ (= control group)
• E.g. child not watching violent TV -> not aggressive
§ -> Will never know for sure but can to experiments with a certain
amount of confidence
• Establishing independent variables:
o Independent variable = the one being studied (will it influence behaviour?)
§ Minimum of 2 levels (i.e. compare 2 situations)
o Field experiment = taken in the field
§ Field research = broader – any research outside the lab (experimental or
not)
o Varieties of independent variables:
§ Situational = features in the environment participants might
encounter
• E.g. how many bystanders affect helping someone or not
§ Task = can vary the types of task performed
• E.g. different optical illusions
§ Instructional = ask different groups to perform a task in different
ways
• E.g. different instructions on how to memorize a list
§ à Possible to combine different types
o Control groups:
§ Experimental group = those who undergo the experiment
§ Control group = don’t do the experiment
• Not necessary in all research
§ “Joint Method” = simple experiment with both groups
• Controlling extraneous variables:
o Extraneous variable = any variables that are not of interest but might
influence the behaviour being studied
§ Failure to control this = can influence measure systematically
§ E.g. only use males in all conditions
o Confound = uncontrolled extraneous variable that ‘covaries’ with the
independent
§ It changes at the same time as the indep (= can’t separate the two)
§ E.g. use only males in one condition & only females in another
1
,• Measuring dependent variables:
o Dependent variable = behaviour being influenced by independent variable
§ Needs to be defined precisely (otherwise replication impossible)
§ Best to use dependent variables that have been used in prior studies
• Shown to be reliable & valid
o Needing to develop a new measure -> should start with pilot study to avoid:
§ Ceiling effect = when average scores for different groups are so high
that no difference can be determined
• Happens when dependent variable is so easy that everyone
gets high score
• Can look at response time
§ Floor effect = when all scores are extremely low -> can’t differentiate
between the groups – happens when test is too difficult
• Can’t do much about it
o Keep control!!
o à Constructs (e.g. anxiety) can be any of these 3 types of variable
• MANIPULATED VS SUBJECT VARIABLES:
o Subject variable = already existing characteristics in participants
§ E.g. sex, age, SES, mental disorder…
§ Don’t manipulate this -> select people from both groups (have &
don’t)
o “True” experiment = with manipulated variables
§ ≠ Quasi experiment/ex post facto/ natural group studies = subject
variable
o à Studies often combine both
• Drawing conclusions when using subject variables:
o (!!!) Can’t draw causal conclusions with subject variables
o Little control -> other aspects of each participant may differ (e.g. self-
confidence)
o Can’t say difference between groups was caused by subject variables
§ Can only say the 2 groups performed differently
o Causal conclusions can be drawn in manipulated samples
• Example of the Bobo doll:
o Independent =
§ Manipulated -> seeing aggression (cartoon, person, control)
§ Non manipulated -> sex; sex congruent vs incongruent with
experimenter
o Extraneous =
§ Avoided size of doll effect (smaller doll for kids)
§ All mildly frustrated (toys taken away from them) – same emotional
arousal
o Dependent =
§ Aggressive behaviour (imitative or not)
2
,• THE VALIDITY OF EXPERIMENTAL RESEARCH:
• Statistical conclusion validity:
o Statistical conclusion validity = extent to which the researcher uses
statistics properly – draws appropriate conclusions from them
• Construct validity:
o Does that test measure what it intends to
o Construct validity (in experimental research) = adequacy of definition of
both independent & dependent variables
• External validity:
o External validity = degree to which findings generalize to larger population,
environment, time (e.g. not only undergraduates)
o Other populations:
§ 75% of published research in 1980 was done on undergraduates
• Often kinda forced into participating
§ WEIRD population
§ Culture -> should generalize to all cultures when only studied
Western
§ Gender -> lot of studies only use males
• E.g. Kohlberg’s theory of moral development
§ Guidelines for recruiting students:
o Other environment:
§ Do results apply to real life settings
§ Ecological validity = research with relevance for everyday cognitive
activities
§ Should make more studies out of lab
o Other times:
§ Historical events can impact results of past research
3
, • E.g. probably for anxiety in war times or economic crises
(covid)
o à External validity of some research can increase with new research
confirming it (e.g. finding out about false memory – now applied to
eyewitness testimony)
• Internal validity:
o Internal validity = degree to which an experiment is methodologically
sound & confound free
o Confidence results are measured by dependent variable + directly associated
with independent variable + not result of other uncontrolled factor
• THREATS TO INTERNAL VALIDITY:
o Uncontrolled extraneous factors
§ Especially when no control group
o Often happens in longitudinal studies when several measures are taken (pre
& post test)
• Studies extending over a period of time:
o Pretest & posttest -> evaluate participants before & after experience
§ Pretest -> treatment (or control) -> posttest
o History & maturation:
§ History = when an event occurs between pre & posttest that impacts
results
• E.g. change in grading policy can lead to great decreased anxiety
§ Maturation = getting used to what is being tested
• E.g. students get used to college life (reduced test anxiety)
§ à Use of control group can confirm effective results (if same pre &
posttest)
o Regression: (p.221 – 195 of book)
§ Regression to the mean = more likely to pick scores close to the
mean than outliers
§ Need control group of same extremity as testing group
• E.g. testing test anxiety (should keep high anxiety control
group)
• “The improvement could be a regression effect” -> if only 1
measure
• Might get more extreme result the 1st time à 2nd time will be
closer to average
o Testing & instrumentation:
§ Testing = when taking a pretest has an effect on posttest
• E.g. remember some items of pretest
§ Instrumentation = problem when there are changes in
measurements
• E.g. if posttest is easier than pretest
§ à Control group can help detect this
4
