Can Results Generalize to Other Populations?
● If we wish to generalize a set of results to all of humanity, the best—but completely impractical
—method for doing so would be to randomly select a sample from the earth’s entire population.
In contrast, most psychological research relies on convenience samples, people selected simply
because they are available and willing
Beyond University Students
● undergraduate students tend to be young and to possess the characteristics of late adolescence: a
developing sense of self-identity, social and political attitudes that are in a state of flux, a high
need for peer approval, and unstable peer relationships. They are also intelligent, have good
cognitive skills, and know how to win approval from authority
● Researchers at the University of British Columbia have described this group as WEIRD: Western,
Educated, Industrialized . As a result of being WEIRD, the conclusions drawn from these
participants seem unlikely to generalize to all of humanity.
Beyond Volunteers
● At many universities, introductory psychology students are often required to volunteer for
experiments for course credit. Recruitment for these populations might involve asking parents
visiting a science center to participate in a study on parenting, or asking users of a particular
Internet forum to complete a survey online.Volunteers tend to be more highly educated, more in
need of approval, and more social
● Another factor that can influence who signs up for a study is whether some financial
compensation is offered. Studies that recruit participants by emphasizing financial rewards tend
to attract less-altruistic participants than studies emphasizing the potential learning
opportunity.Similarly, studies offering course credit for participating tend to attract less motivated
participants than studies that do not offer course credit , although offering either money or course
credit boosts volunteer rates more than offering nothing.
Beyond the Gender of Participants
● Sometimes, researchers use either mostly females or males, or only one gender, simply because
this is convenient or the procedures seem better suited to one gender. Similarly, individuals who
identify as a gender minority might be excluded from analysis
● In psychological research with humans, gender could influence results and subsequently the
conclusions drawn from these results. Denmark and colleagues have identified several ways that
gender bias may arise throughout the research process. If a sample is solely or predominantly one
gender, it is best to avoid concluding that a widely generalizable truth has been found.
● It is also possible, when a sample includes a good balance of genders, to include gender as a
variable in the analyses to investigate if the results generalize across the genders studied.
Beyond Culture
● Today, however, many samples of university students are ethnically diverse because the
population of university students across North America has become increasingly diverse. In
addition, more and more psychological research is being done in countries around the world. As a
result, the overall external validity of research has improved. It is also now much easier to
compare ethnic groups, to examine cross-cultural differences and similarities
● So far, much of cultural research has centered on identifying cross-cultural similarities and
differences in responses to the same environments, along with personality and other
characteristics
, ● Once again, students with a collectivist cultural background reported more parental involvement
in their choice of romantic partner than students with an individualist cultural background
(Cohen’s d = 1.42). This type of research informs us about the generality of effects across cultural
groups.
Can Results Generalize beyond the Specific Study Situation?
Beyond the Experimenter
● The person who actually conducts the experiment can trigger another generalization problem. In
some studies, only one experimenter is used to reduce variability in how the experimenter
influences participants. Because little attention is typically paid to the personal characteristics of
experimenters , it is possible that the results of a study using only one experimenter cannot be
generalized to other types of experimenters. Some of the important characteristics of
experimenters include personality, gender, and amount of practice in the role of an experimenter
A warm, friendly experimenter may produce different results than a cold, unfriendly
experimenter. Participants are also more productive and cooperative when experimenters are
dressed in accordance with stereotyped gender roles
● One solution to the problem of generalizing to other experimenters is to use two or more
experimenters, with differing characteristics .Another option is to deliver instructions using a
computer, which minimizes the amount of interaction between experimenters and participants,
thereby reducing the potential for influence
Beyond a Pretest
● Researchers must often decide whether to give a pretest . Intuitively, pretesting seems to be a
good idea. The researcher can examine whether groups are equivalent on the pretest, and
sometimes it is imps. In longitudinal studies that have the risk of participants withdrawing from
the study, a pretest allows uortant to examine changes in people’s scores from pretest to posttest,
rather than simply comparing posttest scores to look for any effects of selective attrition. A
pretest lets us determine whether the people who withdrew from the study were different from
those who completed it.
● Pretesting, however, may limit the ability to generalize any results to populations that do not
receive a pretest. In the real world, people are rarely given a pretest. For example, people do not
regularly take stock of their attitudes before listening to a political speech or viewing an
advertisement
● In the Solomon four group design, the same experiment is conducted with and without the
pretest. The researcher can then examine whether there is an interaction between the independent
variable and the pretest variable. If the pretest has no effect, posttest scores on the dependent
variable are the same regardless of whether or not the pretest was given.
Beyond the Laboratory
● Research conducted in a laboratory setting has the advantage of allowing the experimenter to
study variables under highly controlled conditions. In experiments, the goal of high internal
validity may sometimes conflict with the goal of external validity
● Field experiments are one way that researchers try to examine phenomena under more realistic
circumstances, and thereby increase the external validity of their experiments (Chapter 4). In a
field experiment, the researcher manipulates the independent variable in some natural setting, like
a factory, a school
● If we wish to generalize a set of results to all of humanity, the best—but completely impractical
—method for doing so would be to randomly select a sample from the earth’s entire population.
In contrast, most psychological research relies on convenience samples, people selected simply
because they are available and willing
Beyond University Students
● undergraduate students tend to be young and to possess the characteristics of late adolescence: a
developing sense of self-identity, social and political attitudes that are in a state of flux, a high
need for peer approval, and unstable peer relationships. They are also intelligent, have good
cognitive skills, and know how to win approval from authority
● Researchers at the University of British Columbia have described this group as WEIRD: Western,
Educated, Industrialized . As a result of being WEIRD, the conclusions drawn from these
participants seem unlikely to generalize to all of humanity.
Beyond Volunteers
● At many universities, introductory psychology students are often required to volunteer for
experiments for course credit. Recruitment for these populations might involve asking parents
visiting a science center to participate in a study on parenting, or asking users of a particular
Internet forum to complete a survey online.Volunteers tend to be more highly educated, more in
need of approval, and more social
● Another factor that can influence who signs up for a study is whether some financial
compensation is offered. Studies that recruit participants by emphasizing financial rewards tend
to attract less-altruistic participants than studies emphasizing the potential learning
opportunity.Similarly, studies offering course credit for participating tend to attract less motivated
participants than studies that do not offer course credit , although offering either money or course
credit boosts volunteer rates more than offering nothing.
Beyond the Gender of Participants
● Sometimes, researchers use either mostly females or males, or only one gender, simply because
this is convenient or the procedures seem better suited to one gender. Similarly, individuals who
identify as a gender minority might be excluded from analysis
● In psychological research with humans, gender could influence results and subsequently the
conclusions drawn from these results. Denmark and colleagues have identified several ways that
gender bias may arise throughout the research process. If a sample is solely or predominantly one
gender, it is best to avoid concluding that a widely generalizable truth has been found.
● It is also possible, when a sample includes a good balance of genders, to include gender as a
variable in the analyses to investigate if the results generalize across the genders studied.
Beyond Culture
● Today, however, many samples of university students are ethnically diverse because the
population of university students across North America has become increasingly diverse. In
addition, more and more psychological research is being done in countries around the world. As a
result, the overall external validity of research has improved. It is also now much easier to
compare ethnic groups, to examine cross-cultural differences and similarities
● So far, much of cultural research has centered on identifying cross-cultural similarities and
differences in responses to the same environments, along with personality and other
characteristics
, ● Once again, students with a collectivist cultural background reported more parental involvement
in their choice of romantic partner than students with an individualist cultural background
(Cohen’s d = 1.42). This type of research informs us about the generality of effects across cultural
groups.
Can Results Generalize beyond the Specific Study Situation?
Beyond the Experimenter
● The person who actually conducts the experiment can trigger another generalization problem. In
some studies, only one experimenter is used to reduce variability in how the experimenter
influences participants. Because little attention is typically paid to the personal characteristics of
experimenters , it is possible that the results of a study using only one experimenter cannot be
generalized to other types of experimenters. Some of the important characteristics of
experimenters include personality, gender, and amount of practice in the role of an experimenter
A warm, friendly experimenter may produce different results than a cold, unfriendly
experimenter. Participants are also more productive and cooperative when experimenters are
dressed in accordance with stereotyped gender roles
● One solution to the problem of generalizing to other experimenters is to use two or more
experimenters, with differing characteristics .Another option is to deliver instructions using a
computer, which minimizes the amount of interaction between experimenters and participants,
thereby reducing the potential for influence
Beyond a Pretest
● Researchers must often decide whether to give a pretest . Intuitively, pretesting seems to be a
good idea. The researcher can examine whether groups are equivalent on the pretest, and
sometimes it is imps. In longitudinal studies that have the risk of participants withdrawing from
the study, a pretest allows uortant to examine changes in people’s scores from pretest to posttest,
rather than simply comparing posttest scores to look for any effects of selective attrition. A
pretest lets us determine whether the people who withdrew from the study were different from
those who completed it.
● Pretesting, however, may limit the ability to generalize any results to populations that do not
receive a pretest. In the real world, people are rarely given a pretest. For example, people do not
regularly take stock of their attitudes before listening to a political speech or viewing an
advertisement
● In the Solomon four group design, the same experiment is conducted with and without the
pretest. The researcher can then examine whether there is an interaction between the independent
variable and the pretest variable. If the pretest has no effect, posttest scores on the dependent
variable are the same regardless of whether or not the pretest was given.
Beyond the Laboratory
● Research conducted in a laboratory setting has the advantage of allowing the experimenter to
study variables under highly controlled conditions. In experiments, the goal of high internal
validity may sometimes conflict with the goal of external validity
● Field experiments are one way that researchers try to examine phenomena under more realistic
circumstances, and thereby increase the external validity of their experiments (Chapter 4). In a
field experiment, the researcher manipulates the independent variable in some natural setting, like
a factory, a school
