Wright, K. B. (2005)
Researching Internet-based populations: Advantages and disadvantages of online survey research,
online questionnaire authoring software packages, and Web survey services. Read at least the first
part, up to "Current Web Survey Software Packages and Online Survey-Related Services"
This article considers and evaluates the advantages and disadvantages related to conducting online
surveys identified in previous research.
Advantages of Online Survey Research
1. Access to unique populations
One advantage of online survey research is that it takes advantage of the ability of the Internet to
provide access to groups and individuals who would be difficult, if not impossible, to reach through
other channels. One advantage of virtual communities as sites for research is that they offer a
mechanism through which a researcher can gain access to people who share specific interests,
attitudes, beliefs, and values regarding an issue, problem, or activity. For example, researchers can
find a concentrated number of older individuals who use computers on the Internet-based
community SeniorNet.
Internet enables communication among people who may be hesitant to meet face-to-face. For
example, individuals with unpopular political views may hesitate to express themselves openly, and
groups of individuals such as Arab-Americans may feel uncomfortable talking about anti-Arab
sentiment in public places. Also, individuals with stigmatized diseases or conditions (HIV, eating
disorders, physical disabilities) are easier to reach online. These individuals and groups often can be
reached on the Internet in larger numbers than would be possible using face-to-face research
methods
2. Time
A second advantage is that Internet-based survey research may save time for researchers. As already
noted, online surveys allow a researcher to reach thousands of people with common characteristics
in a short amount of time, despite possibly being separated by great geographic distances.
Online surveys may also save time by allowing researchers to collect data while they work on other
tasks. Once an invitation to participate in a survey is posted to the website of a community of
interest, emailed to people through a listserv service, or distributed through an online survey
research service, researchers may collect data while working on other projects.
3. Costs
Online survey researchers can also save money by moving to an electronic medium from a paper
format. Paper surveys tend to be costly, even when using a relatively small sample, and the costs of a
traditional large-scale survey using mailed questionnaires can be enormous. The use of online
surveys circumvents this problem by eliminating the need for paper and other costs, such as those
incurred through postage, printing, and data entry.
Other costs that are saved: costs for recording equipment, travel, telephone, transcription costs.
Disadvantages Associated with Online Survey Research
1. Sampling Issues
Relatively little may be known about the characteristics of people in online communities, aside from
some basic demographic variables, and even this information may be questionable (because it is self-
reported).
1
,Generating Samples from Virtual Groups and Organizations
Some virtual groups and organizations provide membership email lists that can help researchers
establish a sampling frame. However, not all members of virtual groups and organizations allow their
email addresses to be listed, and some may not allow administrators to provide their email addresses
to researchers. This makes accurately sizing an online population difficult. Problems such as multiple
email addresses for the same person, multiple responses from participants, and invalid/inactive
email addresses make random sampling online a problematic method in many circumstances.
One solution is for researchers to require participants to contact them to obtain a unique
code number (and a place to include this code number on the online questionnaire) prior to
completing a survey. However, requiring this extra step may significantly reduce the
response rate.
Another solution that some newer web survey programs offer is response tracking.
Participants are required to submit their email address in order to complete the survey. Once
they have completed the survey, the survey program remembers the participant's email
address and does not allow anyone using that email address access to the survey.
Generating a Sample from an Online Community
It’s difficult to establish a sampling frame when research in an online community, because:
Online communities do not typically provide participant email addresses
Ebb and flow of communication some are ‘regulars’ (daily), others are ‘lurkers’ (reading
but not posting). Lurkers do not make their presence known to the group, which makes it
difficult to obtain an accurate sampling frame.
Individuals can easily lie about any information they report to community administrators,
there is no guarantee of accuracy.
Other Sampling Concerns
Non-response rate tracking is difficult to ascertain in most large online communities -> One
relatively inexpensive technique used by market researchers to increase response rates is to
offer some type of financial incentive, e.g., a lottery.
Self-selection bias is another major limitation of online survey research -> there is a tendency
of some individuals to respond to an invitation to participate in an online survey, while
others ignore it, leading to a systematic bias.
These sampling issues inhibit researchers' ability to make generalizations about study
findings. This, in turn, limits their ability to estimate population parameters, which presents
the greatest threat to conducting probability research.
2. Access issues
Some researchers access potential participants by posting invitations to participate in a survey on
community bulletin boards, discussion groups, and chat rooms. However, members of online
communities often find this behavior rude or offensive or consider this type of posting to be “spam”.
Although accessing some online communities can be extremely challenging, seeking permission from
the community and taking time to explain the purpose of the study might help a researcher to gain
access. Nonetheless, it may take a long time before receiving a response to a request, and
community sponsors may reject the researcher's request despite his or her attempts to convey the
possible benefits of the study for the community.
Researchers can foster “good will” between themselves and community participants by offering to
provide information about the results of their study to the community. It is important for researchers
to include contact information, information about the study, and something about their credentials
when creating an invitation to participate in a survey.
2
, Checkoway, H., Pearce, N., & Kriebel, D. (2007)
Selecting appropriate study designs to address specific research questions in occupational
epidemiology.
Basic principles of study design
Clearly define the source population, the study base
The new occurrence of disease, the incidence, is the basic measure of disease occurrence
that epidemiologists seek to estimate
Study design options
Conventional study designs
Cohort studies
The cohort design entails follow!up of a population and determination of the subsequent incidence
of health outcomes.
Prospective short-term phenomena, in which the temporal relation between exposure
and risk is relatively short. The span is e.g. a single work shift, a work-week or years.
Historical (retrospective) practical alternative. Investigating diseases with long induction
and latency periods (mortality, chronic disease).
So: A study design where one or more samples (called cohorts) are followed prospectively and
subsequent status evaluations with respect to a disease or outcome are conducted to determine
which initial participants exposure characteristics (risk factors) are associated with it.
Cross-sectional studies
The cross!sectional design involves comparisons of disease prevalence among exposed and non !
exposed groups, or among groups classified according to exposure type and level. Subject selection is
usually based on exposure status.
Disadvantage: such studies may be prone to a “reverse causation” bias—that is, the exposure
status may be an effect of the disease rather than a cause. This could occur, for example, if a
worker changed departments (from a more dusty to a less dusty job) or left employment as a
result of developing respiratory disease.
Repeated measures studies
Design in which exposures and health status are determined at a baseline time point, and re!
assessed throughout a period of follow!up. Repeated measures studies share the identical design as
prospective cohort studies. The distinction between the two is generally the nature of the health
outcomes studied with these approaches: disease incidence or mortality in prospective cohort
studies; and disease symptoms and physiological parameter changes in repeated measures studies.
3
Researching Internet-based populations: Advantages and disadvantages of online survey research,
online questionnaire authoring software packages, and Web survey services. Read at least the first
part, up to "Current Web Survey Software Packages and Online Survey-Related Services"
This article considers and evaluates the advantages and disadvantages related to conducting online
surveys identified in previous research.
Advantages of Online Survey Research
1. Access to unique populations
One advantage of online survey research is that it takes advantage of the ability of the Internet to
provide access to groups and individuals who would be difficult, if not impossible, to reach through
other channels. One advantage of virtual communities as sites for research is that they offer a
mechanism through which a researcher can gain access to people who share specific interests,
attitudes, beliefs, and values regarding an issue, problem, or activity. For example, researchers can
find a concentrated number of older individuals who use computers on the Internet-based
community SeniorNet.
Internet enables communication among people who may be hesitant to meet face-to-face. For
example, individuals with unpopular political views may hesitate to express themselves openly, and
groups of individuals such as Arab-Americans may feel uncomfortable talking about anti-Arab
sentiment in public places. Also, individuals with stigmatized diseases or conditions (HIV, eating
disorders, physical disabilities) are easier to reach online. These individuals and groups often can be
reached on the Internet in larger numbers than would be possible using face-to-face research
methods
2. Time
A second advantage is that Internet-based survey research may save time for researchers. As already
noted, online surveys allow a researcher to reach thousands of people with common characteristics
in a short amount of time, despite possibly being separated by great geographic distances.
Online surveys may also save time by allowing researchers to collect data while they work on other
tasks. Once an invitation to participate in a survey is posted to the website of a community of
interest, emailed to people through a listserv service, or distributed through an online survey
research service, researchers may collect data while working on other projects.
3. Costs
Online survey researchers can also save money by moving to an electronic medium from a paper
format. Paper surveys tend to be costly, even when using a relatively small sample, and the costs of a
traditional large-scale survey using mailed questionnaires can be enormous. The use of online
surveys circumvents this problem by eliminating the need for paper and other costs, such as those
incurred through postage, printing, and data entry.
Other costs that are saved: costs for recording equipment, travel, telephone, transcription costs.
Disadvantages Associated with Online Survey Research
1. Sampling Issues
Relatively little may be known about the characteristics of people in online communities, aside from
some basic demographic variables, and even this information may be questionable (because it is self-
reported).
1
,Generating Samples from Virtual Groups and Organizations
Some virtual groups and organizations provide membership email lists that can help researchers
establish a sampling frame. However, not all members of virtual groups and organizations allow their
email addresses to be listed, and some may not allow administrators to provide their email addresses
to researchers. This makes accurately sizing an online population difficult. Problems such as multiple
email addresses for the same person, multiple responses from participants, and invalid/inactive
email addresses make random sampling online a problematic method in many circumstances.
One solution is for researchers to require participants to contact them to obtain a unique
code number (and a place to include this code number on the online questionnaire) prior to
completing a survey. However, requiring this extra step may significantly reduce the
response rate.
Another solution that some newer web survey programs offer is response tracking.
Participants are required to submit their email address in order to complete the survey. Once
they have completed the survey, the survey program remembers the participant's email
address and does not allow anyone using that email address access to the survey.
Generating a Sample from an Online Community
It’s difficult to establish a sampling frame when research in an online community, because:
Online communities do not typically provide participant email addresses
Ebb and flow of communication some are ‘regulars’ (daily), others are ‘lurkers’ (reading
but not posting). Lurkers do not make their presence known to the group, which makes it
difficult to obtain an accurate sampling frame.
Individuals can easily lie about any information they report to community administrators,
there is no guarantee of accuracy.
Other Sampling Concerns
Non-response rate tracking is difficult to ascertain in most large online communities -> One
relatively inexpensive technique used by market researchers to increase response rates is to
offer some type of financial incentive, e.g., a lottery.
Self-selection bias is another major limitation of online survey research -> there is a tendency
of some individuals to respond to an invitation to participate in an online survey, while
others ignore it, leading to a systematic bias.
These sampling issues inhibit researchers' ability to make generalizations about study
findings. This, in turn, limits their ability to estimate population parameters, which presents
the greatest threat to conducting probability research.
2. Access issues
Some researchers access potential participants by posting invitations to participate in a survey on
community bulletin boards, discussion groups, and chat rooms. However, members of online
communities often find this behavior rude or offensive or consider this type of posting to be “spam”.
Although accessing some online communities can be extremely challenging, seeking permission from
the community and taking time to explain the purpose of the study might help a researcher to gain
access. Nonetheless, it may take a long time before receiving a response to a request, and
community sponsors may reject the researcher's request despite his or her attempts to convey the
possible benefits of the study for the community.
Researchers can foster “good will” between themselves and community participants by offering to
provide information about the results of their study to the community. It is important for researchers
to include contact information, information about the study, and something about their credentials
when creating an invitation to participate in a survey.
2
, Checkoway, H., Pearce, N., & Kriebel, D. (2007)
Selecting appropriate study designs to address specific research questions in occupational
epidemiology.
Basic principles of study design
Clearly define the source population, the study base
The new occurrence of disease, the incidence, is the basic measure of disease occurrence
that epidemiologists seek to estimate
Study design options
Conventional study designs
Cohort studies
The cohort design entails follow!up of a population and determination of the subsequent incidence
of health outcomes.
Prospective short-term phenomena, in which the temporal relation between exposure
and risk is relatively short. The span is e.g. a single work shift, a work-week or years.
Historical (retrospective) practical alternative. Investigating diseases with long induction
and latency periods (mortality, chronic disease).
So: A study design where one or more samples (called cohorts) are followed prospectively and
subsequent status evaluations with respect to a disease or outcome are conducted to determine
which initial participants exposure characteristics (risk factors) are associated with it.
Cross-sectional studies
The cross!sectional design involves comparisons of disease prevalence among exposed and non !
exposed groups, or among groups classified according to exposure type and level. Subject selection is
usually based on exposure status.
Disadvantage: such studies may be prone to a “reverse causation” bias—that is, the exposure
status may be an effect of the disease rather than a cause. This could occur, for example, if a
worker changed departments (from a more dusty to a less dusty job) or left employment as a
result of developing respiratory disease.
Repeated measures studies
Design in which exposures and health status are determined at a baseline time point, and re!
assessed throughout a period of follow!up. Repeated measures studies share the identical design as
prospective cohort studies. The distinction between the two is generally the nature of the health
outcomes studied with these approaches: disease incidence or mortality in prospective cohort
studies; and disease symptoms and physiological parameter changes in repeated measures studies.
3
