9 Survey Errors
, I. Conceptualization error (from properties to conceptual model)
= constructs do not map on properties
When there are no specified relationships between constructs, but “just questions”, it leads to
conceptualization error.
Validity = Accuracy – Low bias
Reliability = Replicability – Low variance
Conceptualization = Process of translating the research question into its underlying key constructs and
their hypothesized relationships
II. Operationalization error (from conceptual model to measurement/instrument)
= systematic departure from the constructs (invalid measure / bad scale)
- Asking the right questions with the right response options contributes to survey validity by
reducing operationalization error and eventually measurement error.
- Using scales with known reliability reduces operationalization and measurement error as well
Operationalization = the process of selecting and/or developing measurements for the constructs and
attributes in the conceptual model.
III. Coverage error
= when sampling frame ≠ population
Sampling = the procedure by which some members of a given population are selected as representatives
of the entire population.
Online surveys have representation issues and therefore lead to coverage error, sampling error, and
nonresponse error.
How can we prevent coverage error?
- Professional sampling services
o Market research companies with dedicated databases
o Specialized list brokers and profilers
o Crowdsourced platforms
- Multiple frames (for difficult to reach populations)
How can we cope with coverage error?
Post-stratification = weighing the responses of the elements in the final sample based on their socio-
demographic and other characteristics to make them resemble the profile of the target population.
Because the weighing is done after the data have been collected, it is called post-stratification
IV. Sampling error (from sampling frame to sample)
= deliberate error because of cost or feasibility.
- Accidental goals of sampling are unmet and uncorrected (mistakes)
- Deliberate cluster sampling adds sampling error for cost effectiveness, and statistically
corrects for the error. You draw a lot of different samples and hope together it forms a
, I. Conceptualization error (from properties to conceptual model)
= constructs do not map on properties
When there are no specified relationships between constructs, but “just questions”, it leads to
conceptualization error.
Validity = Accuracy – Low bias
Reliability = Replicability – Low variance
Conceptualization = Process of translating the research question into its underlying key constructs and
their hypothesized relationships
II. Operationalization error (from conceptual model to measurement/instrument)
= systematic departure from the constructs (invalid measure / bad scale)
- Asking the right questions with the right response options contributes to survey validity by
reducing operationalization error and eventually measurement error.
- Using scales with known reliability reduces operationalization and measurement error as well
Operationalization = the process of selecting and/or developing measurements for the constructs and
attributes in the conceptual model.
III. Coverage error
= when sampling frame ≠ population
Sampling = the procedure by which some members of a given population are selected as representatives
of the entire population.
Online surveys have representation issues and therefore lead to coverage error, sampling error, and
nonresponse error.
How can we prevent coverage error?
- Professional sampling services
o Market research companies with dedicated databases
o Specialized list brokers and profilers
o Crowdsourced platforms
- Multiple frames (for difficult to reach populations)
How can we cope with coverage error?
Post-stratification = weighing the responses of the elements in the final sample based on their socio-
demographic and other characteristics to make them resemble the profile of the target population.
Because the weighing is done after the data have been collected, it is called post-stratification
IV. Sampling error (from sampling frame to sample)
= deliberate error because of cost or feasibility.
- Accidental goals of sampling are unmet and uncorrected (mistakes)
- Deliberate cluster sampling adds sampling error for cost effectiveness, and statistically
corrects for the error. You draw a lot of different samples and hope together it forms a
