Research Methods in Social Relations ch. 7 Measurement and
Reliability – Maruyama & Ryan (2014)
Concepts (abstract ideas like love or intelligence) are measured by variables, which must have at
least two values. Variables are representations of constructs, measured by multiple operational
definitions, making them objective but not identical to the construct itself (definitional
operationism).
Measurement begins by defining constructs theoretically and linking them to related ideas (the
nomological net), which guides how to measure them.
Survey Design: The wording, content, and order of questions affect how well responses capture the
true construct (construct validity). Errors can arise from response biases, misinterpretations, or
memory issues like memory telescoping (recalling events as more recent). Specific questions yield
more accurate responses than general ones, and sociometric questions assess group interactions.
Behavioural questions should focus on specific, concrete events to improve recall.
Design surveys with enough questions to fully understand responses, avoiding unnecessary details.
Use clear, specific terms and numerical ranges; avoid vague or emotionally charged words. Handle
sensitive topics carefully for valid responses. For exact replication, use the same wording as in the
original study. Adapt questions (conceptual replication) for culturally different groups while
measuring the same concept.
Replication: Exact replication uses identical wording from previous studies, while conceptual
replication adapts questions for different cultural contexts while maintaining the same underlying
construct.
Avoid double-barrelled questions, which mix multiple issues in one question. Forced-choice formats,
where respondents select from given options, are often more valid than agree/disagree formats,
which can be influenced by acquiescence response style—the tendency to agree with statements
regardless of their content. Handle sensitive topics carefully by using neutral, non-judgmental
language to encourage honesty and reduce discomfort. This sensitivity helps respondents feel secure
in providing accurate, valid responses, especially on personal or socially delicate issues.
Open-ended questions allow detailed responses but are harder to analyze. Closed-ended questions
are easier to analyze but may lack depth. Combining both yields rich data. Closed-ended questions
should offer clear, non-overlapping, and inclusive response options, often using numerical ranges and
an "other" option for unlisted answers.
The funnel principle suggests starting with broad, general questions and gradually moving to more
specific ones. This method reduces bias by helping respondents ease into the topic before answering
more detailed questions. To avoid bias further, split-ballot experiments can be used. In these,
different groups receive different versions of a questionnaire to detect how variations in wording or
structure affect responses.
Multiple-item scales are more reliable and valid than single items, as they reduce item-specific errors
like misinterpretation or fatigue. Combining items averages out these errors for a more accurate
measure.
Multiple-item scales are more reliable and valid than single items because they reduce item-specific
errors like misinterpretation or fatigue. By averaging responses, these errors tend to cancel out,
Reliability – Maruyama & Ryan (2014)
Concepts (abstract ideas like love or intelligence) are measured by variables, which must have at
least two values. Variables are representations of constructs, measured by multiple operational
definitions, making them objective but not identical to the construct itself (definitional
operationism).
Measurement begins by defining constructs theoretically and linking them to related ideas (the
nomological net), which guides how to measure them.
Survey Design: The wording, content, and order of questions affect how well responses capture the
true construct (construct validity). Errors can arise from response biases, misinterpretations, or
memory issues like memory telescoping (recalling events as more recent). Specific questions yield
more accurate responses than general ones, and sociometric questions assess group interactions.
Behavioural questions should focus on specific, concrete events to improve recall.
Design surveys with enough questions to fully understand responses, avoiding unnecessary details.
Use clear, specific terms and numerical ranges; avoid vague or emotionally charged words. Handle
sensitive topics carefully for valid responses. For exact replication, use the same wording as in the
original study. Adapt questions (conceptual replication) for culturally different groups while
measuring the same concept.
Replication: Exact replication uses identical wording from previous studies, while conceptual
replication adapts questions for different cultural contexts while maintaining the same underlying
construct.
Avoid double-barrelled questions, which mix multiple issues in one question. Forced-choice formats,
where respondents select from given options, are often more valid than agree/disagree formats,
which can be influenced by acquiescence response style—the tendency to agree with statements
regardless of their content. Handle sensitive topics carefully by using neutral, non-judgmental
language to encourage honesty and reduce discomfort. This sensitivity helps respondents feel secure
in providing accurate, valid responses, especially on personal or socially delicate issues.
Open-ended questions allow detailed responses but are harder to analyze. Closed-ended questions
are easier to analyze but may lack depth. Combining both yields rich data. Closed-ended questions
should offer clear, non-overlapping, and inclusive response options, often using numerical ranges and
an "other" option for unlisted answers.
The funnel principle suggests starting with broad, general questions and gradually moving to more
specific ones. This method reduces bias by helping respondents ease into the topic before answering
more detailed questions. To avoid bias further, split-ballot experiments can be used. In these,
different groups receive different versions of a questionnaire to detect how variations in wording or
structure affect responses.
Multiple-item scales are more reliable and valid than single items, as they reduce item-specific errors
like misinterpretation or fatigue. Combining items averages out these errors for a more accurate
measure.
Multiple-item scales are more reliable and valid than single items because they reduce item-specific
errors like misinterpretation or fatigue. By averaging responses, these errors tend to cancel out,
