Understanding Variables
➔ construct validitylooks at the extent to which themeasured or manipulated variables
represent the theoretical constructs they’re meant to represent
◆ “construct” = key concept, phenomenon, or attribute
Framework for Assessment
➔ (looks at measurement tools)
➔ design- how are variables defined, designed, andscaled
➔ reliability- is the measurement tool consistent acrossitems, raters, or time periods
➔ validity- does the measurement tool assess what it’ssupposed to assess
➔ conceptual definitions: theoretical constructs ofthe the variable being looked at
◆ “what does variable A mean”
◆ the concept of the variable
◆ constructs becomes variables once they’ve been conceptually defined
➔ after beingconceptually defined, the variable getsoperationally defined
◆ how is the variable measured or manipulated within the context of the study
➔ operationalizing variables
◆ self-report measures:participants report their ownthoughts, feelings, and actions [i.e.
likert scale]
● advantage - easy ,relatively fast, often only way to measure private thoughts,
opinions, and attitudes
● limits
○ response sets: not putting in effort leading to acquiescence (agreeing to
everything) or fence sitting (neutral to everything)
○ socially desirable responding: responding in a way that avoids the
unpopular response: make themselves look good rather than be honest
○ lack of awareness: biased answers due to poor memory, lack of
self-knowledge, or strong emotions; not entirely aware of mental
state/behaviour
● solutions to the limitations
○ reverse-worded/reverse-scored items: alternate between positively and
negatively worded items to detect acquiescence (how happy are you?
how sad are you? → truly answering would mean different answers)
○ socially desirability items: identify “do gooders” by adding questions that
are too good to be true for most [i.e. I never lie]
○ anonymity: concealing identity makes individuals more likely to be
honest
○ question wording: keep survey items brief, relevant, unambiguous,
specific, and objective (BRUSO); reduces cognitive load and increase
likelihood of remembering what’s benign asked
◆ behaviour observation: participants’ behaviour isobserved and recorded
● advantage - provides rich behavioural data, high in ecological validity
, ● limitations
○ observer bias - researcher’s expectations influence interpretations
◆ halo/horns effect: halo giving positive review on the observed
behaviour because positive expectation; horn giving negative
review because negative expectation
○ reactivity - participants change behaviour because aware of being
watched
◆ reactance
○ observer-expectancy effects - expectations of researcher change the
behaviour of the participants; researcher behaves a certain way towards
participants due to hypothesis and that influences the participant to
behave a certain way
● solutions to limitations
○ codebooks - observers are trained based on clear coding procedures to
avoid observer bias
◆ detailed description of how observations are taken
○ unobtrusive observations - researcher makes self less noticeable to avoid
reactance effects
◆ i.e. setting up a camera in advanced to have participant feel
comfortable with the camera
○ masked design - participants and observers are kept unaware of the
hypothesis to avoid expectancy effects
◆ physiological measures: recording any of a wide varietyof physiological processes [i.e.
heart rate and BP, galvanic skin response, hormone levels, and electrical activity and
blood flow in the brain
● advantage - “peak into” fascinating workings of the body and the mind
● limitations
○ lack of specificity - physiological reactions are often linked to multiple
functions and can lack specificity
○ reactivity - physiological reactions can change when stressed, worried,
shy, excited, aroused…all of which can be caused by the knowledge of
being observed; in some cases blood tests may cause anxiety which leads
to stress that could impact the results
○ instrumentation threats - inaccurate measurements if tool is unreliable
or used inconsistently
● solutions to limitations
○ multi-method approach - using multiple methods to measure a
construct to increase specificity
○ multiple measures - taking multiple measurements to let people get used
to the tool prior to recording actual results
○ tool maintenance - maintenance of the measurement tool to ensure
quality and accuracy
➔ construct validitylooks at the extent to which themeasured or manipulated variables
represent the theoretical constructs they’re meant to represent
◆ “construct” = key concept, phenomenon, or attribute
Framework for Assessment
➔ (looks at measurement tools)
➔ design- how are variables defined, designed, andscaled
➔ reliability- is the measurement tool consistent acrossitems, raters, or time periods
➔ validity- does the measurement tool assess what it’ssupposed to assess
➔ conceptual definitions: theoretical constructs ofthe the variable being looked at
◆ “what does variable A mean”
◆ the concept of the variable
◆ constructs becomes variables once they’ve been conceptually defined
➔ after beingconceptually defined, the variable getsoperationally defined
◆ how is the variable measured or manipulated within the context of the study
➔ operationalizing variables
◆ self-report measures:participants report their ownthoughts, feelings, and actions [i.e.
likert scale]
● advantage - easy ,relatively fast, often only way to measure private thoughts,
opinions, and attitudes
● limits
○ response sets: not putting in effort leading to acquiescence (agreeing to
everything) or fence sitting (neutral to everything)
○ socially desirable responding: responding in a way that avoids the
unpopular response: make themselves look good rather than be honest
○ lack of awareness: biased answers due to poor memory, lack of
self-knowledge, or strong emotions; not entirely aware of mental
state/behaviour
● solutions to the limitations
○ reverse-worded/reverse-scored items: alternate between positively and
negatively worded items to detect acquiescence (how happy are you?
how sad are you? → truly answering would mean different answers)
○ socially desirability items: identify “do gooders” by adding questions that
are too good to be true for most [i.e. I never lie]
○ anonymity: concealing identity makes individuals more likely to be
honest
○ question wording: keep survey items brief, relevant, unambiguous,
specific, and objective (BRUSO); reduces cognitive load and increase
likelihood of remembering what’s benign asked
◆ behaviour observation: participants’ behaviour isobserved and recorded
● advantage - provides rich behavioural data, high in ecological validity
, ● limitations
○ observer bias - researcher’s expectations influence interpretations
◆ halo/horns effect: halo giving positive review on the observed
behaviour because positive expectation; horn giving negative
review because negative expectation
○ reactivity - participants change behaviour because aware of being
watched
◆ reactance
○ observer-expectancy effects - expectations of researcher change the
behaviour of the participants; researcher behaves a certain way towards
participants due to hypothesis and that influences the participant to
behave a certain way
● solutions to limitations
○ codebooks - observers are trained based on clear coding procedures to
avoid observer bias
◆ detailed description of how observations are taken
○ unobtrusive observations - researcher makes self less noticeable to avoid
reactance effects
◆ i.e. setting up a camera in advanced to have participant feel
comfortable with the camera
○ masked design - participants and observers are kept unaware of the
hypothesis to avoid expectancy effects
◆ physiological measures: recording any of a wide varietyof physiological processes [i.e.
heart rate and BP, galvanic skin response, hormone levels, and electrical activity and
blood flow in the brain
● advantage - “peak into” fascinating workings of the body and the mind
● limitations
○ lack of specificity - physiological reactions are often linked to multiple
functions and can lack specificity
○ reactivity - physiological reactions can change when stressed, worried,
shy, excited, aroused…all of which can be caused by the knowledge of
being observed; in some cases blood tests may cause anxiety which leads
to stress that could impact the results
○ instrumentation threats - inaccurate measurements if tool is unreliable
or used inconsistently
● solutions to limitations
○ multi-method approach - using multiple methods to measure a
construct to increase specificity
○ multiple measures - taking multiple measurements to let people get used
to the tool prior to recording actual results
○ tool maintenance - maintenance of the measurement tool to ensure
quality and accuracy
