Chapter 1
Psychometrics and the Importance of Psychological Measurement
Psychological tests can be life or death:
o Caution must be taken with test administration to avoid misinterpreting or
misusing info derived from psychological tests
Measure both observable things (eg height) and unobservable things (Eg.
time)
Use psychological tests as instruments to measure observable events in
physical world
o Usually, behaviour
Often psychologists use psychological tests to test behaviour they think is
sensitive to underlying psychological attribute eg. depression
Validity- if scores from a measure seem to be actually measuring mental state or
mental process, we think they are measuring
3 things to consider when measuring unobservable (UC) using observable
characteristics:
1. Assume that behaviour observed is a measure of the UC
- Valid score interpretation: if scores from a measure seem to be measuring
the mental state or mental process that we think we’re measuring
2. Task must be theoretically linked to UC for score interpretation to be valid
- Often make inferences from observable behaviour to unobservable
psychological attribute
- Measurement in psychology often involves type of theory linking
psychological characteristics, processes, states, to observable behaviour
that is thought to reflect differences in psychological processes
3. UC itself often a theoretical concept: must assume it is more than a figment of
imagination
Hypothetical constructs (latent variables)- theoretical concepts used to
explain differences in people’s behaviour, which cannot be directly observed
- Operational definitions- operations or procedures used to measure
hypothetical constructs
All sciences rely on unobservable constructs to some degree, and measure them by
measuring observable events or behaviours
(Cronbach definition) Psychological test- systematic procedure for comparing
the behaviour of 2 or more people
3 important components:
1. Tests involve behavioural samples of some kind
2. Behavioural samples must be collected in some systematic way
3. Purpose of tests is to compare behaviours of 2 + people
- Modify this to include comparison of performance by same individuals at
different time points
Evaluation of Cronbach Definition
Generality
Including type of info produced by tests
General purpose of psychological tests must be capable of comparing
interindividual differences (behaviour of different people) or
intraindividual differences (behaviour of same individuals at different
points in time or under different circumstances)
Tests vary in:
- content
o eg. achievement tests, personality tests, attitude surveys, etc.
- response type required
o eg. open-ended, close-ended
, - methods used to administer them
o eg. individually administered, group testing
- intended purpose of test scores:
criterion-referenced- settings where a decision must be made about a
person’s skill level: fixed, predetermined cut-off test score established,
used to sort people into 2 groups 1) exceed performance criterion 2)
doesn’t exceed performance criterion
norm-referenced- compare a person’s test score with scores from
reference sample or normative sample, to see how they compare with
other people
difference between them often blurred: criterion cut-off scores not
determined at random, but based on a standard/expectation
speeded tests- time limited tests, likely all questions answered correctly:
scored by counting number of question answers in time period
power tests- not time limited, expected to answer all test question:
scored by counting number of correct answers on test
range in difficulty if scores from this used to discriminate
among people regarding the attribute of interest
Battery- bundled tests- instruments intended to be administer together but are not
necessarily designed to measure a single psychological attribute
Psychometrics- science concerned with evaluating attributes of psychological tests
Important attributes:
1. type of information generated by use of the tests
2. reliability of data from the tests
3. issues about validity of data obtained using the tests
Challenges to Measurement in Psychometrics (V. Important)
- All measurements are affected by challenges reducing measurement
accuracy
- Challenges in behavioural sciences affect confidence in understanding and
interpretation of behavioural observations
must try identify and capture important aspects of human psychological
attributes in single number
participant reactivity- act of measurement can itself influence
psychological state or process being measured: knowing you’re being
observed causes some to react in ways influencing the interpretation of the
observed behaviour
o Demand characteristics- try to figure out researcher’s purpose
o Social desirability- people become apprehensive, change behaviour
to try to impress person doing the measurement
o Malingering- change their behaviour to convey a poor impression to
the person doing the measure
Compromises validity of the measure
Biases and expectations from people conducting the measure
o Hard to detect
Rely of composite scores when measuring psychological attributes
Composite score- score each question and sum them to create total score
Score sensitivity- ability of a measure to discriminate adequately between
meaningful amounts or units of thee dimension that is being measured
o Exacerbated in psychology as don’t anticipate size of meaningful
differences with the attributes being measured
Lack of awareness of important psychometric information
o Psychological measurement often social/cultural activity
o Therefore often conducted with little/no regard for psychometric
quality of the tests
Psychometrics and the Importance of Psychological Measurement
Psychological tests can be life or death:
o Caution must be taken with test administration to avoid misinterpreting or
misusing info derived from psychological tests
Measure both observable things (eg height) and unobservable things (Eg.
time)
Use psychological tests as instruments to measure observable events in
physical world
o Usually, behaviour
Often psychologists use psychological tests to test behaviour they think is
sensitive to underlying psychological attribute eg. depression
Validity- if scores from a measure seem to be actually measuring mental state or
mental process, we think they are measuring
3 things to consider when measuring unobservable (UC) using observable
characteristics:
1. Assume that behaviour observed is a measure of the UC
- Valid score interpretation: if scores from a measure seem to be measuring
the mental state or mental process that we think we’re measuring
2. Task must be theoretically linked to UC for score interpretation to be valid
- Often make inferences from observable behaviour to unobservable
psychological attribute
- Measurement in psychology often involves type of theory linking
psychological characteristics, processes, states, to observable behaviour
that is thought to reflect differences in psychological processes
3. UC itself often a theoretical concept: must assume it is more than a figment of
imagination
Hypothetical constructs (latent variables)- theoretical concepts used to
explain differences in people’s behaviour, which cannot be directly observed
- Operational definitions- operations or procedures used to measure
hypothetical constructs
All sciences rely on unobservable constructs to some degree, and measure them by
measuring observable events or behaviours
(Cronbach definition) Psychological test- systematic procedure for comparing
the behaviour of 2 or more people
3 important components:
1. Tests involve behavioural samples of some kind
2. Behavioural samples must be collected in some systematic way
3. Purpose of tests is to compare behaviours of 2 + people
- Modify this to include comparison of performance by same individuals at
different time points
Evaluation of Cronbach Definition
Generality
Including type of info produced by tests
General purpose of psychological tests must be capable of comparing
interindividual differences (behaviour of different people) or
intraindividual differences (behaviour of same individuals at different
points in time or under different circumstances)
Tests vary in:
- content
o eg. achievement tests, personality tests, attitude surveys, etc.
- response type required
o eg. open-ended, close-ended
, - methods used to administer them
o eg. individually administered, group testing
- intended purpose of test scores:
criterion-referenced- settings where a decision must be made about a
person’s skill level: fixed, predetermined cut-off test score established,
used to sort people into 2 groups 1) exceed performance criterion 2)
doesn’t exceed performance criterion
norm-referenced- compare a person’s test score with scores from
reference sample or normative sample, to see how they compare with
other people
difference between them often blurred: criterion cut-off scores not
determined at random, but based on a standard/expectation
speeded tests- time limited tests, likely all questions answered correctly:
scored by counting number of question answers in time period
power tests- not time limited, expected to answer all test question:
scored by counting number of correct answers on test
range in difficulty if scores from this used to discriminate
among people regarding the attribute of interest
Battery- bundled tests- instruments intended to be administer together but are not
necessarily designed to measure a single psychological attribute
Psychometrics- science concerned with evaluating attributes of psychological tests
Important attributes:
1. type of information generated by use of the tests
2. reliability of data from the tests
3. issues about validity of data obtained using the tests
Challenges to Measurement in Psychometrics (V. Important)
- All measurements are affected by challenges reducing measurement
accuracy
- Challenges in behavioural sciences affect confidence in understanding and
interpretation of behavioural observations
must try identify and capture important aspects of human psychological
attributes in single number
participant reactivity- act of measurement can itself influence
psychological state or process being measured: knowing you’re being
observed causes some to react in ways influencing the interpretation of the
observed behaviour
o Demand characteristics- try to figure out researcher’s purpose
o Social desirability- people become apprehensive, change behaviour
to try to impress person doing the measurement
o Malingering- change their behaviour to convey a poor impression to
the person doing the measure
Compromises validity of the measure
Biases and expectations from people conducting the measure
o Hard to detect
Rely of composite scores when measuring psychological attributes
Composite score- score each question and sum them to create total score
Score sensitivity- ability of a measure to discriminate adequately between
meaningful amounts or units of thee dimension that is being measured
o Exacerbated in psychology as don’t anticipate size of meaningful
differences with the attributes being measured
Lack of awareness of important psychometric information
o Psychological measurement often social/cultural activity
o Therefore often conducted with little/no regard for psychometric
quality of the tests
