Course 2.5
PSYCHOMETRICS
Meeting 1
Chapter 1
Psychometrics & the Importance of Psychological Measurement
Psychometrics
= principles which test whether a test produces scores that are psychologically meaningful & trustworthy
- science concerned w/ evaluating the attributes of psychological tests
1) the type of info (in most cases, scores) generated by the use of psychological tests
2) the reliability of data from psychological tests
3) issues concerning validity of data obtained from tests
- Galton: founding father of modern psychometrics
- validity
- measurement in psychology often involves some type of theory linking psychological characteristics, processes, or
states to an observable behaviour that is thought to reflect differences in the psychological attribute
- operations or procedures used to measure hypothetical constructs (= psychological characteristics, attributes,
processes, or states that can’t be directly observed) are called operational definitions
Psychological Tests
= a systematic procedure for comparing the behaviour of two or more people
1) tests involve behavioural samples of some kind (all kinds of behaviour)
2) the behavioural samples must be collected in some systematic way (standardized conditions)
3) the purpose of the tests is to compare the behaviours of 2 or more people (intra- or interindividual differences)
- tests must be capable of comparing the behaviour of different people (interindividual differences) or the behaviour of
the same individuals @ different points in time or under different circumstances (intraindividual differences)
= contribute to test score variability (= necessary component of an attempt to measure any psychological
attribute)
Types of Test
- are categorized as either
- criterion referenced (aka domain referenced)
= most often seen in settings in which a decision must be made about a person’s skill level
- a fixed, predetermined cut-off test score is established & sorts people into two groups 1) those
1
, whose performance exceeds the performance criterion & 2) those whose performance doesn’t
- representative of some well-defined population
- norm referenced
= usually used to compare a person’s test score w/ scores from a reference sample or a normative
sample – to understand how the person compares w/ other people
- Speeded tests
= time-limited tests
- are scored by counting the number of questions answered in the allotted time period
- difficulty of questions is the same
- Power tests
= not time limited – expected to answer all questions
- items must range in difficulty
- Performance vs. Behaviour
Psychometrics
= the science concerned w/ evaluating the attributes of psychological
tests
-> Type of Information (scores)
-> Reliability
-> Validity
Challenges to Measurement
- Identify & Capture human psychological attributes in a single number
- Participant reactivity (= the act of measurement can itself influence the psychological state or process being
measured)
- people’s knowledge that they’re being observed can cause them to react in ways that obscure the
interpretation of the behaviour that is being observed
1) demand characteristics (= trying to figure out the researcher’s purpose for a study – changing their
behaviour to accommodate the researcher)
2) social desirability (= some people might become apprehensive, others might change their behaviour to try
to impress the person doing the measurement)
3) malingering (= change behaviour to convey a poor impression doing the measurement)
=> validity of the measure is compromised – the person’s true psychological characteristic is obscured by a temporary
motivation/state that is a reaction to the very act of being measured
- Objectivity = Expectation & Bias effects (= people collecting the behavioural data can bring biases & expectations to
their task)
- Composite Scores (= doesn’t always make sense to add all scores together)
- Score sensitivity (= the ability of a measure to discriminate adequately between meaningful amounts/units of the
dimension that is being measured)
- Lack of Awareness of Important Psychometric Info
Chapter 2
Scaling
“Measurement is the assignment of numerals to objects or events according to rules”
- Scaling (= concerns the way numerical values are assigned to psychological attributes)
- numerals can represent psychological attributes in different way, depending on the nature of the numeral that is
used to represent the attribute
2
,Property of Identity
- individuals within a category should be the same as each other’s in terms of sharing a psychological feature, but they
should be different from the individuals in another category – objects/events can be sorted into categories that are
based on similarity of features
-> categories must be mutually exclusive, exhaustive & each person within the category must satisfy the
property of identity
- numerals only convey info about whether two individuals are similar or different but nothing more
Property of Order
- convey info about the relative amount of an attribute that people possess
- they indicate the rank order of people relative to each other along some dimension
- convey no info about the exact amounts of that attribute
Property of Quantity
- provide info about the magnitude of differences between people
- reflect real numbers
Number 0
- reflects a state in which an attribute of an object/event has no existence
- or an arbitrary quantity of an attribute (e.g.: temperature)
= three properties of numerals & the meaning of zero are fundamental issues that shape our understanding of
psychological test scores
Units of Measurement
- Arbitrariness (= distinguishes between different kinds of measurement units)
1) unit size can be arbitrary – specific size of a unit
2) there might be no inherent restriction on the objects to which a unit of measurement might be applied
3) some units of measurement can be used to measure different features of objects
Additivity
- requires unit size to remain constant – one unit increase @ one point in the measurement process must be the same
as a unit increase @ any other point
- unit size of measurement doesn’t change - all units being counted are identical
Scales of Measurement
- Nominal (= numerals that have the property of identity are used to label observations in which behaviours haven
sorted into categories according to some psychological attribute)
- Ordinal (= defines measurement in terms of numerals that have the property of order)
- Interval & Ratio(= are based on numbers that represent quantitative differences between people in terms of the
attribute being measured -> difference on the meaning of zero – ratio scales have an absolute zero, interval scale an
arbitrary zero (temperature etc))
3
, Chapter 3
Individual Differences & Correlations
Variability & Distribution of Scores
- Central Tendency (= typical score in the distribution/ the score that is most representative of the entire distribution)
- mean most common
- Variability
- variance & standard deviation most common ones = reflect variability as the degree to which the score in a
distribution deviate from the mean of the distribution
-> the standard deviation has the advantage of reflecting variability in terms of the size of raw deviation scores
(sometimes viewed as more intuitive), whereas the variance reflects variability in terms of squared deviation
scores
Quantifying the Association Between Distributions
- Covariability (= the degree to which two distributions of scores vary in corresponding manner)
- Direction of the Association
= do people who obtain relatively high scores on one variable tend to obtain relatively high scores on the other
-> positive / direct association
- Magnitude of the Association
= are two variables very strongly associated with each other or only weakly?
4
PSYCHOMETRICS
Meeting 1
Chapter 1
Psychometrics & the Importance of Psychological Measurement
Psychometrics
= principles which test whether a test produces scores that are psychologically meaningful & trustworthy
- science concerned w/ evaluating the attributes of psychological tests
1) the type of info (in most cases, scores) generated by the use of psychological tests
2) the reliability of data from psychological tests
3) issues concerning validity of data obtained from tests
- Galton: founding father of modern psychometrics
- validity
- measurement in psychology often involves some type of theory linking psychological characteristics, processes, or
states to an observable behaviour that is thought to reflect differences in the psychological attribute
- operations or procedures used to measure hypothetical constructs (= psychological characteristics, attributes,
processes, or states that can’t be directly observed) are called operational definitions
Psychological Tests
= a systematic procedure for comparing the behaviour of two or more people
1) tests involve behavioural samples of some kind (all kinds of behaviour)
2) the behavioural samples must be collected in some systematic way (standardized conditions)
3) the purpose of the tests is to compare the behaviours of 2 or more people (intra- or interindividual differences)
- tests must be capable of comparing the behaviour of different people (interindividual differences) or the behaviour of
the same individuals @ different points in time or under different circumstances (intraindividual differences)
= contribute to test score variability (= necessary component of an attempt to measure any psychological
attribute)
Types of Test
- are categorized as either
- criterion referenced (aka domain referenced)
= most often seen in settings in which a decision must be made about a person’s skill level
- a fixed, predetermined cut-off test score is established & sorts people into two groups 1) those
1
, whose performance exceeds the performance criterion & 2) those whose performance doesn’t
- representative of some well-defined population
- norm referenced
= usually used to compare a person’s test score w/ scores from a reference sample or a normative
sample – to understand how the person compares w/ other people
- Speeded tests
= time-limited tests
- are scored by counting the number of questions answered in the allotted time period
- difficulty of questions is the same
- Power tests
= not time limited – expected to answer all questions
- items must range in difficulty
- Performance vs. Behaviour
Psychometrics
= the science concerned w/ evaluating the attributes of psychological
tests
-> Type of Information (scores)
-> Reliability
-> Validity
Challenges to Measurement
- Identify & Capture human psychological attributes in a single number
- Participant reactivity (= the act of measurement can itself influence the psychological state or process being
measured)
- people’s knowledge that they’re being observed can cause them to react in ways that obscure the
interpretation of the behaviour that is being observed
1) demand characteristics (= trying to figure out the researcher’s purpose for a study – changing their
behaviour to accommodate the researcher)
2) social desirability (= some people might become apprehensive, others might change their behaviour to try
to impress the person doing the measurement)
3) malingering (= change behaviour to convey a poor impression doing the measurement)
=> validity of the measure is compromised – the person’s true psychological characteristic is obscured by a temporary
motivation/state that is a reaction to the very act of being measured
- Objectivity = Expectation & Bias effects (= people collecting the behavioural data can bring biases & expectations to
their task)
- Composite Scores (= doesn’t always make sense to add all scores together)
- Score sensitivity (= the ability of a measure to discriminate adequately between meaningful amounts/units of the
dimension that is being measured)
- Lack of Awareness of Important Psychometric Info
Chapter 2
Scaling
“Measurement is the assignment of numerals to objects or events according to rules”
- Scaling (= concerns the way numerical values are assigned to psychological attributes)
- numerals can represent psychological attributes in different way, depending on the nature of the numeral that is
used to represent the attribute
2
,Property of Identity
- individuals within a category should be the same as each other’s in terms of sharing a psychological feature, but they
should be different from the individuals in another category – objects/events can be sorted into categories that are
based on similarity of features
-> categories must be mutually exclusive, exhaustive & each person within the category must satisfy the
property of identity
- numerals only convey info about whether two individuals are similar or different but nothing more
Property of Order
- convey info about the relative amount of an attribute that people possess
- they indicate the rank order of people relative to each other along some dimension
- convey no info about the exact amounts of that attribute
Property of Quantity
- provide info about the magnitude of differences between people
- reflect real numbers
Number 0
- reflects a state in which an attribute of an object/event has no existence
- or an arbitrary quantity of an attribute (e.g.: temperature)
= three properties of numerals & the meaning of zero are fundamental issues that shape our understanding of
psychological test scores
Units of Measurement
- Arbitrariness (= distinguishes between different kinds of measurement units)
1) unit size can be arbitrary – specific size of a unit
2) there might be no inherent restriction on the objects to which a unit of measurement might be applied
3) some units of measurement can be used to measure different features of objects
Additivity
- requires unit size to remain constant – one unit increase @ one point in the measurement process must be the same
as a unit increase @ any other point
- unit size of measurement doesn’t change - all units being counted are identical
Scales of Measurement
- Nominal (= numerals that have the property of identity are used to label observations in which behaviours haven
sorted into categories according to some psychological attribute)
- Ordinal (= defines measurement in terms of numerals that have the property of order)
- Interval & Ratio(= are based on numbers that represent quantitative differences between people in terms of the
attribute being measured -> difference on the meaning of zero – ratio scales have an absolute zero, interval scale an
arbitrary zero (temperature etc))
3
, Chapter 3
Individual Differences & Correlations
Variability & Distribution of Scores
- Central Tendency (= typical score in the distribution/ the score that is most representative of the entire distribution)
- mean most common
- Variability
- variance & standard deviation most common ones = reflect variability as the degree to which the score in a
distribution deviate from the mean of the distribution
-> the standard deviation has the advantage of reflecting variability in terms of the size of raw deviation scores
(sometimes viewed as more intuitive), whereas the variance reflects variability in terms of squared deviation
scores
Quantifying the Association Between Distributions
- Covariability (= the degree to which two distributions of scores vary in corresponding manner)
- Direction of the Association
= do people who obtain relatively high scores on one variable tend to obtain relatively high scores on the other
-> positive / direct association
- Magnitude of the Association
= are two variables very strongly associated with each other or only weakly?
4
