W1.1
CHAPTER 1 (FURR) – PSYCHOMETRICS AND THE IMPORTANCE OF
PSYCHOLOGICAL MEASUREMENT
PSYCHOLOGICAL TESTING
Constructs/latent variables: Theoretical psychological characteristics, attributes, processes, or
states that cannot be directly observed (E.g.: knowledge, intelligence, self-esteem, attitudes,
hunger, personality traits, depression etc.)
Operational definition: The procedure that is used to measure hypothetical constructs –
Translates abstract or theoretical constructs into observable and measurable terms
Psychological test: A systematic procedure for comparing the behaviour of two or more
people (Cronbach) – Very general definition!
1. Behavioural samples
2. Systematic collection: Collecting in a clear and standardised way
3. The purpose of the tests is to detect differences between people
The purpose of measurement in psychology is to identify and quantify interindividual or
intraindividual differences
- Interindividual differences: Behaviour of different people
- Intraindividual differences: Behaviour of the same individual at different points
Types of psychological tests:
- Open-ended tests: People can answer test questions by saying anything they want in
response to the questions on the test
- Closed-ended tests: Require people to answer questions by choosing among
alternative answers provided in the test
- Criterion referenced/domain referenced: A cutoff test score is established as a criterion
and is used to sort people into two groups: a) those whose performance exceeds the
criterion score and b) those whose performance does not (E.g.: driving test)
- Norm referenced: Used to understand how a person compares with other people (E.g.:
IQ test) – Usually done by comparing a person’s test score with scores from a
reference sample or normative sample
- Speeded tests: Time-limited tests – People are not expected to complete the entire test
in the allotted time – Scored by counting the number of questions answered in the
allowed time period – Questions are of comparable difficulty
- Power tests: Not time-limited – Test takers are expected to answer all the test
questions – Scored by counting the number of correct answers made on the test – Test
items range in difficulty
PSYCHOMETRICS
Psychometrics: The science concerned with estimating and evaluating the attributes of tests
1. The type of information generated by the use of psychological tests
2. The reliability of data from psychological tests
3. The validity of data from psychological tests
,History of psychometrics:
- Ancient China: Applicants for governmental positions had to complete various exams
- 19th century: Psychological science emerged and researchers began systematically
measuring various qualities and responses of individuals in experimental studies
- 20th century: The development of early intelligence tests and early personality
inventories
Two key foundations to psychometrics:
1. The practice of psychological testing and measurement
2. The development of particular statistical concepts and procedures
Classical test theory: A body of related psychometric theory that predicts outcomes of
psychological testing such as the difficulty of items or the ability of test-takers – A theory of
testing based on the idea that a person’s observed or obtained score on a test is the sum of a
true score (error-free score) and an error score
Formative (causal) indicators: Observed variables that are
causes, antecedents or drivers of a latent variable
- Items are not necessarily correlated with each other
Reflective (effect) indicators: Observed variables that are
assumed to be affected by a common underlying latent variable
- Items are necessarily correlated with each other
CHALLENGES TO MEASUREMENT
Challenges to measurement in behavioural sciences:
- Complexity of psychological phenomena – Notions such as intelligence, self-esteem,
anxiety, depression etc. have many different aspects to them
- Participant reactivity: For people who are conscious of the fact that they are being
measured, the act of measurement can itself influence the psychological state or
process being measured
- Demand characteristics: When participants (try) to figure out the researcher’s
purpose for a study and change their behaviour to accommodate the researcher
- Social desirability: When people change their behaviour to try to impress the
person doing the measurement
- Malingering: Falsification or exaggeration of illness to gain external benefits
- Observer expectancy bias: When a researcher’s expectations, opinions or prejudices
influence what they perceive or record in a study
- Composite scores: A single variable or data point that represents a combination of
information from multiple variables or data points – Ideally measure one dimension
- Score sensitivity: A measure’s ability to discriminate between meaningful amounts of
the dimension being measured
- Lack of awareness of psychometrics
Differential psychology: The study of individual differences – A misunderstanding is that
psychometrics is only relevant to differential psychology!
, CHAPTER 3 (FURR) – DIFFERENCES, CONSISTENCY AND THE MEANING OF
TEST SCORES
VARIABILITY
Distribution of scores: The set of test scores – Variability: The degree of differences within a
set of test scores or among the values of a psychological attribute
Central tendency: What is the ‘typical’ score in the distribution or what is the score that is
most representative of the entire distribution?
∑x
- Mean: Average score ( x = )
N
number of scores+1
- Median: The middle score when scores are ranked ( )
2
- Mode: The score that occurs the most
Ways to represent variability:
- Deviance/error: The difference between each score and the mean (error =x−x )
- Total deviance: All deviances for each data point added up – Equals zero!
- Sum of squared errors: All squared deviances added up – Indicator of the total
dispersion or ‘fit’ of a model (∑ ( outcome−model )2 =∑ ( x−x )2)
- Variance/mean squared error ( s2): A statistical way of quantifying variability or
individual differences in a distribution or set of scores – Cannot be negative (
∑ of squares =
∑ ( x−x )
2
)
number of observations −1 df
- Standard deviation (sd): Square root of the variance – How well the mean represents
the sample data (i.e. a small standard deviation represents a scenario in which most
√
2
( )
data points are close to the mean) – Cannot be negative ( ∑ x− x )
df
Factors to consider when interpreting variance or standard deviation:
1. Neither one can ever be less than 0
2. There is no simple way to interpret a variance or standard deviation as large or small –
One has to take what the distribution refers to into account
3. The importance of variance and standard deviation lies mainly in their effects on other
values that are more directly interpretable (E.g.: confidence intervals)
Skewed distributions: Asymmetrical distributions that have skew
values below or above 0
- Positive skew: Bulk of the data falls below the mean
- Negative skew: Bulk of the data falls above the mean
COVARIABILITY
Covariance: The degree to which variability in one set of scores
corresponds with variability in another set of scores (E.g.: is there a strong association
CHAPTER 1 (FURR) – PSYCHOMETRICS AND THE IMPORTANCE OF
PSYCHOLOGICAL MEASUREMENT
PSYCHOLOGICAL TESTING
Constructs/latent variables: Theoretical psychological characteristics, attributes, processes, or
states that cannot be directly observed (E.g.: knowledge, intelligence, self-esteem, attitudes,
hunger, personality traits, depression etc.)
Operational definition: The procedure that is used to measure hypothetical constructs –
Translates abstract or theoretical constructs into observable and measurable terms
Psychological test: A systematic procedure for comparing the behaviour of two or more
people (Cronbach) – Very general definition!
1. Behavioural samples
2. Systematic collection: Collecting in a clear and standardised way
3. The purpose of the tests is to detect differences between people
The purpose of measurement in psychology is to identify and quantify interindividual or
intraindividual differences
- Interindividual differences: Behaviour of different people
- Intraindividual differences: Behaviour of the same individual at different points
Types of psychological tests:
- Open-ended tests: People can answer test questions by saying anything they want in
response to the questions on the test
- Closed-ended tests: Require people to answer questions by choosing among
alternative answers provided in the test
- Criterion referenced/domain referenced: A cutoff test score is established as a criterion
and is used to sort people into two groups: a) those whose performance exceeds the
criterion score and b) those whose performance does not (E.g.: driving test)
- Norm referenced: Used to understand how a person compares with other people (E.g.:
IQ test) – Usually done by comparing a person’s test score with scores from a
reference sample or normative sample
- Speeded tests: Time-limited tests – People are not expected to complete the entire test
in the allotted time – Scored by counting the number of questions answered in the
allowed time period – Questions are of comparable difficulty
- Power tests: Not time-limited – Test takers are expected to answer all the test
questions – Scored by counting the number of correct answers made on the test – Test
items range in difficulty
PSYCHOMETRICS
Psychometrics: The science concerned with estimating and evaluating the attributes of tests
1. The type of information generated by the use of psychological tests
2. The reliability of data from psychological tests
3. The validity of data from psychological tests
,History of psychometrics:
- Ancient China: Applicants for governmental positions had to complete various exams
- 19th century: Psychological science emerged and researchers began systematically
measuring various qualities and responses of individuals in experimental studies
- 20th century: The development of early intelligence tests and early personality
inventories
Two key foundations to psychometrics:
1. The practice of psychological testing and measurement
2. The development of particular statistical concepts and procedures
Classical test theory: A body of related psychometric theory that predicts outcomes of
psychological testing such as the difficulty of items or the ability of test-takers – A theory of
testing based on the idea that a person’s observed or obtained score on a test is the sum of a
true score (error-free score) and an error score
Formative (causal) indicators: Observed variables that are
causes, antecedents or drivers of a latent variable
- Items are not necessarily correlated with each other
Reflective (effect) indicators: Observed variables that are
assumed to be affected by a common underlying latent variable
- Items are necessarily correlated with each other
CHALLENGES TO MEASUREMENT
Challenges to measurement in behavioural sciences:
- Complexity of psychological phenomena – Notions such as intelligence, self-esteem,
anxiety, depression etc. have many different aspects to them
- Participant reactivity: For people who are conscious of the fact that they are being
measured, the act of measurement can itself influence the psychological state or
process being measured
- Demand characteristics: When participants (try) to figure out the researcher’s
purpose for a study and change their behaviour to accommodate the researcher
- Social desirability: When people change their behaviour to try to impress the
person doing the measurement
- Malingering: Falsification or exaggeration of illness to gain external benefits
- Observer expectancy bias: When a researcher’s expectations, opinions or prejudices
influence what they perceive or record in a study
- Composite scores: A single variable or data point that represents a combination of
information from multiple variables or data points – Ideally measure one dimension
- Score sensitivity: A measure’s ability to discriminate between meaningful amounts of
the dimension being measured
- Lack of awareness of psychometrics
Differential psychology: The study of individual differences – A misunderstanding is that
psychometrics is only relevant to differential psychology!
, CHAPTER 3 (FURR) – DIFFERENCES, CONSISTENCY AND THE MEANING OF
TEST SCORES
VARIABILITY
Distribution of scores: The set of test scores – Variability: The degree of differences within a
set of test scores or among the values of a psychological attribute
Central tendency: What is the ‘typical’ score in the distribution or what is the score that is
most representative of the entire distribution?
∑x
- Mean: Average score ( x = )
N
number of scores+1
- Median: The middle score when scores are ranked ( )
2
- Mode: The score that occurs the most
Ways to represent variability:
- Deviance/error: The difference between each score and the mean (error =x−x )
- Total deviance: All deviances for each data point added up – Equals zero!
- Sum of squared errors: All squared deviances added up – Indicator of the total
dispersion or ‘fit’ of a model (∑ ( outcome−model )2 =∑ ( x−x )2)
- Variance/mean squared error ( s2): A statistical way of quantifying variability or
individual differences in a distribution or set of scores – Cannot be negative (
∑ of squares =
∑ ( x−x )
2
)
number of observations −1 df
- Standard deviation (sd): Square root of the variance – How well the mean represents
the sample data (i.e. a small standard deviation represents a scenario in which most
√
2
( )
data points are close to the mean) – Cannot be negative ( ∑ x− x )
df
Factors to consider when interpreting variance or standard deviation:
1. Neither one can ever be less than 0
2. There is no simple way to interpret a variance or standard deviation as large or small –
One has to take what the distribution refers to into account
3. The importance of variance and standard deviation lies mainly in their effects on other
values that are more directly interpretable (E.g.: confidence intervals)
Skewed distributions: Asymmetrical distributions that have skew
values below or above 0
- Positive skew: Bulk of the data falls below the mean
- Negative skew: Bulk of the data falls above the mean
COVARIABILITY
Covariance: The degree to which variability in one set of scores
corresponds with variability in another set of scores (E.g.: is there a strong association
