Summary Psychometrics An
Introduction
1 - Psychometrics and the Importance of Psychological Measurement...............................2
3 - Differences, Consistency, and the Meaning of Test Scores...........................................4
Variability and Distributions..........................................................................................4
Quantifying the association between distributions........................................................4
Interpreting test scores................................................................................................5
4 - Test Dimensionality and Factor Analysis.......................................................................6
Test dimensionality......................................................................................................7
Factor Analysis............................................................................................................ 7
5 - Reliability: Conceptual Basis..........................................................................................8
Reliability and Scores (observed, true and error).........................................................8
Four models.................................................................................................................9
6 - Empirical Estimates of Reliability.................................................................................10
Alternate Forms......................................................................................................... 10
Test-Retest................................................................................................................ 11
Internal Consistency..................................................................................................11
Difference scores.......................................................................................................12
7 - The Importance of Reliability.......................................................................................12
Point Estimate and Confidence Interval.....................................................................12
Attenuation................................................................................................................. 13
Item Discrimination....................................................................................................13
, 1 - Psychometrics and the Importance of Psychological
Measurement
Psychological tests are used to make a psychological construct observable. The inference is
that observable behaviour is systematically related to an unobservable construct. If this
inference is true then the measure is valid.
- Construct/latent variable: theoretical psychological characteristics, attributes,
processes, or states that cannot be directly observed, and they include things such
as knowledge, intelligence, self-esteem, attitudes, hunger, memory, personality traits,
depression, and attention
- Operational definitions: operations or procedures used to measure these
constructs
Cronbach definition of psychological test: “a systematic procedure for comparing the
behaviour of two or more people”. This has three components:
1. It involves behavioural samples
2. The samples are collected in a systematic way
3. The purpose is to detect differences between people
Interindividual differences: comparing behaviour of different people
Intraindividual differences: comparing behaviour of the same individuals at different
points in time or under different circumstances
Differences between tests:
Differences Examples
Content Intelligence, personality, etc.
Response required Open-ended:
- answer anything you want
Close-ended:
- choose among alternative answers that are given
Method of administration Individual vs. Group
Use Criterion referenced:
- Seen when a decision is made about the skill level. A criterion
score is established to divide the group in two, people who
score past the criterion and people that don’t. (The cut-off is
predetermined, so it is normed in some sense)
Norm referenced:
- Usually seen when a person is compared to other people. The
person's test score is compared to a reference/normative
sample (representative sample that completed the test).
Timing Speeded tests:
- Time-limited, relatively easy to see how far people get
Power tests:
- Not time-limited, differences in difficulty to see what can be
solved
, The meaning of indicators Reflective/effect indicators:
- Construct causes differences in test scores and the items on
the test are correlated. E.g. scores on intelligence test.
Formative/causal indicators:
- Item responses define the construct and the items
independently contribute to the construct. E.g. defining the
socioeconomic status of a person.
Challenges in Psychological Measurement:
- Complexity of concepts:
Concepts have different aspects to them and it is difficult to identify the important
aspects and change it into a single score.
- Participant reactivity:
People’s responses on a test can be influenced by a number of factors. Demand
characteristics: people can figure out the meaning of the test and change their
responses according to this meaning. Social desirability: people change their
responses to impress the person doing the measurement.
- Observer bias:
Expectation of the researcher can have an effect on the test.
- Composite scores:
Multiple scores from different item need to be merged into one score
- Score sensitivity:
Defining the sensitivity of the scale is challenging. Too few response categories may
miss out on individual differences and too many will make it difficult to distinguish
between differences.
- Awareness:
A lot of people that conduct measurements don’t know about the psychometric
quality of the tests.
Introduction
1 - Psychometrics and the Importance of Psychological Measurement...............................2
3 - Differences, Consistency, and the Meaning of Test Scores...........................................4
Variability and Distributions..........................................................................................4
Quantifying the association between distributions........................................................4
Interpreting test scores................................................................................................5
4 - Test Dimensionality and Factor Analysis.......................................................................6
Test dimensionality......................................................................................................7
Factor Analysis............................................................................................................ 7
5 - Reliability: Conceptual Basis..........................................................................................8
Reliability and Scores (observed, true and error).........................................................8
Four models.................................................................................................................9
6 - Empirical Estimates of Reliability.................................................................................10
Alternate Forms......................................................................................................... 10
Test-Retest................................................................................................................ 11
Internal Consistency..................................................................................................11
Difference scores.......................................................................................................12
7 - The Importance of Reliability.......................................................................................12
Point Estimate and Confidence Interval.....................................................................12
Attenuation................................................................................................................. 13
Item Discrimination....................................................................................................13
, 1 - Psychometrics and the Importance of Psychological
Measurement
Psychological tests are used to make a psychological construct observable. The inference is
that observable behaviour is systematically related to an unobservable construct. If this
inference is true then the measure is valid.
- Construct/latent variable: theoretical psychological characteristics, attributes,
processes, or states that cannot be directly observed, and they include things such
as knowledge, intelligence, self-esteem, attitudes, hunger, memory, personality traits,
depression, and attention
- Operational definitions: operations or procedures used to measure these
constructs
Cronbach definition of psychological test: “a systematic procedure for comparing the
behaviour of two or more people”. This has three components:
1. It involves behavioural samples
2. The samples are collected in a systematic way
3. The purpose is to detect differences between people
Interindividual differences: comparing behaviour of different people
Intraindividual differences: comparing behaviour of the same individuals at different
points in time or under different circumstances
Differences between tests:
Differences Examples
Content Intelligence, personality, etc.
Response required Open-ended:
- answer anything you want
Close-ended:
- choose among alternative answers that are given
Method of administration Individual vs. Group
Use Criterion referenced:
- Seen when a decision is made about the skill level. A criterion
score is established to divide the group in two, people who
score past the criterion and people that don’t. (The cut-off is
predetermined, so it is normed in some sense)
Norm referenced:
- Usually seen when a person is compared to other people. The
person's test score is compared to a reference/normative
sample (representative sample that completed the test).
Timing Speeded tests:
- Time-limited, relatively easy to see how far people get
Power tests:
- Not time-limited, differences in difficulty to see what can be
solved
, The meaning of indicators Reflective/effect indicators:
- Construct causes differences in test scores and the items on
the test are correlated. E.g. scores on intelligence test.
Formative/causal indicators:
- Item responses define the construct and the items
independently contribute to the construct. E.g. defining the
socioeconomic status of a person.
Challenges in Psychological Measurement:
- Complexity of concepts:
Concepts have different aspects to them and it is difficult to identify the important
aspects and change it into a single score.
- Participant reactivity:
People’s responses on a test can be influenced by a number of factors. Demand
characteristics: people can figure out the meaning of the test and change their
responses according to this meaning. Social desirability: people change their
responses to impress the person doing the measurement.
- Observer bias:
Expectation of the researcher can have an effect on the test.
- Composite scores:
Multiple scores from different item need to be merged into one score
- Score sensitivity:
Defining the sensitivity of the scale is challenging. Too few response categories may
miss out on individual differences and too many will make it difficult to distinguish
between differences.
- Awareness:
A lot of people that conduct measurements don’t know about the psychometric
quality of the tests.
