PYC4807: Introduction to Psychological Assessment in the South African context
(Oxford).
Fifth Edition.
This contains notes/summaries on Theme 1: Psychometric Theory, Topic 1.1.
Psychometric Properties needed for Assignment 1 (2022).
Chapter 3 (3.6): Norms:
Norm – measurement against which a person’s raw score is evaluated so that the
individual’s position relative to that of the normative sample can be determined.
Some measures – norm-referenced – test-taker’s performance is compared to a sample of
their peers (normative sample).
measure student performance against a fixed set of predetermined criteria or learning
standards
Other measures – criterion-referenced – test-taker’s performance is compared to a fixed set
of predetermined criteria (attainment of a defined skill or content).
3.6.1 The standard normal distribution:
Constantly referred to in psychometrics is the standard normal (bell-shaped) distribution.
Mean of 0 & Standard deviation (SD) of 1.
The raw scores obtained by test-takers on measures have little to no meaning. These scores
are converted to normal (standardised) scores through statistical transformation to make it
meaningful.
Procedure for establishing normative samples/norm groups – very important in any
selection procedure.
, 3.6.2 Establishing norm groups:
Norm groups have 2 subgroups – the applicant pool (those not yet competent) & the
incumbent population (those already competent).
The norm group chosen should be representative of BOTH the applicant pool and the
incumbent population.
Comparative aspects (ethnicity, gender, age, educational background) are used to establish
similarities between the norm group and the applicant and incumbent groups.
Norm group should be representative of these two groups.
The norm group completes the entire test battery. Standardised/normal scores (discussed
below) are calculated for each of the tests in the battery. New applicants’ scores can now be
compared to these newly established (normal) scores.
3.6.3 Co-norming of measures:
The process where two or more related but different measures are administered and
standardised as a unit on the same norm group.
Can address a range of potential issues – test-order effects (between such measures):
- learning and memory effects
- gender
- age & education effects, and
- variations of scaling format effects.
Co-norming may yield many practical benefits where the same norm group is used for
comparable, but different measures.
, 3.6.4 Types of test norms:
Developmental scales:
Rationale for developmental scales is that certain human characteristics increase
progressively with increases in age and experience.
Mental age scales:
A basal age is computed – the highest age at and below which a measure was passed.
Child’s mental age on a measure is the sum of the basal age plus additional months of
credit earned at higher age levels.
Child with a mental age of 10 corresponds to the mental development of the average 10-
year-old, no matter their chronological age.
Grade equivalence:
Scores on educational achievement measures are often interpreted in terms of grade
equivalents.
Drawbacks of this scale are that scale units are not necessarily equal and that they
represent median performance with overlapping from one grade to another.
Percentiles:
Frequently used for describing individual test performance.
Percentile rank score – the % of people in a normative standardisation sample who fall at or
below a given raw score.
Example:
A person obtains a percentile score of 70. This means that 70 per cent of the normative
population obtained a raw score lower than the individual.
50th percentile corresponds to the median.
, 25th & 75th percentile are the first (Q1) and third (Q3) quartiles, because they cut off the
lowest and highest quarters of the normal distribution.
Percentile rank scores can be obtained from frequency tables & conversion tables.
These tables estimate the placement of raw scores in the score distribution by relying on a
raw score conversion to standard scores (z-scores).
Percentiles ARE NOT percentages (%).
Percentages (%) - raw scores expressed in terms of percentage correct answers.
Percentiles - derived scores expressed in terms of percentage of persons surpassing a
specific raw score.
Disadvantage of percentiles:
- inequality of scale units (especially under extreme ends of the distribution).
- Percentile marks are ordinal level measures, can't be used for normal arithmetic
calculations.
- Percentile ranks calculated for various variables & on various populations are not
directly comparable.
Standard scores:
Can be classified as z-scores, linearly transformed z-scores, and normalised standard scores.
Z-scores:
Expresses an individual’s distance from the mean in standard deviation units.
The z-score for a particular raw score is easily calculated by subtracting the mean from it
and dividing the result by the standard deviation (s).
A raw score equal to the mean is equal to a z-score of 0.
Positive z-scores - above-average performance.
Negative z-scores - below-average performance.
Z-scores routinely range between approximately -3.0 to +3.0.
(Oxford).
Fifth Edition.
This contains notes/summaries on Theme 1: Psychometric Theory, Topic 1.1.
Psychometric Properties needed for Assignment 1 (2022).
Chapter 3 (3.6): Norms:
Norm – measurement against which a person’s raw score is evaluated so that the
individual’s position relative to that of the normative sample can be determined.
Some measures – norm-referenced – test-taker’s performance is compared to a sample of
their peers (normative sample).
measure student performance against a fixed set of predetermined criteria or learning
standards
Other measures – criterion-referenced – test-taker’s performance is compared to a fixed set
of predetermined criteria (attainment of a defined skill or content).
3.6.1 The standard normal distribution:
Constantly referred to in psychometrics is the standard normal (bell-shaped) distribution.
Mean of 0 & Standard deviation (SD) of 1.
The raw scores obtained by test-takers on measures have little to no meaning. These scores
are converted to normal (standardised) scores through statistical transformation to make it
meaningful.
Procedure for establishing normative samples/norm groups – very important in any
selection procedure.
, 3.6.2 Establishing norm groups:
Norm groups have 2 subgroups – the applicant pool (those not yet competent) & the
incumbent population (those already competent).
The norm group chosen should be representative of BOTH the applicant pool and the
incumbent population.
Comparative aspects (ethnicity, gender, age, educational background) are used to establish
similarities between the norm group and the applicant and incumbent groups.
Norm group should be representative of these two groups.
The norm group completes the entire test battery. Standardised/normal scores (discussed
below) are calculated for each of the tests in the battery. New applicants’ scores can now be
compared to these newly established (normal) scores.
3.6.3 Co-norming of measures:
The process where two or more related but different measures are administered and
standardised as a unit on the same norm group.
Can address a range of potential issues – test-order effects (between such measures):
- learning and memory effects
- gender
- age & education effects, and
- variations of scaling format effects.
Co-norming may yield many practical benefits where the same norm group is used for
comparable, but different measures.
, 3.6.4 Types of test norms:
Developmental scales:
Rationale for developmental scales is that certain human characteristics increase
progressively with increases in age and experience.
Mental age scales:
A basal age is computed – the highest age at and below which a measure was passed.
Child’s mental age on a measure is the sum of the basal age plus additional months of
credit earned at higher age levels.
Child with a mental age of 10 corresponds to the mental development of the average 10-
year-old, no matter their chronological age.
Grade equivalence:
Scores on educational achievement measures are often interpreted in terms of grade
equivalents.
Drawbacks of this scale are that scale units are not necessarily equal and that they
represent median performance with overlapping from one grade to another.
Percentiles:
Frequently used for describing individual test performance.
Percentile rank score – the % of people in a normative standardisation sample who fall at or
below a given raw score.
Example:
A person obtains a percentile score of 70. This means that 70 per cent of the normative
population obtained a raw score lower than the individual.
50th percentile corresponds to the median.
, 25th & 75th percentile are the first (Q1) and third (Q3) quartiles, because they cut off the
lowest and highest quarters of the normal distribution.
Percentile rank scores can be obtained from frequency tables & conversion tables.
These tables estimate the placement of raw scores in the score distribution by relying on a
raw score conversion to standard scores (z-scores).
Percentiles ARE NOT percentages (%).
Percentages (%) - raw scores expressed in terms of percentage correct answers.
Percentiles - derived scores expressed in terms of percentage of persons surpassing a
specific raw score.
Disadvantage of percentiles:
- inequality of scale units (especially under extreme ends of the distribution).
- Percentile marks are ordinal level measures, can't be used for normal arithmetic
calculations.
- Percentile ranks calculated for various variables & on various populations are not
directly comparable.
Standard scores:
Can be classified as z-scores, linearly transformed z-scores, and normalised standard scores.
Z-scores:
Expresses an individual’s distance from the mean in standard deviation units.
The z-score for a particular raw score is easily calculated by subtracting the mean from it
and dividing the result by the standard deviation (s).
A raw score equal to the mean is equal to a z-score of 0.
Positive z-scores - above-average performance.
Negative z-scores - below-average performance.
Z-scores routinely range between approximately -3.0 to +3.0.
