1
JANUARY/FEBRUARY
2013
JANUARY/FEBRUARY 2013 EXAM
1
, 2
Question 1
Define the two types of criterion-related validity, describe the process of determining these and
give an example of each. Briefly explain what it means if a test is biased in terms of criterion related
validity.
ANSWER:
There are three types of validity or validation procedures, namely:
Content-description procedure,
construct-identification procedure and
criterion-prediction procedures. Criterion-prediction validity or criterion-related validity falls
under the criterion-prediction procedures.
Criterion-related validity, a quantitative statistical procedure, can be defined as
validity that involves the calculation of a correlation coefficient between a predictor or
more than one predictor and a criterion.
Criterion-related validity can also be defined as the degree to which a measure is
related to some standard or criterion that is known to indicate the construct
accurately.
There are two types of criterion-related validity that are distinguished by temporal positioning of
the criterion measure in relation to the measure of the scale which is being validated: namely;
Concurrent validity: the degree to which a new measure is related to pre-existing
measures of the construct. It involves the accuracy with which a measure can identify or
diagnose the current behaviour or status regarding specific skills or characteristics of an
individual. Implies the correlation of two (or more) concurrent sets of behaviour or
constructs.
Predictive validity: refers to the accuracy with which a measure can predict the future
behaviour or category status of an individual that are logically related to the construct.
Implies that psychological measures can be used for decision-making.
The distinction between these two types of criterion-related validity is based on the purpose for
which the measure is used.
To establish Criterion-related validity one has to compare the measure with another measure of
the same construct, called criterion measure.
2
, 3
Test bias in terms of criterion-related validity is referred to as criterion contamination. This is the
effect of any factor or variable on a criterion such that it is no longer a valid measure.
The criterion must be free from any form of bias as this will influence the correlation coefficient
with a predictor.
Rating scales (e.g. performance ratings) are often used but they are subject to rating biases,
where the rater may err on being too lenient or make judgements on a “general impression” of a
person.
The essential characteristics of predictive bias are that -
it is a type of invalidity that prejudices one group more than another group;
group differences in test achievement are not reflected by corresponding differences in the
behaviour domain that the test is meant to measure;
it involves constant and systematic errors (e.g. attenuation as a result of the unreliability of
the criterion), in contrast to errors that can be ascribed to coincidental or chance factors
(sample errors) in the estimation of the criterion score the constant or systematic errors are
usually associated with group membership;
it leads to unfair discrimination against the group whose criterion score is under predicted,
i.e. in practice the group does better in respect of the criterion than is predicted on the
basis of the test scores.
Bias in the predictive validity of a test can be investigated by making use of the
o validity coefficient (i.e. the correlation between the test score and the criterion
score),
o slope and cut-off of the regression line and the standard error of estimate.
Possible criterion measures include,
academic achievements,
performance in specialised training,
job performance,
psychiatric diagnoses, ASJPR
rating and other valid tests.
3
, 4
Question 2
(a) Critically discuss the psychometric approach to intelligence and relate
yourdiscussion to the South African context
4
JANUARY/FEBRUARY
2013
JANUARY/FEBRUARY 2013 EXAM
1
, 2
Question 1
Define the two types of criterion-related validity, describe the process of determining these and
give an example of each. Briefly explain what it means if a test is biased in terms of criterion related
validity.
ANSWER:
There are three types of validity or validation procedures, namely:
Content-description procedure,
construct-identification procedure and
criterion-prediction procedures. Criterion-prediction validity or criterion-related validity falls
under the criterion-prediction procedures.
Criterion-related validity, a quantitative statistical procedure, can be defined as
validity that involves the calculation of a correlation coefficient between a predictor or
more than one predictor and a criterion.
Criterion-related validity can also be defined as the degree to which a measure is
related to some standard or criterion that is known to indicate the construct
accurately.
There are two types of criterion-related validity that are distinguished by temporal positioning of
the criterion measure in relation to the measure of the scale which is being validated: namely;
Concurrent validity: the degree to which a new measure is related to pre-existing
measures of the construct. It involves the accuracy with which a measure can identify or
diagnose the current behaviour or status regarding specific skills or characteristics of an
individual. Implies the correlation of two (or more) concurrent sets of behaviour or
constructs.
Predictive validity: refers to the accuracy with which a measure can predict the future
behaviour or category status of an individual that are logically related to the construct.
Implies that psychological measures can be used for decision-making.
The distinction between these two types of criterion-related validity is based on the purpose for
which the measure is used.
To establish Criterion-related validity one has to compare the measure with another measure of
the same construct, called criterion measure.
2
, 3
Test bias in terms of criterion-related validity is referred to as criterion contamination. This is the
effect of any factor or variable on a criterion such that it is no longer a valid measure.
The criterion must be free from any form of bias as this will influence the correlation coefficient
with a predictor.
Rating scales (e.g. performance ratings) are often used but they are subject to rating biases,
where the rater may err on being too lenient or make judgements on a “general impression” of a
person.
The essential characteristics of predictive bias are that -
it is a type of invalidity that prejudices one group more than another group;
group differences in test achievement are not reflected by corresponding differences in the
behaviour domain that the test is meant to measure;
it involves constant and systematic errors (e.g. attenuation as a result of the unreliability of
the criterion), in contrast to errors that can be ascribed to coincidental or chance factors
(sample errors) in the estimation of the criterion score the constant or systematic errors are
usually associated with group membership;
it leads to unfair discrimination against the group whose criterion score is under predicted,
i.e. in practice the group does better in respect of the criterion than is predicted on the
basis of the test scores.
Bias in the predictive validity of a test can be investigated by making use of the
o validity coefficient (i.e. the correlation between the test score and the criterion
score),
o slope and cut-off of the regression line and the standard error of estimate.
Possible criterion measures include,
academic achievements,
performance in specialised training,
job performance,
psychiatric diagnoses, ASJPR
rating and other valid tests.
3
, 4
Question 2
(a) Critically discuss the psychometric approach to intelligence and relate
yourdiscussion to the South African context
4
