PYC4807 FINAL EXAM MEMO 2021

Question 1 Define the two types of criterion-related validity, describe the process of determining these and give an example of each. Briefly explain what it means if a test is biased in terms of criterion related validity. ANSWER: There are three types of validity or validation procedures, namely:  Content-description procedure,  construct-identification procedure and  criterion-prediction procedures. Criterion-prediction validity or criterion-related validity falls under the criterion-prediction procedures.  Criterion-related validity, a quantitative statistical procedure, can be defined as validity that involves the calculation of a correlation coefficient between a predictor or more than one predictor and a criterion.  Criterion-related validity can also be defined as the degree to which a measure is related to some standard or criterion that is known to indicate the construct accurately. There are two types of criterion-related validity that are distinguished by temporal positioning of the criterion measure in relation to the measure of the scale which is being validated: namely;  Concurrent validity: the degree to which a new measure is related to pre-existing measures of the construct. It involves the accuracy with which a measure can identify or diagnose the current behaviour or status regarding specific skills or characteristics of an individual. Implies the correlation of two (or more) concurrent sets of behaviour or constructs.  Predictive validity: refers to the accuracy with which a measure can predict the future behaviour or category status of an individual that are logically related to the construct. Implies that psychological measures can be used for decision-making. The distinction between these two types of criterion-related validity is based on the purpose for which the measure is used. To establish Criterion-related validity one has to compare the measure with another measure of the same construct, called criterion measure. 2 3 Test bias in terms of criterion-related validity is referred to as criterion contamination. This is the effect of any factor or variable on a criterion such that it is no longer a valid measure. The criterion must be free from any form of bias as this will influence the correlation coefficient with a predictor. Rating scales (e.g. performance ratings) are often used but they are subject to rating biases, where the rater may err on being too lenient or make judgements on a “general impression” of a person. The essential characteristics of predictive bias are that - it is a type of invalidity that prejudices one group more than another group;  group differences in test achievement are not reflected by corresponding differences in the behaviour domain that the test is meant to measure;  it involves constant and systematic errors (e.g. attenuation as a result of the unreliability of the criterion), in contrast to errors that can be ascribed to coincidental or chance factors (sample errors) in the estimation of the criterion score the constant or systematic errors are usually associated with group membership;  it leads to unfair discrimination against the group whose criterion score is under predicted, i.e. in practice the group does better in respect of the criterion than is predicted on the basis of the test scores. Bias in the predictive validity of a test can be investigated by making use of the o validity coefficient (i.e. the correlation between the test score and the criterion score), o slope and cut-off of the regression line and the standard error of estimate. Possible criterion measures include,  academic achievements,  performance in specialised training,  job performance,