CHAPTER 9. INTERPRETING VALIDITY 1
Validity interpretation
1. Squared correlations and Variance explained
Squared correlation is interpreted as proportion of
variance in one variable explained or “accounted for”
the other.
Appealing Approach that we use to interpret the
validity that is explained
Disadvantages:
Technically incorrect in some cases (when
correlation-and not squared correlation- is
interpretable as the proportion of variance)
Variance not intuitive metric because it is on a
quadratic scale
o E.g. distance to your home is 9 squared miles
>Not intuitive
Squaring makes the correlation between two
variables seem to small (10% in a difficult measure
is actually a lot)
2. Statistical significance
A high correlation of sample might be 0 in a
population (esp when sample is small),
Important for convergent and discriminant
validity
Predictive validity results found in the sample do not
represent the actual predictive validity in the entire
population > need inferential statistics
Dichotomous decision: there is/there is not
association between the test and the criterion
How confident we are?
o Confidence of non-zero correlation in the population is
influenced by size of sample’s correlation & sample
size
Are we confident enough?
, CHAPTER 9. INTERPRETING VALIDITY 2
o Confidence levels (e.g. 95% )
o Interpretation: if we have a non-significant convergent
validity We should look at size of correlation and
the sample size (if correlation is small then we are
certain of the poor validity, if correlation is large and
sample size small > poorly conceived study)
3. Estimating Practical Effects
Estimate the impact of a correlation on real life
decision making and predictions.
o The
higher the
correlation (criterion/prediction) the more
successful in using the test for predictions &
decisions.
I. Binomial Effect-Size Display How many successful
and unsuccessful predictions will be made on the basis
of a correlation?
r=0
n= 200
Total in each row and column= 100
Validity interpretation
1. Squared correlations and Variance explained
Squared correlation is interpreted as proportion of
variance in one variable explained or “accounted for”
the other.
Appealing Approach that we use to interpret the
validity that is explained
Disadvantages:
Technically incorrect in some cases (when
correlation-and not squared correlation- is
interpretable as the proportion of variance)
Variance not intuitive metric because it is on a
quadratic scale
o E.g. distance to your home is 9 squared miles
>Not intuitive
Squaring makes the correlation between two
variables seem to small (10% in a difficult measure
is actually a lot)
2. Statistical significance
A high correlation of sample might be 0 in a
population (esp when sample is small),
Important for convergent and discriminant
validity
Predictive validity results found in the sample do not
represent the actual predictive validity in the entire
population > need inferential statistics
Dichotomous decision: there is/there is not
association between the test and the criterion
How confident we are?
o Confidence of non-zero correlation in the population is
influenced by size of sample’s correlation & sample
size
Are we confident enough?
, CHAPTER 9. INTERPRETING VALIDITY 2
o Confidence levels (e.g. 95% )
o Interpretation: if we have a non-significant convergent
validity We should look at size of correlation and
the sample size (if correlation is small then we are
certain of the poor validity, if correlation is large and
sample size small > poorly conceived study)
3. Estimating Practical Effects
Estimate the impact of a correlation on real life
decision making and predictions.
o The
higher the
correlation (criterion/prediction) the more
successful in using the test for predictions &
decisions.
I. Binomial Effect-Size Display How many successful
and unsuccessful predictions will be made on the basis
of a correlation?
r=0
n= 200
Total in each row and column= 100
