Reliability and Validity
Reliability: the consistency with which a measure, measures what it is supposed to
measure.
Two Types of Reliability
Test-retest Reliability:
We take the scores from the same group of people on the same test at
different times and correlate them
The results can be contaminated by the respondents repeated
exposure to the test
If it is an ability test then the scores will increase
It can cause faking after awerness of intent of questions
Candidates can get bored and stop participating.
Internal Reliability:
The reliability of the test at a given point in time
Use the split half reliability test to check internal reliability
Split the test into two halves and correlate the two
Most common way to split is to put all even and odd numbered items
together.
Reliability and Error
Environmental factors such as noise, light and time of day may lead to error in
the reliability of the test
Also important are the factors within the respondent such as health, mood and
motivation
Factors in the test also play a role. Wording, scoring, length of test and
ambiguity influence the error
Ability: if the test is too easy everyone will get high scores
Language: if the perso doesn’t understand the item they tend to guess
, Creating a reliable scale
Items must be clear and unambiguous
The language must be at an appropriate level for the respondent
Avoid asking too many things at once
Keep items short and simple
Avoid negative wording of items
Item Analysis
Remove items that have low item scale correlations
Respondents tend to be confused about those items
You can either fix the item or remove it
Reliability Interpretation
Reliability varies between 0 – 1
1 = perfectly reliable
0 = not reliable
When decisions are made about people the reliability = 0.85
Standardized assessments have a reliability of 0.80 – 0.90
Ability test can have a minimum reliability of 0.85
A personality test can have a minimum reliability of 0.70
Higher the reliability the higher the validity
Validity: refers to how well a test measures what it is purported to measure.
Content Descriptive Validity
Face Validity:
measure appears to be suitable for the purpose of which the test was
designed
Reliability: the consistency with which a measure, measures what it is supposed to
measure.
Two Types of Reliability
Test-retest Reliability:
We take the scores from the same group of people on the same test at
different times and correlate them
The results can be contaminated by the respondents repeated
exposure to the test
If it is an ability test then the scores will increase
It can cause faking after awerness of intent of questions
Candidates can get bored and stop participating.
Internal Reliability:
The reliability of the test at a given point in time
Use the split half reliability test to check internal reliability
Split the test into two halves and correlate the two
Most common way to split is to put all even and odd numbered items
together.
Reliability and Error
Environmental factors such as noise, light and time of day may lead to error in
the reliability of the test
Also important are the factors within the respondent such as health, mood and
motivation
Factors in the test also play a role. Wording, scoring, length of test and
ambiguity influence the error
Ability: if the test is too easy everyone will get high scores
Language: if the perso doesn’t understand the item they tend to guess
, Creating a reliable scale
Items must be clear and unambiguous
The language must be at an appropriate level for the respondent
Avoid asking too many things at once
Keep items short and simple
Avoid negative wording of items
Item Analysis
Remove items that have low item scale correlations
Respondents tend to be confused about those items
You can either fix the item or remove it
Reliability Interpretation
Reliability varies between 0 – 1
1 = perfectly reliable
0 = not reliable
When decisions are made about people the reliability = 0.85
Standardized assessments have a reliability of 0.80 – 0.90
Ability test can have a minimum reliability of 0.85
A personality test can have a minimum reliability of 0.70
Higher the reliability the higher the validity
Validity: refers to how well a test measures what it is purported to measure.
Content Descriptive Validity
Face Validity:
measure appears to be suitable for the purpose of which the test was
designed
