Problem 1
Psychological test
A standardized measure of a sample of behaviour that establishes norms and uses
important test items that correspond to what the test is to discover about the test maker.
Norms
Rely on the number of test-takers who take a given test, to establish what is normal in the
group.
The diagnostic process (REPIE)
1. Recognition: What are the problems; what works and what does not work?
2. Explanation: Why do certain problems exist and what is maintaining them?
3. Prediction: How will the client's problems subsequently develop in the future?
4. Indication: How can the problems be resolved?
5. Evaluation: Have the problems been adequately resolved as a result of the
intervention?
Sensitivity
Measure of how well a test can identify true positives
Specificity
The ability of a test to correctly identify people without the disease, true negatives
Cut-off points
Used to classify patients vs. non-patients.
Problem 2
Reliability
How consistently a method measures something
4 main types of reliability (TIPI)
● Test-retest reliability
Measures the consistency of the same test over time
● Interrater reliability
Measures the consistency of the same test conducted by different people
● Parallel forms reliability
Measures the consistency of different versions of a test which are designed to be equivalent
● Internal consistency reliability
Measures the consistency of the individual items of a test
- Two common methods are used to measure internal consistency.
- Average inter-item correlation: For a set of measures designed to assess
the same construct, you calculate the correlation between the results of all
possible pairs of items and then calculate the average.
- Split-half reliability: You randomly split a set of measures into two sets. After
testing the entire set on the respondents, you calculate the correlation
between the two sets of responses.
, Cronbach’s alpha
Estimate of internal consistency reliability
Classical test score theory
Assumes that each person has a true score that would be obtained if there were no errors in
measurement.
Item response theory
Machine learning → when constantly giving correct answers on a computer, the computer
gives increasingly harder questions to match the participants level
Domain sampling model
It is not about the domain of participant, but the number of items in a test
Problem 3
Validity
The agreement between a test score or measure and the quality it is believed to measure
Face validity
Measures whether a test looks like it tests what it is supposed to test.
Content-related
Considers the adequacy of representation of the conceptual domain the test is designed to
cover.
- Construct underrepresentation → the failure to capture important components of a
construct.
- Construct-irrelevant variance → occurs when scores are influenced by factors
irrelevant to the construct.
Construct validity evidence
Used when a specific criterion is not well defined.
- Convergent evidence for validity → when a measure correlates well with other
tests believed to measure the same construct.
- Discriminant evidence for validity (divergent validation) → proof that the test
measures something unique / distinct from other tests → demonstration of
uniqueness.
Criterion validity evidence
Tells us how well a test corresponds with a particular criterion.
- Predictive validity evidence → comes from studies that use a test to forecast
performance on a criterion that is measured at some point in the future → the
forecasting function of tests.
- Concurrent validity evidence → the degree to which the measures gathered from
one tool agree with the measures gathered from other assessment techniques.
- Retrospective validity evidence → looking back in time, how well did the
measurements taken now fall in line with already measured results.
Psychological test
A standardized measure of a sample of behaviour that establishes norms and uses
important test items that correspond to what the test is to discover about the test maker.
Norms
Rely on the number of test-takers who take a given test, to establish what is normal in the
group.
The diagnostic process (REPIE)
1. Recognition: What are the problems; what works and what does not work?
2. Explanation: Why do certain problems exist and what is maintaining them?
3. Prediction: How will the client's problems subsequently develop in the future?
4. Indication: How can the problems be resolved?
5. Evaluation: Have the problems been adequately resolved as a result of the
intervention?
Sensitivity
Measure of how well a test can identify true positives
Specificity
The ability of a test to correctly identify people without the disease, true negatives
Cut-off points
Used to classify patients vs. non-patients.
Problem 2
Reliability
How consistently a method measures something
4 main types of reliability (TIPI)
● Test-retest reliability
Measures the consistency of the same test over time
● Interrater reliability
Measures the consistency of the same test conducted by different people
● Parallel forms reliability
Measures the consistency of different versions of a test which are designed to be equivalent
● Internal consistency reliability
Measures the consistency of the individual items of a test
- Two common methods are used to measure internal consistency.
- Average inter-item correlation: For a set of measures designed to assess
the same construct, you calculate the correlation between the results of all
possible pairs of items and then calculate the average.
- Split-half reliability: You randomly split a set of measures into two sets. After
testing the entire set on the respondents, you calculate the correlation
between the two sets of responses.
, Cronbach’s alpha
Estimate of internal consistency reliability
Classical test score theory
Assumes that each person has a true score that would be obtained if there were no errors in
measurement.
Item response theory
Machine learning → when constantly giving correct answers on a computer, the computer
gives increasingly harder questions to match the participants level
Domain sampling model
It is not about the domain of participant, but the number of items in a test
Problem 3
Validity
The agreement between a test score or measure and the quality it is believed to measure
Face validity
Measures whether a test looks like it tests what it is supposed to test.
Content-related
Considers the adequacy of representation of the conceptual domain the test is designed to
cover.
- Construct underrepresentation → the failure to capture important components of a
construct.
- Construct-irrelevant variance → occurs when scores are influenced by factors
irrelevant to the construct.
Construct validity evidence
Used when a specific criterion is not well defined.
- Convergent evidence for validity → when a measure correlates well with other
tests believed to measure the same construct.
- Discriminant evidence for validity (divergent validation) → proof that the test
measures something unique / distinct from other tests → demonstration of
uniqueness.
Criterion validity evidence
Tells us how well a test corresponds with a particular criterion.
- Predictive validity evidence → comes from studies that use a test to forecast
performance on a criterion that is measured at some point in the future → the
forecasting function of tests.
- Concurrent validity evidence → the degree to which the measures gathered from
one tool agree with the measures gathered from other assessment techniques.
- Retrospective validity evidence → looking back in time, how well did the
measurements taken now fall in line with already measured results.
