Test Theory – Lecture Notes
Lecture 1 – Introduction
Test theory
- Developing and ensuring high-quality psychological tests is essential
Without high-quality test psychology is barely a science
Without high-quality test psychology can contribute little to society
- An entire discipline within psychology is devoted to researching and improving the quality of test = Test theory
(also called Psychometrics)
Introduction
When assessing individuals you generally use a test with a lot of items. Items as indicators for the construct:
1) Answers are assigned scores
2) Item scores are transformed to test scores
3) Test scores are interpreted
Tests are used a lot for measurement in psychology, generally the most convenient way to collect data.
If you want to measure well and accurately you need a good test Otherwise sloppy science.
Unfortunately there are no clear-cut roles for creating a good test. It requires constant thought, good knowledge of the
property you want to measure, proper use of statistical methods. There are tools to measure this Test theory.
Triangle of position towards psychological testing
Introduction – Examples of use of psychological tests
Type of psychologist Measuring what? Why?
Clinical psychologist Psychological disorders Facilities new prisoners
Educational psychologist Learning abilities Placement of children in the correct
types of secondary education (CITO)
Social psychologist Affection Scientific research
Occupational psychologist (HRM) Intelligence Filling job vacancy for manager
Cultural psychologist Individualism/collectivism Explaining cultural differences
Developmental psychologist Attitude concerning upbringing Advice to parents with problem
children
Teacher Student-mastery of the domain of Passing or failing
test theory
,Science
Psychological research is about non-directly observable properties. Tests are always necessary to measure these
properties. (Almost) all psychological theories created because of test theory (Big Five, intelligence).
Formulas
N
Average:
∑ Xi
X = i=1
N
Deviations: x=X −X
N
Variance:
∑ X 2i
2 i=1
SX=
N
Standard deviation: S X = S X √ 2
x
Standardized scores: Z-scores = Z X =
SX
N
Covariance:
∑ xi yi
S XY =C XY = i=1
N
S XY
Correlation: r XY =
S X ∗S Y
Examples
Covariance and correlations
Variance-covariance matrix = Variance on the diagonal elements, covariance on the off-diagonal elements
Correlation matrix = Ones on the diagonal elements, correlations on the off-diagonal elements.
,
, Lecture 2 – Properties of Tests and Items
What is a psychological test?
Cronbach (1960) = A systematic procedure for comparing the behavior of two or more people.
This ‘procedure’ can take on many forms:
- Multiple-choice aptitude test
- Personality test with open-ended questions
- Systematic behavioral observation
- Rorschach inkblot test
Three crucial properties
1) Aimed at measuring behavior (observable)
2) Systematic (objective)
3) Comparison of different people (comparative) Can also be a comparison of people over time.
Type of test
Test for maximum performance vs ‘typical’ performance
- Maximum performance test for measuring skills/aptitude
If there is a right answer on a questionnaire.
- Typical performance tests for measuring personality traits, attitudes, disorders
Searching for something that is typical for you, that describes you.
- Big differences in the approach of test development
- Few differences in the statistical analysis of test scores
Two types of maximum performance tests
- ‘Power’ and ‘speed’ tests
- Power tests = Measure skill without time pressure (most common)
More skilled people give more correct answers
This is what you think about when you think about a test
- Speed test = Measure performance under severe time pressure
Question difficulty is trivial
- More skilled people answer more questions within the time limit.
- Example: Bourdon dot concentration test Speed test
Used to test the quick visual response, used in testing for jobs that require quick accurate scanning.
Norm-referenced or criterion-referenced tests:
- Norm-referenced test = Compare people to the rest of the population
- Good norm data on this population of great importance
- Criterion-referenced test = Compare people with an absolute standard
- Test inferences not tied to performance level in the population
- E.g., exam test theory in criterion referenced
What does a psychological test contain?
- Test material
- Test forms
- Test manual
1) Precise test instructions 3) Norm tables
2) Score-processing procedure 4) Discussion of scientific qualities
Item scores are determined such that they are indicative of the construct you want to measure: Higher item scores =
‘higher’ on that attribute.
Properties of the test score
- Test score is generally the sum of the item scores
- Most important outcome of the test that is used
Lecture 1 – Introduction
Test theory
- Developing and ensuring high-quality psychological tests is essential
Without high-quality test psychology is barely a science
Without high-quality test psychology can contribute little to society
- An entire discipline within psychology is devoted to researching and improving the quality of test = Test theory
(also called Psychometrics)
Introduction
When assessing individuals you generally use a test with a lot of items. Items as indicators for the construct:
1) Answers are assigned scores
2) Item scores are transformed to test scores
3) Test scores are interpreted
Tests are used a lot for measurement in psychology, generally the most convenient way to collect data.
If you want to measure well and accurately you need a good test Otherwise sloppy science.
Unfortunately there are no clear-cut roles for creating a good test. It requires constant thought, good knowledge of the
property you want to measure, proper use of statistical methods. There are tools to measure this Test theory.
Triangle of position towards psychological testing
Introduction – Examples of use of psychological tests
Type of psychologist Measuring what? Why?
Clinical psychologist Psychological disorders Facilities new prisoners
Educational psychologist Learning abilities Placement of children in the correct
types of secondary education (CITO)
Social psychologist Affection Scientific research
Occupational psychologist (HRM) Intelligence Filling job vacancy for manager
Cultural psychologist Individualism/collectivism Explaining cultural differences
Developmental psychologist Attitude concerning upbringing Advice to parents with problem
children
Teacher Student-mastery of the domain of Passing or failing
test theory
,Science
Psychological research is about non-directly observable properties. Tests are always necessary to measure these
properties. (Almost) all psychological theories created because of test theory (Big Five, intelligence).
Formulas
N
Average:
∑ Xi
X = i=1
N
Deviations: x=X −X
N
Variance:
∑ X 2i
2 i=1
SX=
N
Standard deviation: S X = S X √ 2
x
Standardized scores: Z-scores = Z X =
SX
N
Covariance:
∑ xi yi
S XY =C XY = i=1
N
S XY
Correlation: r XY =
S X ∗S Y
Examples
Covariance and correlations
Variance-covariance matrix = Variance on the diagonal elements, covariance on the off-diagonal elements
Correlation matrix = Ones on the diagonal elements, correlations on the off-diagonal elements.
,
, Lecture 2 – Properties of Tests and Items
What is a psychological test?
Cronbach (1960) = A systematic procedure for comparing the behavior of two or more people.
This ‘procedure’ can take on many forms:
- Multiple-choice aptitude test
- Personality test with open-ended questions
- Systematic behavioral observation
- Rorschach inkblot test
Three crucial properties
1) Aimed at measuring behavior (observable)
2) Systematic (objective)
3) Comparison of different people (comparative) Can also be a comparison of people over time.
Type of test
Test for maximum performance vs ‘typical’ performance
- Maximum performance test for measuring skills/aptitude
If there is a right answer on a questionnaire.
- Typical performance tests for measuring personality traits, attitudes, disorders
Searching for something that is typical for you, that describes you.
- Big differences in the approach of test development
- Few differences in the statistical analysis of test scores
Two types of maximum performance tests
- ‘Power’ and ‘speed’ tests
- Power tests = Measure skill without time pressure (most common)
More skilled people give more correct answers
This is what you think about when you think about a test
- Speed test = Measure performance under severe time pressure
Question difficulty is trivial
- More skilled people answer more questions within the time limit.
- Example: Bourdon dot concentration test Speed test
Used to test the quick visual response, used in testing for jobs that require quick accurate scanning.
Norm-referenced or criterion-referenced tests:
- Norm-referenced test = Compare people to the rest of the population
- Good norm data on this population of great importance
- Criterion-referenced test = Compare people with an absolute standard
- Test inferences not tied to performance level in the population
- E.g., exam test theory in criterion referenced
What does a psychological test contain?
- Test material
- Test forms
- Test manual
1) Precise test instructions 3) Norm tables
2) Score-processing procedure 4) Discussion of scientific qualities
Item scores are determined such that they are indicative of the construct you want to measure: Higher item scores =
‘higher’ on that attribute.
Properties of the test score
- Test score is generally the sum of the item scores
- Most important outcome of the test that is used
