Summary measurement theory and assessment 1
CH1: applications and consequences of psychological testing
A psychometrician is a specialist in psychology or education who develops and evaluates
psychological tests. A test is a standardised procedure for sampling behaviour and describing
it with categories or scores. Diagnostics refers to getting to know a situation well enough to
be able to make a decision. Psychodiagnostics refers to getting to know an individual’s
psychosocial functioning. The following features must be included in (norm-referenced)
tests:
Standardised procedure. A test is standardised if the procedures for administering it
are uniform from one examiner and setting to another. Standardisation relies mainly
on the precision of the directions for administration found in the instructional manual
of the test.
Behaviour sample. A small sample of behaviour is tested to make inferences about
the total domain of relevant behaviours. A good test permits the examiner to predict
other behaviours (correlations), it does not necessarily mirror the to-be-predicted
behaviours.
Scores or categories. Psychological testing sums up performance in numbers or
classifications. The score resulting from a test always contains a measurement error:
X (observed score) = T (true score) + e (error component). In a good test, the error is
as small as possible. However, it is impossible to eliminate it or indicate its exact
impact in the individual case.
Norms or standards. A test must possess norms; a summary of test results for a
large representative group of subjects. This group of subjects is referred to as the
standardisation sample. Individual test scores are compared to the norm to reach a
conclusion.
Prediction of non-test behaviour. The results of a test are used to predict real-life
behaviour outside of the test. How well a test predicts non-test behaviour depends on
its validity.
For norm-referenced tests, scores are compared to a norm (relative score). However, in rare
cases, there is no norm and a criterion-referenced test must be used. In a criterion-
referenced test, the objective is to determine where the examinee stands with respect to very
tightly defined educational objective (i.e. 95% accuracy, absolute score). Two types of
criterion-referenced tests with absolute scores are a summative and formative assessment. A
summative assessment assesses what an individual has learned, e.g. the exam of this course.
A formative assessment assesses the progress of an individual, e.g. the quizzes that can be
used to study for this course.
,The COTAN is the committee on tests and testing in the Netherlands. It informs users about
the quality of instruments and provides feedback to developers of instruments. The COTAN
has five criteria for a good test:
Principles of the test construction must be described clearly.
The test must be standardised.
The norm group used to standardise the test must be representative.
The test must be reliable.
The test must be valid.
Testing and assessment are often considered equivalent. However, assessment is a more
comprehensive term, referring to the entire process of compiling information about a person
and using it to make inferences about characteristics and to predict behaviour. The
assessment of human characteristics involves observations, interviews, checklists,
inventories, projectives, and other psychological tests. A test is only one source of
information in an assessment process.
There are eight main types of psychological tests:
Intelligence tests. Measures an individual’s ability in relatively global areas such as
verbal comprehension, perceptual organisation, or reasoning and thereby helps
determine the potential for scholastic work or certain occupations.
Aptitude tests. Measures the capability for a relatively specific task or type of skill;
aptitude tests are, in effect, a narrow form of ability testing.
Achievement tests. Measures a person’s degree of learning, success, or
accomplishment in a subject or task.
Creativity tests. Assess novel, original thinking and the capacity to find unusual or
unexpected solutions, especially for vaguely defined problems.
Personality tests. Measures the traits, qualities, or behaviours that determine a
person’s individuality; such tests include checklists, inventories, and projective
techniques.
Interest inventories. Measures an individual’s preference for certain activities or
topics and thereby helps determine occupational choice.
Behavioural procedures. Objectively describes and counts the frequency of a
behaviour, identifying the antecedents and consequences of the behaviour.
Neuropsychological tests. Measures cognitive, sensory, perceptual, and motor
performance to determine the extent, locus, and behavioural consequences of brain
damage.
, There are five uses of psychological tests:
Classification. Classification occurs in four different forms. Placement is the sorting
of persons into different programs appropriate to their needs or skills. Screening
refers to quick and simple tests or procedures to identify persons who might have
special characteristics or needs. Certification and selection both have a pass/fail
quality; it typically implies that a person has at least a minimum proficiency in some
discipline or activity.
Diagnosis and treatment planning. A diagnosis consists of two intertwined tasks;
determining the nature and source of a person’s abnormal behaviour, and classifying
the behaviour pattern within an accepted diagnostic system. A diagnosis is usually a
precursor to treatment of personal distress or impaired performance.
Self-knowledge. Based on new knowledge about themselves, people can make
decisions to change something in their lives.
Program evaluation. Tests can be used to evaluate the impact of a program. If the
program turns out to be insufficient, it can be improved.
Research.
Rapport is a comfortable, warm atmosphere that serves to motivate examinees and elicit
cooperation. Rapport is important to get the best results from a test. Rapport is especially
important for individual testing and testing with children.
Test anxiety can interfere greatly with test results. Test anxiety refers to those
phenomenological, physiological, and behavioural responses that accompany concern about
the possible failure of a test. Time pressure causes even more test anxiety.
Tests should only be used by qualified professionals. Therefore, access to tests is restricted so
that only qualified users are able to purchase them. The APA proposes three different levels
of complexity for tests and who can purchase them:
Level A. These instruments are straightforward paper-and-pencil measures that can be
administered, scored, and interpreted with minimal training. With the aid of a manual,
these tests can be used by responsible non-psychologists such as business executives
or educational administrators. This category includes vocational proficiency and
group educational achievement tests.
Level B. These tests require knowledge of test construction and training in statistics
and psychology. These products are available to people who have completed an
advanced-level course in testing from an accredited college or university, or
equivalent training under the supervision of a qualified psychologist. This category
includes aptitude tests and personality inventories applicable to normal populations.
Level C. These tests require substantial understanding of testing and supporting
topics. Supervised experience is essential for the proper administration, scoring, and
interpretation of these instruments. Level C tests are typically only available to people
with a minimum of a master’s degree in psychology or an allied field. These
instruments include individual tests of intelligence, projective personality tests, and
neuropsychological tests.
CH1: applications and consequences of psychological testing
A psychometrician is a specialist in psychology or education who develops and evaluates
psychological tests. A test is a standardised procedure for sampling behaviour and describing
it with categories or scores. Diagnostics refers to getting to know a situation well enough to
be able to make a decision. Psychodiagnostics refers to getting to know an individual’s
psychosocial functioning. The following features must be included in (norm-referenced)
tests:
Standardised procedure. A test is standardised if the procedures for administering it
are uniform from one examiner and setting to another. Standardisation relies mainly
on the precision of the directions for administration found in the instructional manual
of the test.
Behaviour sample. A small sample of behaviour is tested to make inferences about
the total domain of relevant behaviours. A good test permits the examiner to predict
other behaviours (correlations), it does not necessarily mirror the to-be-predicted
behaviours.
Scores or categories. Psychological testing sums up performance in numbers or
classifications. The score resulting from a test always contains a measurement error:
X (observed score) = T (true score) + e (error component). In a good test, the error is
as small as possible. However, it is impossible to eliminate it or indicate its exact
impact in the individual case.
Norms or standards. A test must possess norms; a summary of test results for a
large representative group of subjects. This group of subjects is referred to as the
standardisation sample. Individual test scores are compared to the norm to reach a
conclusion.
Prediction of non-test behaviour. The results of a test are used to predict real-life
behaviour outside of the test. How well a test predicts non-test behaviour depends on
its validity.
For norm-referenced tests, scores are compared to a norm (relative score). However, in rare
cases, there is no norm and a criterion-referenced test must be used. In a criterion-
referenced test, the objective is to determine where the examinee stands with respect to very
tightly defined educational objective (i.e. 95% accuracy, absolute score). Two types of
criterion-referenced tests with absolute scores are a summative and formative assessment. A
summative assessment assesses what an individual has learned, e.g. the exam of this course.
A formative assessment assesses the progress of an individual, e.g. the quizzes that can be
used to study for this course.
,The COTAN is the committee on tests and testing in the Netherlands. It informs users about
the quality of instruments and provides feedback to developers of instruments. The COTAN
has five criteria for a good test:
Principles of the test construction must be described clearly.
The test must be standardised.
The norm group used to standardise the test must be representative.
The test must be reliable.
The test must be valid.
Testing and assessment are often considered equivalent. However, assessment is a more
comprehensive term, referring to the entire process of compiling information about a person
and using it to make inferences about characteristics and to predict behaviour. The
assessment of human characteristics involves observations, interviews, checklists,
inventories, projectives, and other psychological tests. A test is only one source of
information in an assessment process.
There are eight main types of psychological tests:
Intelligence tests. Measures an individual’s ability in relatively global areas such as
verbal comprehension, perceptual organisation, or reasoning and thereby helps
determine the potential for scholastic work or certain occupations.
Aptitude tests. Measures the capability for a relatively specific task or type of skill;
aptitude tests are, in effect, a narrow form of ability testing.
Achievement tests. Measures a person’s degree of learning, success, or
accomplishment in a subject or task.
Creativity tests. Assess novel, original thinking and the capacity to find unusual or
unexpected solutions, especially for vaguely defined problems.
Personality tests. Measures the traits, qualities, or behaviours that determine a
person’s individuality; such tests include checklists, inventories, and projective
techniques.
Interest inventories. Measures an individual’s preference for certain activities or
topics and thereby helps determine occupational choice.
Behavioural procedures. Objectively describes and counts the frequency of a
behaviour, identifying the antecedents and consequences of the behaviour.
Neuropsychological tests. Measures cognitive, sensory, perceptual, and motor
performance to determine the extent, locus, and behavioural consequences of brain
damage.
, There are five uses of psychological tests:
Classification. Classification occurs in four different forms. Placement is the sorting
of persons into different programs appropriate to their needs or skills. Screening
refers to quick and simple tests or procedures to identify persons who might have
special characteristics or needs. Certification and selection both have a pass/fail
quality; it typically implies that a person has at least a minimum proficiency in some
discipline or activity.
Diagnosis and treatment planning. A diagnosis consists of two intertwined tasks;
determining the nature and source of a person’s abnormal behaviour, and classifying
the behaviour pattern within an accepted diagnostic system. A diagnosis is usually a
precursor to treatment of personal distress or impaired performance.
Self-knowledge. Based on new knowledge about themselves, people can make
decisions to change something in their lives.
Program evaluation. Tests can be used to evaluate the impact of a program. If the
program turns out to be insufficient, it can be improved.
Research.
Rapport is a comfortable, warm atmosphere that serves to motivate examinees and elicit
cooperation. Rapport is important to get the best results from a test. Rapport is especially
important for individual testing and testing with children.
Test anxiety can interfere greatly with test results. Test anxiety refers to those
phenomenological, physiological, and behavioural responses that accompany concern about
the possible failure of a test. Time pressure causes even more test anxiety.
Tests should only be used by qualified professionals. Therefore, access to tests is restricted so
that only qualified users are able to purchase them. The APA proposes three different levels
of complexity for tests and who can purchase them:
Level A. These instruments are straightforward paper-and-pencil measures that can be
administered, scored, and interpreted with minimal training. With the aid of a manual,
these tests can be used by responsible non-psychologists such as business executives
or educational administrators. This category includes vocational proficiency and
group educational achievement tests.
Level B. These tests require knowledge of test construction and training in statistics
and psychology. These products are available to people who have completed an
advanced-level course in testing from an accredited college or university, or
equivalent training under the supervision of a qualified psychologist. This category
includes aptitude tests and personality inventories applicable to normal populations.
Level C. These tests require substantial understanding of testing and supporting
topics. Supervised experience is essential for the proper administration, scoring, and
interpretation of these instruments. Level C tests are typically only available to people
with a minimum of a master’s degree in psychology or an allied field. These
instruments include individual tests of intelligence, projective personality tests, and
neuropsychological tests.
