PSYCHOMETRICS
SUMMARY
IBP 2017-18
Renée Lipka
,Chapter 1: The Importance of Psychological Measurement
1.1 Observable Behavior and Unobservable Psychological attributes
• Behavioral scientists assess unobservable behavior by finding a observable
behavior that reflects the underlying psychological process
• We cannot “see” memory so conclusions have to make an inference = behavior
observed actually measures underlying construct
> validity = scores actually measure mental state/process, are theoretically
linked to the construct, assume that the concept itself is not just imagined
> hypothetical constructs/latent variables = theoretical psychological
characteristics that cannot be observed
• Operational definitions = procedures used to measure hypothetical constructs
(nr. of recalled items to measure STM)
1.2 Psychological Tests
Definition: “systematic procedure to compare the behavior of two or more people”
1. Involve behavioral samples
2. Samples are collected in systematic way
3. Purpose is to compare behaviors (inter- or intraindividual differences) > identify +
quantify differences
Tests can produce
- Scores reflecting amount of knowledge
- Scores producing categorical data (sorting people)
Types of tests
• Vary in content: aptitude, intelligence, personality
• Vary in response required
> open ended tests: saying anything they want
> closed ended tests: choosing among alternatives
• Vary according to methods: individual or group administration
• Vary in purpose of scores:
> criterion (domain) referenced: predetermined cutoff-score (often used for
decisions about skill level)
2
, > norm referenced: compare person to a representative reference sample
(sample must be representative of population, population must be well
defined and no doubt that person is part of it)
> criterion cutoff scores are often normed
• Speeded tests: time limited
> people not expected to finish entire test in time
> count nr. of questions answered in time
> each questioned answered should be of comfortable difficulty and
answered correctly
• Power tests: not time limited
> goal is to answer all questions
> count correct answers
> items range in difficulty to discriminate people with regard to
psychological attribute in question
• Battery = bundled tests
> not necessarily measure single psychological attribute
• Word measure can be used as verb (to measure) or noun (is a good measure)
What is psychometrics?
• Focus not on testing but attributes of tests
• Psychometrics is the science concerned with evaluating theoretical attributes of
psychological tests
> type of information (scores)
> reliability of data from psychological tests
> validity of data
Francis Galton and the scope of psychometrics
• Anthropometrics = measurement of human features
• Differential psychology as his general approach = study of individual differences
> at contrast with experimental psychology that focusses more on the
average person
1.3 Challenges to Measurement in Psychology
• Psychological phenomena are complex: we are trying to combine all the different
aspects of a concept (e.g. intelligence) in one number
• Participant reactivity: people are conscious of being measured
3
, > social desirability
> demand characteristics (figure out purpose of study)
> malingering (give poor impression on purpose)
• Expectation bias from the researchers themselves
> mostly unconscious due to vested interests
> observer/scorer bias: influence testing procedure
• Reliance on composite scores: combine all item results to one score that
supposedly represents construct
• Score sensitivity: need to find a procedure that is sensitive enough for what we
want to measure
> hard to anticipate magnitude of meaningful differences
• Apparent lack of awareness of important psychometric information
> often use to facilitate flow of information between people (from students
score to teacher)
> instructors often know little about the properties of reasons for their
examination
4
SUMMARY
IBP 2017-18
Renée Lipka
,Chapter 1: The Importance of Psychological Measurement
1.1 Observable Behavior and Unobservable Psychological attributes
• Behavioral scientists assess unobservable behavior by finding a observable
behavior that reflects the underlying psychological process
• We cannot “see” memory so conclusions have to make an inference = behavior
observed actually measures underlying construct
> validity = scores actually measure mental state/process, are theoretically
linked to the construct, assume that the concept itself is not just imagined
> hypothetical constructs/latent variables = theoretical psychological
characteristics that cannot be observed
• Operational definitions = procedures used to measure hypothetical constructs
(nr. of recalled items to measure STM)
1.2 Psychological Tests
Definition: “systematic procedure to compare the behavior of two or more people”
1. Involve behavioral samples
2. Samples are collected in systematic way
3. Purpose is to compare behaviors (inter- or intraindividual differences) > identify +
quantify differences
Tests can produce
- Scores reflecting amount of knowledge
- Scores producing categorical data (sorting people)
Types of tests
• Vary in content: aptitude, intelligence, personality
• Vary in response required
> open ended tests: saying anything they want
> closed ended tests: choosing among alternatives
• Vary according to methods: individual or group administration
• Vary in purpose of scores:
> criterion (domain) referenced: predetermined cutoff-score (often used for
decisions about skill level)
2
, > norm referenced: compare person to a representative reference sample
(sample must be representative of population, population must be well
defined and no doubt that person is part of it)
> criterion cutoff scores are often normed
• Speeded tests: time limited
> people not expected to finish entire test in time
> count nr. of questions answered in time
> each questioned answered should be of comfortable difficulty and
answered correctly
• Power tests: not time limited
> goal is to answer all questions
> count correct answers
> items range in difficulty to discriminate people with regard to
psychological attribute in question
• Battery = bundled tests
> not necessarily measure single psychological attribute
• Word measure can be used as verb (to measure) or noun (is a good measure)
What is psychometrics?
• Focus not on testing but attributes of tests
• Psychometrics is the science concerned with evaluating theoretical attributes of
psychological tests
> type of information (scores)
> reliability of data from psychological tests
> validity of data
Francis Galton and the scope of psychometrics
• Anthropometrics = measurement of human features
• Differential psychology as his general approach = study of individual differences
> at contrast with experimental psychology that focusses more on the
average person
1.3 Challenges to Measurement in Psychology
• Psychological phenomena are complex: we are trying to combine all the different
aspects of a concept (e.g. intelligence) in one number
• Participant reactivity: people are conscious of being measured
3
, > social desirability
> demand characteristics (figure out purpose of study)
> malingering (give poor impression on purpose)
• Expectation bias from the researchers themselves
> mostly unconscious due to vested interests
> observer/scorer bias: influence testing procedure
• Reliance on composite scores: combine all item results to one score that
supposedly represents construct
• Score sensitivity: need to find a procedure that is sensitive enough for what we
want to measure
> hard to anticipate magnitude of meaningful differences
• Apparent lack of awareness of important psychometric information
> often use to facilitate flow of information between people (from students
score to teacher)
> instructors often know little about the properties of reasons for their
examination
4
