measurement theory & assessment II – summary
week 1
psychological measurement: assignment of numerals to psych characteristics of individuals
- assignment → test procedure
- characteristic → psych variable/construct
psychometrics: science concerned with evaluating the attributes of psych tests
- most important attributes:
» type of info generated using psych tests (scores)
» reliability of data from psych tests
» issues concerning the validity of data obtained from psych tests
theory: what are relevant latent variables?
- operational definition of latent variable in terms of observable variables
- validity plays important role
statistics:
» analysis of individual diff in item responses
- latent variable: unobservable trait/characteristic
» linking latent and observable variables
- linear regression: dependent variable is continuously distributed
- logistic regression: dependent variable is binary or dichotomous
» association between variables
- correlation ≠ causation
- correlation doesn’t tell you everything
theories inform stats → theory prescribes the distribution of latent variables
red has a lower mean than green,
and shows less variance
yellow has the same mean as green
and black, and shows more variance
psych variables/constructs are latent (unobservable)
problems with measuring latent variables
- complexity of psych phenomena; hard to measure
- participant reactivity (demand characteristics, social desirability, malingering)
- biased data
- psychologists relying on composite scores
- score sensitivity
- lack of awareness of important psychometric info
1
,measurement theory & assessment II – summary
psychological test: way to sample behavior that presumably reflects an underlying psych attribute
- involve behavioral samples of some kind
- behavioral samples must be collected in some systematic way
- purpose of the tests is to compare the behaviors of ≥ 2 ppl
» speed test: limited with the score based on number of questions answered within the time limit
» power test: not time-limited and measure someone’s max capacity rather than their speed
» criterion-referenced test: often seen in settings in which someone’s skill level is being measured
» norm-referenced test: used to understand how a person scores compared with other ppl
various basic statistics
∑𝑋
- mean (𝑀): average of all scores → 𝑀 = 𝑁
- standard deviation (𝑆𝐷): degree of dispersion in group of scores
- variance: squared standard deviation
path diagram
the latent variable is depicted in a circle
- psychopathological, personality, cognitive, development
responses to items are observable
- performance, self-report, ratings
all measurement is subject to error; the observed variable is related to the latent
trait, but never perfectly
properties of numeral and measurement levels
property of identity: assigning numeral values to represent status on a psych variable
- mutually exclusive and exhaustive → everyone can be assigned to only a single value at a time
property of order: indicate the rank order of ppl relative to each other in a single dimension
- implies transitivity
property of quantity: adds info concerning amount of numeral
- distance between numerals interpretable in terms on measurement units
absolute zero: absence of the attribute
- relative zero: assignment of zero to an arbitrary value
2
, measurement theory & assessment II – summary
measurement levels
nominal: way to code categorical info
- exhaustive and mutually exclusive classes
- distribution is determined by probabilities of discrete classes
ordinal: demonstrate a rank order of the classes
- conveys rank order along a single dimension
- distribution is determined by probability of discrete values
interval: rank order of objects where diff in scale values express diff in amounts
- diff in values is interpretable
ratio: interval measurement with an absolute zero
growth curves: info abt the person compared to the norm
one score vs. the norm
z-score (standard score): score expressed in number of 𝑆𝐷 a score falls from the 𝑀
- scale with 𝑀 = 0 and 𝑆𝐷 = 1
(𝑋 − 𝑀)
- 𝑧= 𝑆𝐷
t-score (standardized score)
- 𝑡 = 10𝑧 + 50
percentile score: % of ppl with a score ≤ a specific value
- empirical vs. theoretical
type of score how is it calculated? normal distribution required?
z-score with M and SD no
empirical P score based on data no
theoretical P score based on z-score (M and SD) yes, of the test scores
normalized z-score based on empirical P score yes, in the population
validity: extent to which the test measures what it’s supposed to measure
- degree to which evidence and theory support the interpretation of tests scores
» construct validity: the appropriateness of inferences abt the underlying construct
- internal structure: to what extent does the relationship between items and the latent
construct match expectations based on theory / empirical evidence?
- can be investigated with factor analysis
- insufficient? due to:
▪ unreliable test
▪ theory incorrectly used
▪ validity shrinkage
» content validity: extent to which items are representative of behaviors the test was designed for
- focus on content (duh)
- avoid contamination (irrelevance)
- avoid deficiency (missing relevant items)
» face validity: perceived validity of the test by test-takers and other non-experts
- if test appears nonsensical, may lead to confusion or refusal to participate
» response validity: are the psych processes that ppl use when answering the items the intended
processes?
3
week 1
psychological measurement: assignment of numerals to psych characteristics of individuals
- assignment → test procedure
- characteristic → psych variable/construct
psychometrics: science concerned with evaluating the attributes of psych tests
- most important attributes:
» type of info generated using psych tests (scores)
» reliability of data from psych tests
» issues concerning the validity of data obtained from psych tests
theory: what are relevant latent variables?
- operational definition of latent variable in terms of observable variables
- validity plays important role
statistics:
» analysis of individual diff in item responses
- latent variable: unobservable trait/characteristic
» linking latent and observable variables
- linear regression: dependent variable is continuously distributed
- logistic regression: dependent variable is binary or dichotomous
» association between variables
- correlation ≠ causation
- correlation doesn’t tell you everything
theories inform stats → theory prescribes the distribution of latent variables
red has a lower mean than green,
and shows less variance
yellow has the same mean as green
and black, and shows more variance
psych variables/constructs are latent (unobservable)
problems with measuring latent variables
- complexity of psych phenomena; hard to measure
- participant reactivity (demand characteristics, social desirability, malingering)
- biased data
- psychologists relying on composite scores
- score sensitivity
- lack of awareness of important psychometric info
1
,measurement theory & assessment II – summary
psychological test: way to sample behavior that presumably reflects an underlying psych attribute
- involve behavioral samples of some kind
- behavioral samples must be collected in some systematic way
- purpose of the tests is to compare the behaviors of ≥ 2 ppl
» speed test: limited with the score based on number of questions answered within the time limit
» power test: not time-limited and measure someone’s max capacity rather than their speed
» criterion-referenced test: often seen in settings in which someone’s skill level is being measured
» norm-referenced test: used to understand how a person scores compared with other ppl
various basic statistics
∑𝑋
- mean (𝑀): average of all scores → 𝑀 = 𝑁
- standard deviation (𝑆𝐷): degree of dispersion in group of scores
- variance: squared standard deviation
path diagram
the latent variable is depicted in a circle
- psychopathological, personality, cognitive, development
responses to items are observable
- performance, self-report, ratings
all measurement is subject to error; the observed variable is related to the latent
trait, but never perfectly
properties of numeral and measurement levels
property of identity: assigning numeral values to represent status on a psych variable
- mutually exclusive and exhaustive → everyone can be assigned to only a single value at a time
property of order: indicate the rank order of ppl relative to each other in a single dimension
- implies transitivity
property of quantity: adds info concerning amount of numeral
- distance between numerals interpretable in terms on measurement units
absolute zero: absence of the attribute
- relative zero: assignment of zero to an arbitrary value
2
, measurement theory & assessment II – summary
measurement levels
nominal: way to code categorical info
- exhaustive and mutually exclusive classes
- distribution is determined by probabilities of discrete classes
ordinal: demonstrate a rank order of the classes
- conveys rank order along a single dimension
- distribution is determined by probability of discrete values
interval: rank order of objects where diff in scale values express diff in amounts
- diff in values is interpretable
ratio: interval measurement with an absolute zero
growth curves: info abt the person compared to the norm
one score vs. the norm
z-score (standard score): score expressed in number of 𝑆𝐷 a score falls from the 𝑀
- scale with 𝑀 = 0 and 𝑆𝐷 = 1
(𝑋 − 𝑀)
- 𝑧= 𝑆𝐷
t-score (standardized score)
- 𝑡 = 10𝑧 + 50
percentile score: % of ppl with a score ≤ a specific value
- empirical vs. theoretical
type of score how is it calculated? normal distribution required?
z-score with M and SD no
empirical P score based on data no
theoretical P score based on z-score (M and SD) yes, of the test scores
normalized z-score based on empirical P score yes, in the population
validity: extent to which the test measures what it’s supposed to measure
- degree to which evidence and theory support the interpretation of tests scores
» construct validity: the appropriateness of inferences abt the underlying construct
- internal structure: to what extent does the relationship between items and the latent
construct match expectations based on theory / empirical evidence?
- can be investigated with factor analysis
- insufficient? due to:
▪ unreliable test
▪ theory incorrectly used
▪ validity shrinkage
» content validity: extent to which items are representative of behaviors the test was designed for
- focus on content (duh)
- avoid contamination (irrelevance)
- avoid deficiency (missing relevant items)
» face validity: perceived validity of the test by test-takers and other non-experts
- if test appears nonsensical, may lead to confusion or refusal to participate
» response validity: are the psych processes that ppl use when answering the items the intended
processes?
3
