Psychometrics 2.5
Problem 1 - General Statistics
Important Terms:
- Validity
o we make an inference from an observable behavior to an unobservable psychological
attribute. That is, we assume that the behavior that we observe reflects or reveals
working memory. If this inference is reasonable, then we would say that our
interpretation of the behavior has a degree of validity.
o is a matter of degree
➔ measurement in psychology often, but not always, involves some type of theory linking a
psychological characteristic, process, or state to an observable behavior that is thought to
reflect differences in that psychological attribute.
- Hypothetical constructs or Latent variables
o theoretical psychological characteristics, attributes, processes, or states that cannot be
directly observed
o e.g., knowledge, intelligence, self-esteem, attitudes, hunger, memory, personality
traits, depression, and attention
o are tested via operational definitions: e.g., recall of numbers to measure working
memory
- Psychological Test:
o Chronbach: a systematic procedure for comparing the behavior of two or more people
o Conditions:
▪ (1) tests involve behavioral samples of some kind
▪ (2) the behavioral samples must be collected in some systematic (i.e., clear
and standardized) way
▪ (3) the purpose of the tests is to detect differences between people
▪ Criterion-referenced tests → test score is established as a criterion, and it is
used to sort people into two groups: (1) those whose performance exceeds the
criterion score and (2) those whose performance does not
▪ Norm-referenced tests → comparing a person’s test score with scores
from a reference sample.
, ▪ Speeded tests → time-limited tests; subjects are not expected to complete the
entire test in the allotted time. Speeded tests are scored by counting the
number of questions answered in the allotted time; difficulty should be same
▪ Power tests → not time limited; subjects expected to answer each test
questions; scored by number of correct answers made on the test; test items
must range in difficulty for establishing demographic differences
▪ Reflective (or effect) indicators → often used in Psychology; measured
behavior is caused by hypothetical construct (e.g., IQ quotient for
Intelligence)
▪ formative (or causal) indicators → indicators are not viewed as being
“caused” by the hypothetical construct (e.g., income for SES)
Biases
o Participant reactivity
▪ Subject’s awareness of being tested, which might influence test scores
▪ Examples:
• Adapting to demand characteristics (researcher’s expected
outcomes)
• Social desirability: Trying to leave a good impression
• Malingering: Trying to leave a poor impression
o Observer (or scorer) bias
▪ (un)conscious manipulation of test scores
Measurement
o Composite scores
▪ psychological attributes are measured via tests involving a series of questions
(not only one) intended to measure a specific psychological attribute or
process.
o Score Sensitivity
▪ measure’s/instrument’s ability to discriminate between meaningful amounts
of the dimension being measured (e.g. questions 1,2,3 really measure
extraversion and meaningful extraversion differences between subjects?)
VARIABILITY
- kinds of differences that behavioral scientists attempt to measure
- is the heart of behavioral sciences
o In traindividual variability (Within-Subjects): differences that emerge in one person
over time or under different circumstances
o Interindividual variability (Between-Subjects): differences that exist between people
(e.g., SAT scores)
o distribution of scores: a set of test scores from each subject taken either from between-
or within subject
o Variance: a statistical way of quantifying variability(individual differences) in a
distribution of scores or
o Covariance: a way of quantifying the connection between variability in one set of
scores and variability in another set of scores
- key goal of statistics is to describe a distribution of scores in a meaningful way
o naming central tendency, variability and shape → necessary for establishing reliability
and valididty
,
, Sum of Squares: ( )
Standard deviation vs Variance:
- As an index of variability, the standard deviation has the advantage of reflecting variability in
terms of the size of raw deviation scores, whereas the variance reflects variability in terms of
squared deviation scores.
- standard deviation is sometimes viewed as more intuitive
Factors influencing SD and Variance:
- real variability (degree of variation of the sample)
- metric: unit of length (e.g., IQ has higher values then GPA values) → does not mean actual
variability is higher for Intelligence dimension than Academic Performance dimensions
Characteristics of SD and Variance:
1. At a minimum, they can be 0 which indicates that the scores in the distribution do not vary at all
2. a positive value indicates that there is some amount of variability. Both mathematically and
conceptually, it is impossible to have a negative variance or standard deviation
3. there is no simple way to interpret a variance or standard deviation as large or small (because of
different metrics for different concepts that are not universally representative of the same level of
difference across those fields)
4. variance of a distribution of scores is most interpretable and meaningful when it is put into some kind of
context (e.g., If two distributions are based on the same measure (e.g., IQ test) then the distribution
with the larger variance has greater variability than the distribution with the smaller variance
5. importance of variance and standard deviation lies mainly in their effects on other values that
are more directly interpretable (e.g., correlation coefficient, reliability are based on SD →
seldom hearing about SD interpretations) #
Problem 1 - General Statistics
Important Terms:
- Validity
o we make an inference from an observable behavior to an unobservable psychological
attribute. That is, we assume that the behavior that we observe reflects or reveals
working memory. If this inference is reasonable, then we would say that our
interpretation of the behavior has a degree of validity.
o is a matter of degree
➔ measurement in psychology often, but not always, involves some type of theory linking a
psychological characteristic, process, or state to an observable behavior that is thought to
reflect differences in that psychological attribute.
- Hypothetical constructs or Latent variables
o theoretical psychological characteristics, attributes, processes, or states that cannot be
directly observed
o e.g., knowledge, intelligence, self-esteem, attitudes, hunger, memory, personality
traits, depression, and attention
o are tested via operational definitions: e.g., recall of numbers to measure working
memory
- Psychological Test:
o Chronbach: a systematic procedure for comparing the behavior of two or more people
o Conditions:
▪ (1) tests involve behavioral samples of some kind
▪ (2) the behavioral samples must be collected in some systematic (i.e., clear
and standardized) way
▪ (3) the purpose of the tests is to detect differences between people
▪ Criterion-referenced tests → test score is established as a criterion, and it is
used to sort people into two groups: (1) those whose performance exceeds the
criterion score and (2) those whose performance does not
▪ Norm-referenced tests → comparing a person’s test score with scores
from a reference sample.
, ▪ Speeded tests → time-limited tests; subjects are not expected to complete the
entire test in the allotted time. Speeded tests are scored by counting the
number of questions answered in the allotted time; difficulty should be same
▪ Power tests → not time limited; subjects expected to answer each test
questions; scored by number of correct answers made on the test; test items
must range in difficulty for establishing demographic differences
▪ Reflective (or effect) indicators → often used in Psychology; measured
behavior is caused by hypothetical construct (e.g., IQ quotient for
Intelligence)
▪ formative (or causal) indicators → indicators are not viewed as being
“caused” by the hypothetical construct (e.g., income for SES)
Biases
o Participant reactivity
▪ Subject’s awareness of being tested, which might influence test scores
▪ Examples:
• Adapting to demand characteristics (researcher’s expected
outcomes)
• Social desirability: Trying to leave a good impression
• Malingering: Trying to leave a poor impression
o Observer (or scorer) bias
▪ (un)conscious manipulation of test scores
Measurement
o Composite scores
▪ psychological attributes are measured via tests involving a series of questions
(not only one) intended to measure a specific psychological attribute or
process.
o Score Sensitivity
▪ measure’s/instrument’s ability to discriminate between meaningful amounts
of the dimension being measured (e.g. questions 1,2,3 really measure
extraversion and meaningful extraversion differences between subjects?)
VARIABILITY
- kinds of differences that behavioral scientists attempt to measure
- is the heart of behavioral sciences
o In traindividual variability (Within-Subjects): differences that emerge in one person
over time or under different circumstances
o Interindividual variability (Between-Subjects): differences that exist between people
(e.g., SAT scores)
o distribution of scores: a set of test scores from each subject taken either from between-
or within subject
o Variance: a statistical way of quantifying variability(individual differences) in a
distribution of scores or
o Covariance: a way of quantifying the connection between variability in one set of
scores and variability in another set of scores
- key goal of statistics is to describe a distribution of scores in a meaningful way
o naming central tendency, variability and shape → necessary for establishing reliability
and valididty
,
, Sum of Squares: ( )
Standard deviation vs Variance:
- As an index of variability, the standard deviation has the advantage of reflecting variability in
terms of the size of raw deviation scores, whereas the variance reflects variability in terms of
squared deviation scores.
- standard deviation is sometimes viewed as more intuitive
Factors influencing SD and Variance:
- real variability (degree of variation of the sample)
- metric: unit of length (e.g., IQ has higher values then GPA values) → does not mean actual
variability is higher for Intelligence dimension than Academic Performance dimensions
Characteristics of SD and Variance:
1. At a minimum, they can be 0 which indicates that the scores in the distribution do not vary at all
2. a positive value indicates that there is some amount of variability. Both mathematically and
conceptually, it is impossible to have a negative variance or standard deviation
3. there is no simple way to interpret a variance or standard deviation as large or small (because of
different metrics for different concepts that are not universally representative of the same level of
difference across those fields)
4. variance of a distribution of scores is most interpretable and meaningful when it is put into some kind of
context (e.g., If two distributions are based on the same measure (e.g., IQ test) then the distribution
with the larger variance has greater variability than the distribution with the smaller variance
5. importance of variance and standard deviation lies mainly in their effects on other values that
are more directly interpretable (e.g., correlation coefficient, reliability are based on SD →
seldom hearing about SD interpretations) #
