Lecture 2
Evaluation of personality measures
● Reliability
● Validity
● Generalizability
Reliability: precision
● What makes a friend or car reliable?
● Reliability refers to the consistency or stability of a measure
● Types of reliability
○ Test-retest reliability: repeated measurement reliability, if test was
reliable, test scores would be the same since personality is not that
malleable from week to week
○ Inter-rater reliability: looks at agreement, correlate observations and
coding
○ Internal consistency reliability: individual items of personality
questionnaire asking questions about the same concept
Cronbach’s alpha: measure of internal consistency, that is, how closely related
a set of items are as a group. It is considered to be a measure of scale
reliability.
Validity
● Degree to which a test measures what it claims to measure
● Types of validity
○ Face validity
○ Predictive or criterion validity
○ Convergent validity
○ Discriminant validity
○ Construct validity
NOTES FROM FYSM 1310
Face validity (does it seem to be measuring/manipulating what it is supposed to be
measuring/manipulating?)
Content validity (does it cover the full breadth of the concept?)
Criterion validity (is it linked with predicted outcomes?)
Convergent validity (is it related to a gold standard, i.e. an established measure of the
construct)
Discriminant validity (does it have a 0 correlation with unrelated variables?)
Construct validity: the degree to which a test measures what it claims, or purports, to be
measuring.
Precision and accuracy
● Reliable but not valid: won’t use scale, not targeting what it is supposed to target
● Reliable and valid: will use
● Unreliable therefore not valid: random collection of questions, not even considered a
scale
, Generalizability
● Degree to which a measure retains validity across different context, including
different groups of people and different conditions
● Greater generalizability not always better; what is important is to identify empirical
contexts in which a measure is and is not applicable
● If it can be generalized, it is more useful, although not always favourable, sometimes
we want to be very specific with what we measure
Research designs and personality
● Experimental methods
○ Used to determine causality
○ Whether one variable causes another
○ 2 key requirements: manipulation of one or more variables and ensuring that
participants in each experimental condition are equivalent to each other at the
start of study
○ Randomly assign participants to groups
○ Ex. introversion-extraversion and positive affect study
○ Manipulate variable
○ When people come in a lab they have to have the same chance of being in
the experimental or control group
Do the results infer causality or correlation? (midterm question)
● Correlational studies
○ Correlational method: statistical procedure for determining whether there
is a relationship between two variables
○ Designed to identify “what goes with what” in nature
○ Major disadvantage: identify relationships among variables as they occur
naturally
○ Correlation coefficient varies from -1 (perfect negative relationship) through 0
(no relationship) to +1 (perfect positive relationship)
○ Correlation does not equal causation
● Case studies
○ In-depth examination of the life of one person
■ Interview of person and informants, naturalistic observation, archival
research, etc
■ Gordon allport
○ Advantages: personality in great detail, insights into personality to formulate a
general theory to test on a larger sample, in-depth knowledge about an
outstanding figure
○ Phineas Gage: Dr. John Martyn Harlow, effect of severe brain injury,
specifically the frontal lobe, on a person’s personality
■ Metal rod through brain, memory and intellect was in tact, although
personality was affected, reconstructed skull, discovered different
parts of brain
○ Disadvantage of case studies: results based on the study of a single person
cannot be generalized to others; for this reason, case studies often used as a
source of hypotheses and to illustrate principles in real life
FYSM 1310
Evaluation of personality measures
● Reliability
● Validity
● Generalizability
Reliability: precision
● What makes a friend or car reliable?
● Reliability refers to the consistency or stability of a measure
● Types of reliability
○ Test-retest reliability: repeated measurement reliability, if test was
reliable, test scores would be the same since personality is not that
malleable from week to week
○ Inter-rater reliability: looks at agreement, correlate observations and
coding
○ Internal consistency reliability: individual items of personality
questionnaire asking questions about the same concept
Cronbach’s alpha: measure of internal consistency, that is, how closely related
a set of items are as a group. It is considered to be a measure of scale
reliability.
Validity
● Degree to which a test measures what it claims to measure
● Types of validity
○ Face validity
○ Predictive or criterion validity
○ Convergent validity
○ Discriminant validity
○ Construct validity
NOTES FROM FYSM 1310
Face validity (does it seem to be measuring/manipulating what it is supposed to be
measuring/manipulating?)
Content validity (does it cover the full breadth of the concept?)
Criterion validity (is it linked with predicted outcomes?)
Convergent validity (is it related to a gold standard, i.e. an established measure of the
construct)
Discriminant validity (does it have a 0 correlation with unrelated variables?)
Construct validity: the degree to which a test measures what it claims, or purports, to be
measuring.
Precision and accuracy
● Reliable but not valid: won’t use scale, not targeting what it is supposed to target
● Reliable and valid: will use
● Unreliable therefore not valid: random collection of questions, not even considered a
scale
, Generalizability
● Degree to which a measure retains validity across different context, including
different groups of people and different conditions
● Greater generalizability not always better; what is important is to identify empirical
contexts in which a measure is and is not applicable
● If it can be generalized, it is more useful, although not always favourable, sometimes
we want to be very specific with what we measure
Research designs and personality
● Experimental methods
○ Used to determine causality
○ Whether one variable causes another
○ 2 key requirements: manipulation of one or more variables and ensuring that
participants in each experimental condition are equivalent to each other at the
start of study
○ Randomly assign participants to groups
○ Ex. introversion-extraversion and positive affect study
○ Manipulate variable
○ When people come in a lab they have to have the same chance of being in
the experimental or control group
Do the results infer causality or correlation? (midterm question)
● Correlational studies
○ Correlational method: statistical procedure for determining whether there
is a relationship between two variables
○ Designed to identify “what goes with what” in nature
○ Major disadvantage: identify relationships among variables as they occur
naturally
○ Correlation coefficient varies from -1 (perfect negative relationship) through 0
(no relationship) to +1 (perfect positive relationship)
○ Correlation does not equal causation
● Case studies
○ In-depth examination of the life of one person
■ Interview of person and informants, naturalistic observation, archival
research, etc
■ Gordon allport
○ Advantages: personality in great detail, insights into personality to formulate a
general theory to test on a larger sample, in-depth knowledge about an
outstanding figure
○ Phineas Gage: Dr. John Martyn Harlow, effect of severe brain injury,
specifically the frontal lobe, on a person’s personality
■ Metal rod through brain, memory and intellect was in tact, although
personality was affected, reconstructed skull, discovered different
parts of brain
○ Disadvantage of case studies: results based on the study of a single person
cannot be generalized to others; for this reason, case studies often used as a
source of hypotheses and to illustrate principles in real life
FYSM 1310
