Summary questionnaires
1. Reliability: precision of measurement = reliability test {the proportion in the total
variance in the observed scores that can be attributed to variance in true scores (variance
that is systematic)
a. Use estimate of reliability by: test retest correlation, internal consistency (split half
method or Cronbach’s alpha).
b. Measurement error (true score and measurement error = reliability), limits the
criterion orientated validity of a questionnaire and or scale.
i. 0.875 = 87,5% reliable (true score) and 12,5 is caused by error.
2. Validity: measure what you want to measure = internal structure testing
Construct validity: used to find out if Criterion validity: to what extent is the
underlying construct is measured well questionnaire related to an outcome
outside of the questionnaire. Criterion is
a measure of performance beyond the
questionnaire.
- Content domain: does the measure - Concurrent (criterion) validity:
capture all parts of a defined criterion at same/present time /
construct? Does the measure relate to a
- } representative item for the criterion in the present?
content domain
- } construct is well specified
- A problemed could be partial or wrong
coverage.
- Internal structure: - Predictive (criterion) validity:
- } subconstructs are dimensions criterion in the future / does the
- } internal structure can be tested measure relate to key behaviours
with factor analysis in the future?
- Nomological network: relation to Overal:
other constructs - Perfect true score is 1.
- } consisting out of correlation - Relationship between is never
between constructs and other equal to one due to
constructs (theory), between measurement error. Also never
constructs and scales and between equal to correlation between
scales and other constructs. true scores. This is called
- Convergent validity: two scales attenuation of correlation. Aka
measuring the same scale should have there is measurement error and
high intercorrelation – same construct that makes scores closer to 0
- Divergent validity: two scales than when there was no error.
measuring different scales should - We can estimate the correlation
have low correlations. – different between true scores to fix
construct attenuation problemed.
- } Multimethod: expect a low
correlation when measuring different
traits, but using the same method.
,3. Difference between
a. Questionnaires: Behaviours, beliefs, opinions, demographical information, typical
preference.
b. Surveys: technic used
c. Tests: achievement test/maximum performance
4. Basic maths
a. Mean: average of an number
b. Standard deviation: root of the variance / standardised version of the variance
c. Variance: differences between items
d. Covariance: measurement of the strength of an linear relationship [unstandardized
correlation] positive = positive relationship.
i. Positive: higher on X and Higher on Y (more stocks)
ii. Negative: higher on X and lower Y (less stocks)
e. Measurement error: mistakes in the observation/prediction
i. Random measurement error: caused by chance
ii. Systematic measurement error: consistently different was worse
f. Correlations: standardised version of covariance [not the same as causality]
i. Positive: same
ii. Negative: same
iii. Correlations have vive properties.
1. They are very sensitive to outliers
2. Restricted range: sample that is not representative for the
population convince sampling = correlation closer to 0.
3. Different groups can have different correlations
4. The correlation must be linear = no association between x and y
5. Close relationship between reliability and measurement error. A lot
of error = low reliability and low error = high reliability.
g. Pearson correlation:
i. Perfect linear positive: value of 1
ii. Perfect linear negative: value of – 1
iii. Neutral or curflinear: low value
iv. Different groups: ----------------------
h. Standardization: making unstandardized things standardized (Z scores) [lower or
more than average/middle]
Standardized scores have an average of 0 and a standard deviation
of 1
5. Latent construct: not directly measurable constructs
6. Construct: unobservable concept
7. Operationalize: making construct measurable
8. Nominal vs operationalized: own definition in words vs operationalized definition
9. Nomological network: Constructs and dimensions are defined and related constructs are
defined.
10. Homogeneous: Set of items that cover all aspects of constructs (the same)
11. Heterogeneous: set of items that cover parts of the constructs (different)
, 12. Indictive items: positively stated, Contra indictive: negatively stated and need to be recoded.
Contra indictive items need to be recoded and are there because too see if people fill in
the questionnaire correctly.
13. Dimension loading: a component loading can be compared to a correlation. That is, the
component loading of item 1 on component 1 is the correlation between them.
1. Measurement levels
a. Nominal: anything categorical
b. Ordinal: name with order + winning /losing
c. Interval: Rank order and ratio of difference between (no fixed 0 point)
d. Ratio: Rank order, ratio of differences & fixed 0 point
- Scale: 0-20/0-150/200-1500 (aka the the ustanderdized version of mearsureings tests)
- By changing the measurement unit, we can get a completely different value for the
covariance.
2. Likert items: strong agree to strongly disagree items
3. Likert scale: number of Likert items that form a scale together
4. Pearson and covariance are only suitable for data on interval and ratio level and many tests
have Pearson and covariance as a basis.
14. Methods in making questionnaires
Method Characteristics Extra explanation
Construct method = definition - Deductive Convergent validity:
of construct - Nomogeligical network Divergent validity:
- Homogenous set of
items
- Attention for
Convergent and
divergent validity
Faced method = scenario - Deductive Structs: elements in the
creation - 4 steps of analysis scenario sentence in an
- Structs block A1
- Structple Structple: whole scenario
- Mapping questions = figure
mapping sentence Faceds: the block
(Structs + structple 1
A+2D+3E) Mapping question: A1+2D+3E
= sentence out of the scenario
Rational method = extpert - Deductive (based on Face Validity (you can see that
experts) it is directly measuring what it
- High face validity is supposed to measure)
Internal method = - Inductive
combination - Combination of
methods
- Uses reliability and
validity tests
Prototype/prototipical - Inductive Act-nonination: sample of
method - Represents “most people come up with the items
typical” aspects of (what makes a person very
1. Reliability: precision of measurement = reliability test {the proportion in the total
variance in the observed scores that can be attributed to variance in true scores (variance
that is systematic)
a. Use estimate of reliability by: test retest correlation, internal consistency (split half
method or Cronbach’s alpha).
b. Measurement error (true score and measurement error = reliability), limits the
criterion orientated validity of a questionnaire and or scale.
i. 0.875 = 87,5% reliable (true score) and 12,5 is caused by error.
2. Validity: measure what you want to measure = internal structure testing
Construct validity: used to find out if Criterion validity: to what extent is the
underlying construct is measured well questionnaire related to an outcome
outside of the questionnaire. Criterion is
a measure of performance beyond the
questionnaire.
- Content domain: does the measure - Concurrent (criterion) validity:
capture all parts of a defined criterion at same/present time /
construct? Does the measure relate to a
- } representative item for the criterion in the present?
content domain
- } construct is well specified
- A problemed could be partial or wrong
coverage.
- Internal structure: - Predictive (criterion) validity:
- } subconstructs are dimensions criterion in the future / does the
- } internal structure can be tested measure relate to key behaviours
with factor analysis in the future?
- Nomological network: relation to Overal:
other constructs - Perfect true score is 1.
- } consisting out of correlation - Relationship between is never
between constructs and other equal to one due to
constructs (theory), between measurement error. Also never
constructs and scales and between equal to correlation between
scales and other constructs. true scores. This is called
- Convergent validity: two scales attenuation of correlation. Aka
measuring the same scale should have there is measurement error and
high intercorrelation – same construct that makes scores closer to 0
- Divergent validity: two scales than when there was no error.
measuring different scales should - We can estimate the correlation
have low correlations. – different between true scores to fix
construct attenuation problemed.
- } Multimethod: expect a low
correlation when measuring different
traits, but using the same method.
,3. Difference between
a. Questionnaires: Behaviours, beliefs, opinions, demographical information, typical
preference.
b. Surveys: technic used
c. Tests: achievement test/maximum performance
4. Basic maths
a. Mean: average of an number
b. Standard deviation: root of the variance / standardised version of the variance
c. Variance: differences between items
d. Covariance: measurement of the strength of an linear relationship [unstandardized
correlation] positive = positive relationship.
i. Positive: higher on X and Higher on Y (more stocks)
ii. Negative: higher on X and lower Y (less stocks)
e. Measurement error: mistakes in the observation/prediction
i. Random measurement error: caused by chance
ii. Systematic measurement error: consistently different was worse
f. Correlations: standardised version of covariance [not the same as causality]
i. Positive: same
ii. Negative: same
iii. Correlations have vive properties.
1. They are very sensitive to outliers
2. Restricted range: sample that is not representative for the
population convince sampling = correlation closer to 0.
3. Different groups can have different correlations
4. The correlation must be linear = no association between x and y
5. Close relationship between reliability and measurement error. A lot
of error = low reliability and low error = high reliability.
g. Pearson correlation:
i. Perfect linear positive: value of 1
ii. Perfect linear negative: value of – 1
iii. Neutral or curflinear: low value
iv. Different groups: ----------------------
h. Standardization: making unstandardized things standardized (Z scores) [lower or
more than average/middle]
Standardized scores have an average of 0 and a standard deviation
of 1
5. Latent construct: not directly measurable constructs
6. Construct: unobservable concept
7. Operationalize: making construct measurable
8. Nominal vs operationalized: own definition in words vs operationalized definition
9. Nomological network: Constructs and dimensions are defined and related constructs are
defined.
10. Homogeneous: Set of items that cover all aspects of constructs (the same)
11. Heterogeneous: set of items that cover parts of the constructs (different)
, 12. Indictive items: positively stated, Contra indictive: negatively stated and need to be recoded.
Contra indictive items need to be recoded and are there because too see if people fill in
the questionnaire correctly.
13. Dimension loading: a component loading can be compared to a correlation. That is, the
component loading of item 1 on component 1 is the correlation between them.
1. Measurement levels
a. Nominal: anything categorical
b. Ordinal: name with order + winning /losing
c. Interval: Rank order and ratio of difference between (no fixed 0 point)
d. Ratio: Rank order, ratio of differences & fixed 0 point
- Scale: 0-20/0-150/200-1500 (aka the the ustanderdized version of mearsureings tests)
- By changing the measurement unit, we can get a completely different value for the
covariance.
2. Likert items: strong agree to strongly disagree items
3. Likert scale: number of Likert items that form a scale together
4. Pearson and covariance are only suitable for data on interval and ratio level and many tests
have Pearson and covariance as a basis.
14. Methods in making questionnaires
Method Characteristics Extra explanation
Construct method = definition - Deductive Convergent validity:
of construct - Nomogeligical network Divergent validity:
- Homogenous set of
items
- Attention for
Convergent and
divergent validity
Faced method = scenario - Deductive Structs: elements in the
creation - 4 steps of analysis scenario sentence in an
- Structs block A1
- Structple Structple: whole scenario
- Mapping questions = figure
mapping sentence Faceds: the block
(Structs + structple 1
A+2D+3E) Mapping question: A1+2D+3E
= sentence out of the scenario
Rational method = extpert - Deductive (based on Face Validity (you can see that
experts) it is directly measuring what it
- High face validity is supposed to measure)
Internal method = - Inductive
combination - Combination of
methods
- Uses reliability and
validity tests
Prototype/prototipical - Inductive Act-nonination: sample of
method - Represents “most people come up with the items
typical” aspects of (what makes a person very
