Survey accounting research
Reliability and validity
Reliability is a necessary condition for validity!
Reliability is the degree to which measurement (on operational level) is free from random errors
Tested through internal consistency
Examine Cronbach’s alpha (>0.6 and <0.95 = reliable)
Validity is the degree to which measurement (on operational level) is free from systematic errors
(bias)
Random errors: f.ex. ticking by accident wrong box
Systematic errors: f.ex. everyone interpret the question wrong
Different types of validity:
Construct validity = the degree to which an empirical measure effectively captures a theoretical
construct of interest = link 2 in Libby boxes (theory is needed for this)
Content validity = the items collectively have a high degree of conceptual overlap with the
theoretical definition of the construct = take all items together to have the complete theoretical
construct (theory is needed for this)
Multiple items make one construct
Face validity = the items measure what they are intented to measure
Conduct a pretest!
Insufficient by itself (you need other kinds of validity too)
Convergent validity = the degree to which two measures (items) of constructs that theoretically
should be related, are in fact related
Examine unidimensionality (factor analysis)
Discriminant validity = the degree to which two measures (items) of constructs that theoretically
should be unrelated, are in fact unrelated
Examine cross-loadings
Significant cross-loadings should be avoided they impact your further analyses
Cross-loadings can lead to 2 problems:
i) If you keep both components as independent variables
Multicollinearity issues
ii) If one is a dependent variable and the other is an independent variable
Type I error (expect high association which is not the case)
,Factor analysis
Factor analysis = a method to statistically assess whether several items may be combined into
one construct. We reduce (combine) a set of variables (items) into a smaller set of dimensions
(constructs), relying on correlations and theory!
Result: constructs = latent variables = variables that we cannot measure directly
o Besides statistics, you need also theory: “statistics are in line with the theory I found”
Exploratory factor analysis (EFA) (newly developed questionnaire)
Know 4 steps in EFA and explain it!
1) How many constructs? = EXTRACTION, combination of:
o Scree plot: plot each eigenvalue against the factor with which it is associated
Look for the point of inflexion = where the slope of the line changes dramatically
o Kaiser’s criterion: for every eigenvalue > 1, make a construct
o Theory!
2) Factor rotation: calculate factor loadings
Checking for high correlation with each other and with the construct
o Orthogonal rotation: varimax
o Oblique rotation: direct oblimin
3) Reliability analysis: Cronbach’s alpha (should be > 0,6 and > 0,95)
If < 0,6: problem with reliability, so also problem with validity (previous steps are for nothing)
4) Define your constructs
Take sum, average (most common in EFA), based on factor scores, etc. The used method
depends on prior research.
Confirmatory factor analysis (CFA) (existing questionnaire)
Always start your analyses with an exploratory factor analysis, because you can not check the
cross-loadings in CFA.
In CFA, you define which items belong to each construct (= latent variable). Only these factor
loadings are estimated
CFA is typically used when estimating the measurement model when performing Structural
Equations Modeling (SEM)!
, Mediation and moderation
Mediatior = variable that explains the relation between X and Y
Testing for Mediation:
Estimate three regression equations:
If the independent variable is significantly correlated with the mediator multicollinearity
Non sign. T-statistics you think there is no mediation, but there is = Type II – error
Recommended to always test the significance of the indirect effect
o Sobel test: sign. Sobel test = sign. Indirect effect
Reliability and validity
Reliability is a necessary condition for validity!
Reliability is the degree to which measurement (on operational level) is free from random errors
Tested through internal consistency
Examine Cronbach’s alpha (>0.6 and <0.95 = reliable)
Validity is the degree to which measurement (on operational level) is free from systematic errors
(bias)
Random errors: f.ex. ticking by accident wrong box
Systematic errors: f.ex. everyone interpret the question wrong
Different types of validity:
Construct validity = the degree to which an empirical measure effectively captures a theoretical
construct of interest = link 2 in Libby boxes (theory is needed for this)
Content validity = the items collectively have a high degree of conceptual overlap with the
theoretical definition of the construct = take all items together to have the complete theoretical
construct (theory is needed for this)
Multiple items make one construct
Face validity = the items measure what they are intented to measure
Conduct a pretest!
Insufficient by itself (you need other kinds of validity too)
Convergent validity = the degree to which two measures (items) of constructs that theoretically
should be related, are in fact related
Examine unidimensionality (factor analysis)
Discriminant validity = the degree to which two measures (items) of constructs that theoretically
should be unrelated, are in fact unrelated
Examine cross-loadings
Significant cross-loadings should be avoided they impact your further analyses
Cross-loadings can lead to 2 problems:
i) If you keep both components as independent variables
Multicollinearity issues
ii) If one is a dependent variable and the other is an independent variable
Type I error (expect high association which is not the case)
,Factor analysis
Factor analysis = a method to statistically assess whether several items may be combined into
one construct. We reduce (combine) a set of variables (items) into a smaller set of dimensions
(constructs), relying on correlations and theory!
Result: constructs = latent variables = variables that we cannot measure directly
o Besides statistics, you need also theory: “statistics are in line with the theory I found”
Exploratory factor analysis (EFA) (newly developed questionnaire)
Know 4 steps in EFA and explain it!
1) How many constructs? = EXTRACTION, combination of:
o Scree plot: plot each eigenvalue against the factor with which it is associated
Look for the point of inflexion = where the slope of the line changes dramatically
o Kaiser’s criterion: for every eigenvalue > 1, make a construct
o Theory!
2) Factor rotation: calculate factor loadings
Checking for high correlation with each other and with the construct
o Orthogonal rotation: varimax
o Oblique rotation: direct oblimin
3) Reliability analysis: Cronbach’s alpha (should be > 0,6 and > 0,95)
If < 0,6: problem with reliability, so also problem with validity (previous steps are for nothing)
4) Define your constructs
Take sum, average (most common in EFA), based on factor scores, etc. The used method
depends on prior research.
Confirmatory factor analysis (CFA) (existing questionnaire)
Always start your analyses with an exploratory factor analysis, because you can not check the
cross-loadings in CFA.
In CFA, you define which items belong to each construct (= latent variable). Only these factor
loadings are estimated
CFA is typically used when estimating the measurement model when performing Structural
Equations Modeling (SEM)!
, Mediation and moderation
Mediatior = variable that explains the relation between X and Y
Testing for Mediation:
Estimate three regression equations:
If the independent variable is significantly correlated with the mediator multicollinearity
Non sign. T-statistics you think there is no mediation, but there is = Type II – error
Recommended to always test the significance of the indirect effect
o Sobel test: sign. Sobel test = sign. Indirect effect
