Contents
1. Factor Analysis...................................................................................................................................1
2. Reliability analysis..............................................................................................................................6
3.Cluster Analysis...................................................................................................................................7
4. What type of model to use:.............................................................................................................13
5. ANOVA.............................................................................................................................................14
6. ANCOVA...........................................................................................................................................22
7. Relationships among methods.........................................................................................................27
8. Bivariate regression.........................................................................................................................27
9. Multiple regression..........................................................................................................................30
10. Bivariate multiple regression........................................................................................................33
11. Multicollinearity.............................................................................................................................33
12. Assumptions of linear regression...................................................................................................33
13. Interpretations: Transformations..................................................................................................34
14. Nominal variable: dummy coding..................................................................................................34
15. ANOVA vs. Regression...................................................................................................................35
16. T-test vs. AN( C)OVA vs. Regression...............................................................................................35
17. Moderation analysis......................................................................................................................35
18. Mediation Analysis.........................................................................................................................47
19. Binary Regression..........................................................................................................................52
20. Logit (logistic regression)...............................................................................................................55
21. Conjoint analysis............................................................................................................................62
1. Factor Analysis
Factor analysis is used to reduce a large quantity of data by finding common variance to:
, Retrieve underlying dimensions in your dataset, or
Test if the hypothesized dimensions also exist in your dataset
Data is:
Interval or ratio variables
o Often ordinal, but assumed interval (Likert scale)
Outliers vs Odd values:
Outliers = values that are possible on the scale BUT so far off from the rest that they have a
large impact on the variance and mean of the variables, and thus a large impact on the
parameters – results of the analyses
Odd values = values that are NOT possible given the scale of the variable. Obviously this will
lead to erroneous results. Hence, these should be labeled as missings.
Factor analysis steps
1. Assumption check: KMO and Bartlett’s test of sphericity
2. Communalities
3. Check correlation matrix
4. FA output: eigenvalue, % of variance, cumulative % of variance
5. Scree Plot
6. Rotating the factor matrix: communalities check
7. subsequent use of factors
1. KMO and bartletts test of sphericity
KMO >0.5
Bartletts: <0.05
o -H0: variables are uncorrelated, i.e. the identity matrix
o -If we cannot reject H0, no correlations can be established
Both determine the appropriateness of Factor Analysis as both base their measure on the
underlying correlations of the variable.
2. Communalities
The amount of variance a variable shares with all the other variables being considered
o OR the proportion of variance IN A VARIABLE explained by the COMMON factors
Important to mention that it is that this is a metric OF A VARIABLE and not of a factor. And
that it is the variance shared with the variables / explained by factors (plural, and not the
other way around)
the communalities measure the percent of variance in a given variable explained by all the
extracted factors
o This is < 1, since we have fewer factors than variables
, o
o Treshold: >0.4
o too low eliminate that variable and rerun FA Repeat until all are over 0.4
3. Correlation Matrix
Factor 1+2+3 might be similar. Factor 4+5+6 might be similar
Suppose we have X1, X2, X3, X4, X5, X6
Note: SPSS invisibly standardizes X-variables (mu=0, sd=1)
Reforms the variables. So that these variables are comparable to each other
Any set of variables X1….X6 can be expressed as a linear combination of other variables,
called factors F1, F2, F3, F4, F5, F6 based on common variance in X1….X6
You choose only the ‘strongest factors’
Best number of factors unknown upfront
E.g. 2 factors F1 & F2
Lego blocks, tower of different color blocks most blocks is best factor
Will never be done perfectly, info gets lost
E.g. F3, F4, F5, and F6 are not used
4. FA output: eigenvalue, % of variance, cumulative % of variance
,
Criteria for factor selection
Only those for which eigenvalues > 1 (Total)
Eigenvalue: How much variance a factor explains
Sum of eigenvalues is number of variables in this case 11 (as each variable
has a variance of 1)
Those factors that explain > 5% each (in % of variance)
% of variance is eigenvalue divided by number of variables
Total explained variance > 60% (in cumulative %)
5. Scree Plot
Find the elbow is where the size of the eigenvalue starts to be too low dimension
where the scree starts is not the one you want to have anymore
Based on this 4 dimensions, as it levels of at 5
6. Rotation
Unrotated Factor Matrix: hard to interpret
Rotation
o Prevents that all variables load on 1 factor
o Minimizes the number of variables which have high loadings on each given factor
o Does not change the variance explained = no change in communality
Usually: rotate orthogonally (e.g. VARIMAX)
1. Factor Analysis...................................................................................................................................1
2. Reliability analysis..............................................................................................................................6
3.Cluster Analysis...................................................................................................................................7
4. What type of model to use:.............................................................................................................13
5. ANOVA.............................................................................................................................................14
6. ANCOVA...........................................................................................................................................22
7. Relationships among methods.........................................................................................................27
8. Bivariate regression.........................................................................................................................27
9. Multiple regression..........................................................................................................................30
10. Bivariate multiple regression........................................................................................................33
11. Multicollinearity.............................................................................................................................33
12. Assumptions of linear regression...................................................................................................33
13. Interpretations: Transformations..................................................................................................34
14. Nominal variable: dummy coding..................................................................................................34
15. ANOVA vs. Regression...................................................................................................................35
16. T-test vs. AN( C)OVA vs. Regression...............................................................................................35
17. Moderation analysis......................................................................................................................35
18. Mediation Analysis.........................................................................................................................47
19. Binary Regression..........................................................................................................................52
20. Logit (logistic regression)...............................................................................................................55
21. Conjoint analysis............................................................................................................................62
1. Factor Analysis
Factor analysis is used to reduce a large quantity of data by finding common variance to:
, Retrieve underlying dimensions in your dataset, or
Test if the hypothesized dimensions also exist in your dataset
Data is:
Interval or ratio variables
o Often ordinal, but assumed interval (Likert scale)
Outliers vs Odd values:
Outliers = values that are possible on the scale BUT so far off from the rest that they have a
large impact on the variance and mean of the variables, and thus a large impact on the
parameters – results of the analyses
Odd values = values that are NOT possible given the scale of the variable. Obviously this will
lead to erroneous results. Hence, these should be labeled as missings.
Factor analysis steps
1. Assumption check: KMO and Bartlett’s test of sphericity
2. Communalities
3. Check correlation matrix
4. FA output: eigenvalue, % of variance, cumulative % of variance
5. Scree Plot
6. Rotating the factor matrix: communalities check
7. subsequent use of factors
1. KMO and bartletts test of sphericity
KMO >0.5
Bartletts: <0.05
o -H0: variables are uncorrelated, i.e. the identity matrix
o -If we cannot reject H0, no correlations can be established
Both determine the appropriateness of Factor Analysis as both base their measure on the
underlying correlations of the variable.
2. Communalities
The amount of variance a variable shares with all the other variables being considered
o OR the proportion of variance IN A VARIABLE explained by the COMMON factors
Important to mention that it is that this is a metric OF A VARIABLE and not of a factor. And
that it is the variance shared with the variables / explained by factors (plural, and not the
other way around)
the communalities measure the percent of variance in a given variable explained by all the
extracted factors
o This is < 1, since we have fewer factors than variables
, o
o Treshold: >0.4
o too low eliminate that variable and rerun FA Repeat until all are over 0.4
3. Correlation Matrix
Factor 1+2+3 might be similar. Factor 4+5+6 might be similar
Suppose we have X1, X2, X3, X4, X5, X6
Note: SPSS invisibly standardizes X-variables (mu=0, sd=1)
Reforms the variables. So that these variables are comparable to each other
Any set of variables X1….X6 can be expressed as a linear combination of other variables,
called factors F1, F2, F3, F4, F5, F6 based on common variance in X1….X6
You choose only the ‘strongest factors’
Best number of factors unknown upfront
E.g. 2 factors F1 & F2
Lego blocks, tower of different color blocks most blocks is best factor
Will never be done perfectly, info gets lost
E.g. F3, F4, F5, and F6 are not used
4. FA output: eigenvalue, % of variance, cumulative % of variance
,
Criteria for factor selection
Only those for which eigenvalues > 1 (Total)
Eigenvalue: How much variance a factor explains
Sum of eigenvalues is number of variables in this case 11 (as each variable
has a variance of 1)
Those factors that explain > 5% each (in % of variance)
% of variance is eigenvalue divided by number of variables
Total explained variance > 60% (in cumulative %)
5. Scree Plot
Find the elbow is where the size of the eigenvalue starts to be too low dimension
where the scree starts is not the one you want to have anymore
Based on this 4 dimensions, as it levels of at 5
6. Rotation
Unrotated Factor Matrix: hard to interpret
Rotation
o Prevents that all variables load on 1 factor
o Minimizes the number of variables which have high loadings on each given factor
o Does not change the variance explained = no change in communality
Usually: rotate orthogonally (e.g. VARIMAX)
