Factor analysis using principal component analysis
(PCA)
Factor analysis is used when you want to reduce a large set of variables into smaller, underlying factors
or dimensions. Principal Component Analysis (PCA) simplifies data by transforming variables into
components, with the first component capturing the most variance in the data, and each subsequent
component capturing the next largest amount of remaining variance.
1. Remove non-variable columns
Remove columns like 'ID' that do not contain meaningful variables for factor analysis or correlational
analysis, as including them would not make sense for these types of analyses.
2. Test assumptions
Use the KMO-MSA and Bartlett’s test of sphericity to determine if factor analysis is appropriate.
KMO-MSA (Kaiser-Meyer-Olkin measure of sampling adequacy) provides the degree of intercorrelations
among variable and this one must be larger than 0.5.
Ensure the KMO value is > 0.5.
Bartlett's test of sphericity, to test correlation matrix has significant correlation between at least 2 of its
variables:
Ensure the p-value is < 0.05.
3. Create correlation matrix:
Generate a correlation matrix for the dataframe.
4. Determine the number of factors (scree plot):
Determine the number of factors to be used in factor analysis using the components’ Eigenvalues and/or
a scree plot.
An eigenvalue shows how much variation in the data is explained by a factor or component. With an
eigenvalue of ‘1’ indicating that the component explains as much as a single variable from the original
dataset. Eigenvalues must be larger than one because we want each factor to account for the variance for
at least a single variable. If the factor cannot even capture the information of a single variable then it is
not the purpose of dimension reduction.
(PCA)
Factor analysis is used when you want to reduce a large set of variables into smaller, underlying factors
or dimensions. Principal Component Analysis (PCA) simplifies data by transforming variables into
components, with the first component capturing the most variance in the data, and each subsequent
component capturing the next largest amount of remaining variance.
1. Remove non-variable columns
Remove columns like 'ID' that do not contain meaningful variables for factor analysis or correlational
analysis, as including them would not make sense for these types of analyses.
2. Test assumptions
Use the KMO-MSA and Bartlett’s test of sphericity to determine if factor analysis is appropriate.
KMO-MSA (Kaiser-Meyer-Olkin measure of sampling adequacy) provides the degree of intercorrelations
among variable and this one must be larger than 0.5.
Ensure the KMO value is > 0.5.
Bartlett's test of sphericity, to test correlation matrix has significant correlation between at least 2 of its
variables:
Ensure the p-value is < 0.05.
3. Create correlation matrix:
Generate a correlation matrix for the dataframe.
4. Determine the number of factors (scree plot):
Determine the number of factors to be used in factor analysis using the components’ Eigenvalues and/or
a scree plot.
An eigenvalue shows how much variation in the data is explained by a factor or component. With an
eigenvalue of ‘1’ indicating that the component explains as much as a single variable from the original
dataset. Eigenvalues must be larger than one because we want each factor to account for the variance for
at least a single variable. If the factor cannot even capture the information of a single variable then it is
not the purpose of dimension reduction.
