Quantitative research design methods
Week 1: Factor Analysis
First technique: Principal component analysis (PCA) & Exploratory factor
analysis (EFA)
What:
- data reduction and summarization,
- interdependence technique:, no distinction between dependent and
independent,
PCA:
- Considers total variance of variables and its main goal is to express
the dataset by fewer number of variables data reduction.
- Extracts max amount of info from every variable.
EFA:
- Considers only common variance goal is to identify underlying
dimensions that explain correlations among variables.
- Less respect for variables, disregards specific info. Identify
underlying dimensions that Lexplain correlations among the
variables. Just the principal axis factoring (PAF).
- The extracted common factors are not observable, because they are
latent. To address this, scores for the common factors can be
computed as approximations, so they aren’t equal to the common
factor.
assumes variables are prone to random measurement error,
variance is separated into two parts: one explained by the common
factors and another by random measurement error.
Both applications can be:
- Market segmentation (identifying underlying variables on which to
group consumers) ,
- Consumer research (understanding consumer ratings of fast food
restaurants based on underlying dimensions)
- image compression (capture main characteristics of pictures)
Missing values are problematic in both EFA (PAF) and PCA, since they are
both based on the correlation matrix.
KMO criterion is not a statistical test, it is a statistic/metric that is
compared to a cut-off value. Therefore, you cannot draw conclusions about
the population based on this statistic.
1
,Steps shown in the figure (both EFA
and PCA)
Step 1: Select variables
By analyzing what they mean and if they’re
relevant according to past research and
theory. And lastly according to the judgment
of the researcher.
Usually the data is collected in a matrix with
rows and columns.
Step 2: Data screening& testing
assumptions
Data screening: is the data appropriate?
1. Should be interval or ratio
2. Certain min. sample size for reliability
and validity (4/5 times the
observations is the rule of thumb),
3. Check for missing values& outliers
and correct or delete them. You check
this by descriptive statistics and/or
scatterplots.
4. Factorability: ability to conduct factor analysis, if the variables have
a correlation, so they have something in common, if there is no
correlation don’t use factor analysis. You assess this by:
a. Correlation matrix (Pearson): correlations above 0.3
(pearson)
b. Measures of Sampling Adequacy (MSA): should be above
0.5. This value tells you how mch of a variables variance is
shared with other variables.
= 1 if all patrial correlations are zero, the correlation between
two variables can than be fully explained by the other
variables.
= 0 if all correlations are zero, so here is no linear relationship
among the variables.
c. Kaiser-Meyer-Olkin (KMO) criterion: (overall) MSA should be
above 0.5. ranges from 0 to 1.
d. Barletts test of sphericity: statistical test if the population
correlation matrix equals an identity matrix. Should be
significant p < 0.05
Step 3: Extraction method:
which method to use (PCA or EFA)?
2
,- Principal component Analysis (PCA): data reduction technique
wants to find a smaller set of factors/components, that explain as
much as possible of the total variance. Standardizing the variables,
and centering them by their mean and divided by their sd.
After this a line is drawn that captures as much variance as possible.
Then a second line is drawn that is orthogonal to the 1st PC and
captures the remaining variance in the data.
- Exploratory factor analysis (EFA (Principal axis factoring
PAF)): one of the most frequently used techniques in analysis. Does
not consider the total variance of the dataset, but wants to explain
the correlations among observed variables. Its used to identify a
number of factors/dimensions underlying observed variables that
can explain the correlation structure among the observed variables.
It does not assume a variable can be fully explained by the extracted
common factors. So we see an error in the linear equation.
3
, Step 4: Determine the number of factors
We usually want to reduce the number of factors therefore we have
several methods to base them on:
1. Prior knowledge
Just based on knowledge beforehand
2. Eigenvalues
Only Eigenvalues > 1.0 are extracted. This represent the amount of
variance associated with the factor. Factors with a variance less than
1.0 are not ‘better’ than a single variable since each variable has a
variance of 1.0 due to standardization.
3. Percentage of variance
the number of extracted factors is determined so the percentage of
variance explained by the factors reaches a satisfactory level. The
extracted factors should account for at least 60% of the variance.
4. Scree plot
is a plot of Eigenvalues against number of factors. It is believed that
the point where the scree begins is the true number of factors. This
is usually one or a few more than determined by the Eigenvalues
criterion.
5. Parallel analysis
several random datasets are simulated and for each dataset
Eigenvalues of the correlation matrix are calculated. So you make
multiple scree plots.
Step 5: Factor rotation
The unrotated factor matrix indicates the relationship between the factors
it is seldom interpretable because they are highly correlated with many
variables. Through rotation the factor matrix is transformed into a simpler
one that is easier to interpret.
To do this we like each factor to have a non-zero or significant loadings or
coefficients for only some of the variables.
So also we’d like each variable to have non-zdero or significant loadings
with only a few factors, if possible only one.
There are two rotation methods:
4
Week 1: Factor Analysis
First technique: Principal component analysis (PCA) & Exploratory factor
analysis (EFA)
What:
- data reduction and summarization,
- interdependence technique:, no distinction between dependent and
independent,
PCA:
- Considers total variance of variables and its main goal is to express
the dataset by fewer number of variables data reduction.
- Extracts max amount of info from every variable.
EFA:
- Considers only common variance goal is to identify underlying
dimensions that explain correlations among variables.
- Less respect for variables, disregards specific info. Identify
underlying dimensions that Lexplain correlations among the
variables. Just the principal axis factoring (PAF).
- The extracted common factors are not observable, because they are
latent. To address this, scores for the common factors can be
computed as approximations, so they aren’t equal to the common
factor.
assumes variables are prone to random measurement error,
variance is separated into two parts: one explained by the common
factors and another by random measurement error.
Both applications can be:
- Market segmentation (identifying underlying variables on which to
group consumers) ,
- Consumer research (understanding consumer ratings of fast food
restaurants based on underlying dimensions)
- image compression (capture main characteristics of pictures)
Missing values are problematic in both EFA (PAF) and PCA, since they are
both based on the correlation matrix.
KMO criterion is not a statistical test, it is a statistic/metric that is
compared to a cut-off value. Therefore, you cannot draw conclusions about
the population based on this statistic.
1
,Steps shown in the figure (both EFA
and PCA)
Step 1: Select variables
By analyzing what they mean and if they’re
relevant according to past research and
theory. And lastly according to the judgment
of the researcher.
Usually the data is collected in a matrix with
rows and columns.
Step 2: Data screening& testing
assumptions
Data screening: is the data appropriate?
1. Should be interval or ratio
2. Certain min. sample size for reliability
and validity (4/5 times the
observations is the rule of thumb),
3. Check for missing values& outliers
and correct or delete them. You check
this by descriptive statistics and/or
scatterplots.
4. Factorability: ability to conduct factor analysis, if the variables have
a correlation, so they have something in common, if there is no
correlation don’t use factor analysis. You assess this by:
a. Correlation matrix (Pearson): correlations above 0.3
(pearson)
b. Measures of Sampling Adequacy (MSA): should be above
0.5. This value tells you how mch of a variables variance is
shared with other variables.
= 1 if all patrial correlations are zero, the correlation between
two variables can than be fully explained by the other
variables.
= 0 if all correlations are zero, so here is no linear relationship
among the variables.
c. Kaiser-Meyer-Olkin (KMO) criterion: (overall) MSA should be
above 0.5. ranges from 0 to 1.
d. Barletts test of sphericity: statistical test if the population
correlation matrix equals an identity matrix. Should be
significant p < 0.05
Step 3: Extraction method:
which method to use (PCA or EFA)?
2
,- Principal component Analysis (PCA): data reduction technique
wants to find a smaller set of factors/components, that explain as
much as possible of the total variance. Standardizing the variables,
and centering them by their mean and divided by their sd.
After this a line is drawn that captures as much variance as possible.
Then a second line is drawn that is orthogonal to the 1st PC and
captures the remaining variance in the data.
- Exploratory factor analysis (EFA (Principal axis factoring
PAF)): one of the most frequently used techniques in analysis. Does
not consider the total variance of the dataset, but wants to explain
the correlations among observed variables. Its used to identify a
number of factors/dimensions underlying observed variables that
can explain the correlation structure among the observed variables.
It does not assume a variable can be fully explained by the extracted
common factors. So we see an error in the linear equation.
3
, Step 4: Determine the number of factors
We usually want to reduce the number of factors therefore we have
several methods to base them on:
1. Prior knowledge
Just based on knowledge beforehand
2. Eigenvalues
Only Eigenvalues > 1.0 are extracted. This represent the amount of
variance associated with the factor. Factors with a variance less than
1.0 are not ‘better’ than a single variable since each variable has a
variance of 1.0 due to standardization.
3. Percentage of variance
the number of extracted factors is determined so the percentage of
variance explained by the factors reaches a satisfactory level. The
extracted factors should account for at least 60% of the variance.
4. Scree plot
is a plot of Eigenvalues against number of factors. It is believed that
the point where the scree begins is the true number of factors. This
is usually one or a few more than determined by the Eigenvalues
criterion.
5. Parallel analysis
several random datasets are simulated and for each dataset
Eigenvalues of the correlation matrix are calculated. So you make
multiple scree plots.
Step 5: Factor rotation
The unrotated factor matrix indicates the relationship between the factors
it is seldom interpretable because they are highly correlated with many
variables. Through rotation the factor matrix is transformed into a simpler
one that is easier to interpret.
To do this we like each factor to have a non-zero or significant loadings or
coefficients for only some of the variables.
So also we’d like each variable to have non-zdero or significant loadings
with only a few factors, if possible only one.
There are two rotation methods:
4
