Lecture 3: Factor R
Factor Analysis: Step by Step
• Step 1 | Defining the objectives
• Step 2 | Designing the study
• Step 3 | Checking assumptions
• Step 4 | Estimating the model and assessing fit
• Step 5 | Interpreting the results
• Step 6 | Validating the results
Step 1: Defining the objectives
The concept of correlations
´ When two variables have a strong correlation (i.e. close to -1 or 1), this generally indicates:
1. A causal relationship between the variables (e.g. price level influences/ drives sales), or:
2. The variables may both be capturing a same underlying construct (e.g. promotion response and
private label buying may both be indicators of price sensitivity)
´ Factor analysis is based on the latter interpretation
of correlations…
´ …and uses these correlations to group a set of variables into several smaller subsets:
- Within each subset: correlations are strong. Rationale: variables likely reflect the same concept
- Between subsets: correlations are weak. Rationale: variables likely reflect different concepts
Bring the six variables to the subset of two variables
Why would that be interested to do so? à
- If you want to bring in a set of variables as drivers in a subsequent regression model, then it would end
up with a multicollinearity problem if these variables strongly overlap
o If you would like to explain store choice as a function of promotion response and private label
buying, but these variables are strongly correlated, you will not be able to separate out the
impact of these drivers
- If you have overlap between variables, it may be difficult to communicate the impact of them
separately in your marketing strategy or talking about results that you found to management in such
instances it may be good to find out what are the key issues, the basic dimensions, and then focus on
those.
o If you make an ad for toothpaste, you will not bring up these six different item, but you may
focus on attractiveness and health or either one of the two
Why group variables?
´ 'Overlap' between variables may create several problems:
1. Difficulties in distinguishing between effects of related variables (= classic 'multicollinearity' problem)
(e.g. role of promotion response vs. PL buying in store choice).
2. Difficulties in accounting for a large number of facets in your marketing strategy (general 'areas of
attention' may be preferred).
3. Difficulties in communicating results to management (strong 'core message' should be centered on key
issues).
, Purpose of Factor Analysis
´ Basically, factor analysis may serve two functions:
1. Summarize data: identifying (any) structure in a set of variables makes it easier to describe them.
- Factor Analysis mainly used for theoretical purposes
- You want to find the main underlying construct
2. Reduce data: replacing the current variables with a smaller set, while minimizing information loss!
- Factor Analysis mainly used for practical purposes
- Start with a dataset that is too big to handle, too many columns, try to replace with a dataset with
fewer columns, with a minimum of information loss.
Two approaches to factor analysis
´ Confirmatory factor analysis:
- Theory → Data
1. Preliminary hypotheses on variable structure (hypothesis on the link between variables)
2. Data either confirms or rejects hypotheses
§ Statistical fit tests available
- Beyond course scope (requires Structural Equation Modeling)
´ Exploratory factor analysis (rest of this lecture): à
- Data → Theory
(try to find patterns on the number of factors, start to reason, why you start the patterns that you
find) (Have to fallback on your knowledge)
1. Data indicates number and composition of factors
2. Researcher should then seek for (theoretical) explanation
- Statistical fit tests not available
- Interdependence Method: In this approach you don’t set aside causes and effect up prior, we look
at the whole group of variables and we try to find a pattern there)
Step 2: Deciding on the inputs
Requirements on your data:
´ Variables:
- Should be (treatable) as metric (i.e. ratio, interval, ≥5 pt. Likert, 0/1 dummies)
- Should make conceptual sense (garbage-in-garbage-out!) (have some logic)
´ Observations:
- Sample size: ≥5 observations per variable (≥10 preferred) (however, also dependent on strength of
relationships!)
Step 3: Checking assumptions
´ Multicollinearity= required! (Indicates possible overlap)
- For Factor Analysis Multicollinearity is required / necessary, if you no or hardly any overlap
between the variables it is not going to be a meaningful task to capture together variables that
capture the same construct
´ But…how to check? Different methods available (however: not always consistent!):
- Method 1: Visual inspection of the correlation matrix (Is there a sufficient number of high (>|0,3|)
correlations?) (depends on the pattern/ on the size of the problem)
Factor Analysis: Step by Step
• Step 1 | Defining the objectives
• Step 2 | Designing the study
• Step 3 | Checking assumptions
• Step 4 | Estimating the model and assessing fit
• Step 5 | Interpreting the results
• Step 6 | Validating the results
Step 1: Defining the objectives
The concept of correlations
´ When two variables have a strong correlation (i.e. close to -1 or 1), this generally indicates:
1. A causal relationship between the variables (e.g. price level influences/ drives sales), or:
2. The variables may both be capturing a same underlying construct (e.g. promotion response and
private label buying may both be indicators of price sensitivity)
´ Factor analysis is based on the latter interpretation
of correlations…
´ …and uses these correlations to group a set of variables into several smaller subsets:
- Within each subset: correlations are strong. Rationale: variables likely reflect the same concept
- Between subsets: correlations are weak. Rationale: variables likely reflect different concepts
Bring the six variables to the subset of two variables
Why would that be interested to do so? à
- If you want to bring in a set of variables as drivers in a subsequent regression model, then it would end
up with a multicollinearity problem if these variables strongly overlap
o If you would like to explain store choice as a function of promotion response and private label
buying, but these variables are strongly correlated, you will not be able to separate out the
impact of these drivers
- If you have overlap between variables, it may be difficult to communicate the impact of them
separately in your marketing strategy or talking about results that you found to management in such
instances it may be good to find out what are the key issues, the basic dimensions, and then focus on
those.
o If you make an ad for toothpaste, you will not bring up these six different item, but you may
focus on attractiveness and health or either one of the two
Why group variables?
´ 'Overlap' between variables may create several problems:
1. Difficulties in distinguishing between effects of related variables (= classic 'multicollinearity' problem)
(e.g. role of promotion response vs. PL buying in store choice).
2. Difficulties in accounting for a large number of facets in your marketing strategy (general 'areas of
attention' may be preferred).
3. Difficulties in communicating results to management (strong 'core message' should be centered on key
issues).
, Purpose of Factor Analysis
´ Basically, factor analysis may serve two functions:
1. Summarize data: identifying (any) structure in a set of variables makes it easier to describe them.
- Factor Analysis mainly used for theoretical purposes
- You want to find the main underlying construct
2. Reduce data: replacing the current variables with a smaller set, while minimizing information loss!
- Factor Analysis mainly used for practical purposes
- Start with a dataset that is too big to handle, too many columns, try to replace with a dataset with
fewer columns, with a minimum of information loss.
Two approaches to factor analysis
´ Confirmatory factor analysis:
- Theory → Data
1. Preliminary hypotheses on variable structure (hypothesis on the link between variables)
2. Data either confirms or rejects hypotheses
§ Statistical fit tests available
- Beyond course scope (requires Structural Equation Modeling)
´ Exploratory factor analysis (rest of this lecture): à
- Data → Theory
(try to find patterns on the number of factors, start to reason, why you start the patterns that you
find) (Have to fallback on your knowledge)
1. Data indicates number and composition of factors
2. Researcher should then seek for (theoretical) explanation
- Statistical fit tests not available
- Interdependence Method: In this approach you don’t set aside causes and effect up prior, we look
at the whole group of variables and we try to find a pattern there)
Step 2: Deciding on the inputs
Requirements on your data:
´ Variables:
- Should be (treatable) as metric (i.e. ratio, interval, ≥5 pt. Likert, 0/1 dummies)
- Should make conceptual sense (garbage-in-garbage-out!) (have some logic)
´ Observations:
- Sample size: ≥5 observations per variable (≥10 preferred) (however, also dependent on strength of
relationships!)
Step 3: Checking assumptions
´ Multicollinearity= required! (Indicates possible overlap)
- For Factor Analysis Multicollinearity is required / necessary, if you no or hardly any overlap
between the variables it is not going to be a meaningful task to capture together variables that
capture the same construct
´ But…how to check? Different methods available (however: not always consistent!):
- Method 1: Visual inspection of the correlation matrix (Is there a sufficient number of high (>|0,3|)
correlations?) (depends on the pattern/ on the size of the problem)
