Methodology in marketing and strategy research
Complete summary
Topic 0: Prior knowledge
0.1 Multivariate analysis
Multivariate analysis refers to all statistical techniques that simultaneously analyze multiple
measurements on individuals or objects under investigation. Thus, any simultaneous analysis of more
than two variables can be loosely considered multivariate analysis.
0.2 Measurement scales
Researchers must define the measurement type for each variable.
• Nonmetric measurement scales
o Nominal (categorical): assigns numbers as a way to label or identify subjects or
objects. The numbers assigned have no quantitative meaning beyond indicating the
presence or absence of the attribute. Nominal data only represents categories or
classes. For example, male (1) or female (2), or religion, occupation, etc.
o Ordinal: variables can be ranked or ranked in relation to the amount of the attribute
possessed. However, they provide no measure of the actual amount in absolute terms,
only the order of the values. For example, consumer’s satisfaction (not at all satisfied,
kind of satisfied, very satisfied, etc.)
• Metric measurement scales
o Interval: differences between any two adjacent points on any part of the scale are
equal. For example, temperature (Celsius or Fahrenheit) or IQ scores.
▪ In practice, difference between points on Likert scale are often assumed to be
equal. Therefore Likert scale variables are ordinal by strict definition, but they
are frequently treated as interval.
o Ratio: uses an absolute zero point. For example, 20 kg is twice as heavy as 10 kg. There
needs to be a true zero point. We don’t have this in temperature, because 0 degrees
Celsius does not mean no temperature.
Take note of the measurement level of your variables when carrying out univariate analyses. For non-
metric variables, mean and standard deviations, kurtosis, skewness does not make much sense.
0.3 Univariate profiling
The starting point for understanding the nature of any variable is to characterize the shape of its
distribution. This can be done by looking at a histogram, but we can also look at kurtosis and skewness.
Both kurtosis and skewness should be < |3| for there to be a normal distribution.
• Kurtosis represents the peakedness or flatness of the distribution.
• Skewness represents the balance of the distribution, is it shifted to one side or is it centered
and symmetrical.
0.4 Measurement error
Measurement error is the degree to which the observed values are not representative of the ‘true’
values. This can be because of data entry errors, imprecision of the measurement, inability of
1
,respondents to provide accurate information, and so on. Therefore, all variables used in multivariate
techniques must be assumed to have some degree of measurement error.
• Validity is the degree to which a measure accurately represents what is supposed to.
• Reliability is the degree to which the observed variable measures the ‘true’ value and is ‘error
free’. It refers to the consistency or repeatability of a measurement.
0.5 Multicollinearity
Every time a variate has two or more variables there is the potential for multicollinearity: the degree
of correlation among the variables in the variate that may result in a confounding effect in the
interpretation of the individual variables of the variate. Multicollinearity makes is more difficult to
determine the unique contribution of each variable to the model, which is what we often want to know.
It is harder because shared variance between the variables obscures these unique effects.
• The primary decision here is whether to use the individual variables or to perform some form
of dimensional reduction, such as exploratory factor analysis. The objective of exploratory
factor analysis is to find these groups of multicollinear variables.
0.6 Managing dependence model
Single equation models provide an approach for specifying a single variate’s relationship with an
outcome variable (e.g. ANOVA/MANOVA and logistic regression).
• The foundation for almost all of the single equation techniques is the general linear model
(GLM). One limiting characteristic of the GLM is the assumption of an error distribution
following the normal distribution. As such, many times we must test the assumption that the
dependent variable follows a normal distribution (and if not transform it).
But there are also multiple equation models that enable the researcher to relate different equations
to one another, even where they are interrelated (i.e. the outcome measure on one equation becomes
the predictor variable in another equation).
0.7 Dependence techniques
If the analysis involves a single dependent variable that is metric, which is presumed to be related to
two or more metric independent variables, the appropriate technique is multiple regression analysis.
When the research problem involves several dependent variables, we look at the type of dependent
and independent variables. If the several dependent variables are metric, and the independent
variables are nonmetric, MANOVA should be selected. If the several dependent variables are
nonmetric, then they can be transformed using dummy variables.
0.8 Interdependence techniques
With interdependence techniques the variables cannot be classified as either dependent or
independent. All variables are analyzed simultaneously in an effort to find an underlying structure to
the entire set of variables or subjects. If it is about the structure among variables, factor analysis is
used.
0.9 Practical significance
A researcher must not only look at the statistical significance of the results but also at their practical
significance. Practical significance (also called managerial significance) asks the question, ‘so what?’.
Even though the result or relationship is significant, other things can be important as well. For example,
whether the predictive ability was poor or not.
2
,0.10 Sample size
Sample size affects all result. For smaller samples, the sophistication and complexity of the multivariate
technique may easily result in either:
• Too little statistical power for the test to realistically identify significant results; or
• Too easily ‘overfitting’ the data such that the results are artificially good because they fit the
sample yet provide no generalizability.
A similar impact occurs for large sample sizes (400+). They can make the statistical tests overly
sensitive. Unequal sample size among groups can also influence the results and require additional
interpretation or analysis.
0.11 Data preparation – missing data
Data examination is when the researcher evaluates the impact of missing data, identifies outliers, and
tests for the assumptions underlying most multivariate techniques.
Missing data primarily result from errors in data collection/data entry or from the omission of answers
by respondents. It is important to look at missing data because it has a practical impact, the reduction
of the sample size available for analysis. It also has a substantive impact, any statistical results based
on data with a nonrandom missing data process could be inaccurate.
• Ignorable missing data: operating at random or explicitly accommodated in the technique
used (for example from taking a sample instead of the entire population, or skipping questions
that are not applicable).
• Missing data that are not ignorable: typically fall into two classes based on their sources.
o Known processes: for example, failure to complete questionnaire.
o Unknown processes: for example, refusal of the respondent to respond to certain
questions because they have a sensitive nature.
The most direct means to assess the extent of missing data is by tabulating (1) the percentage of
variables with missing data for each case and (2) the number of cases with missing data for each
variable. The researcher should look for any nonrandom patterns. They should also determine the
number of cases with no missing data on any of the variables, which will provide the sample size
available for analysis if remedies are not applied.
- Cases or observations with 10% or less missing data are generally acceptable.
- Be sure that the minimum sample with complete data, is sufficient for model estimation.
First the researcher should consider simply deleting offending cases and/or variables with excessive
levels of missing data. If this is not possible, the researcher can use an imputation method.
0.12 Data preparation – outliers
Outliers, or extreme responses, may unduly influence the outcome of any multivariate. They are
observations identifiable as distinctly different from what is ‘normal’. In assessing the impact of outliers,
we must consider the practical and substantive considerations along with whether we should designate
outliers as “good or bad.”
• Practical impact: For example, assume that we sample 20 individuals to determine the average
household income. In our sample we gather responses that range between €20,000 and
€100,000, so that the average is €45,000. But assume that the 21st person has an income of
€1 million. If we include this value in the analysis, the average income increases to more than
€90,000. Obviously, the outlier is a valid case, but what is the better estimate of the average
3
, household income: €45,000 or €90,000? The researcher must assess whether the outlying
value is retained or eliminated due to its undue influence on the results
• Substantiative impact: The outlier must be viewed in the light of how representative it is of
the population. Again, using our example of household income, how representative of the
more wealthy segment is the millionaire? If the researcher feels that it is a small, but viable
segment in the population, then perhaps the value should be retained. If, however, this
millionaire is the only one in the entire population and truly far above everyone else (i.e.,
unique) and represents an extreme value, then it may be deleted.
0.13 Assumptions of multivariate analysis
• Normality: A possible remedy for variables that are not normally distributed, is transforming
the variables.
• Homoscedasticity: refers to the assumption that dependent variables exhibit equal levels of
variance across the range of predictor variables. This is desirable because the variance of the
dependent variable being explained in the dependence relationship should not be
concentrated in only a limited range of the independent values. If the dispersion is unequal,
the relationship is heteroscedastic.
o The Levene’s test is used to assess homoscedasticity for a single metric variable. We
want to fail to reject the 0-hypothesis, so p should be > than 0.05 (alpha).
o Box’s M test is used to assess homoscedasticity for more than one metric variable. We
want to fail to reject the 0-hypothesis, so p should be > than 0.05 (alpha).
A remedy for heteroscedasticity could be data transformations similar to those used to achieve
normality. Heteroscedasticity is often the result of non-normality of one of the variables.
• Linearity: the idea that there is a linear relationship between the independent and dependent
variables.
• Absence of correlated errors: assumption that any prediction errors are uncorrelated with
each other.
0.14 Data transformations
Data transformations alter the characteristics of the original variable. If we want to achieve
normality/homoscedasticity we can use transformations. If the distribution is flat, the most common
transformation is the inverse. For skewed distributions the variable can be transformed by taking the
square root, logarithms, squared, or cubed, or even the inverse.
We can also use data transformations for interpretation reasons.
• Standardization: Transforming data by subtracting the variable mean from each value and
dividing the result by the variable's standard deviation. This process results in a dataset with a
mean of zero and a standard deviation of 1. Commonly achieved using z-scores.
o Z-score of 0 indicates the value equals the mean.
o Positive or negative z-scores represent deviations above or below the mean, measured
in standard deviations (a z-score of 0.8 is 0.8 standard deviations above the mean).
Standardization allows for the direct comparison of variables with different distributions.
• Centering: Related to standardization. Involves subtracting the mean value from each
observation, making values relative to their mean without dividing by the standard deviation.
o Variable-based centering: commonly used in multiple regression to improve the
interpretation of moderation and interaction effects, and potentially reduce
multicollinearity.
4
Complete summary
Topic 0: Prior knowledge
0.1 Multivariate analysis
Multivariate analysis refers to all statistical techniques that simultaneously analyze multiple
measurements on individuals or objects under investigation. Thus, any simultaneous analysis of more
than two variables can be loosely considered multivariate analysis.
0.2 Measurement scales
Researchers must define the measurement type for each variable.
• Nonmetric measurement scales
o Nominal (categorical): assigns numbers as a way to label or identify subjects or
objects. The numbers assigned have no quantitative meaning beyond indicating the
presence or absence of the attribute. Nominal data only represents categories or
classes. For example, male (1) or female (2), or religion, occupation, etc.
o Ordinal: variables can be ranked or ranked in relation to the amount of the attribute
possessed. However, they provide no measure of the actual amount in absolute terms,
only the order of the values. For example, consumer’s satisfaction (not at all satisfied,
kind of satisfied, very satisfied, etc.)
• Metric measurement scales
o Interval: differences between any two adjacent points on any part of the scale are
equal. For example, temperature (Celsius or Fahrenheit) or IQ scores.
▪ In practice, difference between points on Likert scale are often assumed to be
equal. Therefore Likert scale variables are ordinal by strict definition, but they
are frequently treated as interval.
o Ratio: uses an absolute zero point. For example, 20 kg is twice as heavy as 10 kg. There
needs to be a true zero point. We don’t have this in temperature, because 0 degrees
Celsius does not mean no temperature.
Take note of the measurement level of your variables when carrying out univariate analyses. For non-
metric variables, mean and standard deviations, kurtosis, skewness does not make much sense.
0.3 Univariate profiling
The starting point for understanding the nature of any variable is to characterize the shape of its
distribution. This can be done by looking at a histogram, but we can also look at kurtosis and skewness.
Both kurtosis and skewness should be < |3| for there to be a normal distribution.
• Kurtosis represents the peakedness or flatness of the distribution.
• Skewness represents the balance of the distribution, is it shifted to one side or is it centered
and symmetrical.
0.4 Measurement error
Measurement error is the degree to which the observed values are not representative of the ‘true’
values. This can be because of data entry errors, imprecision of the measurement, inability of
1
,respondents to provide accurate information, and so on. Therefore, all variables used in multivariate
techniques must be assumed to have some degree of measurement error.
• Validity is the degree to which a measure accurately represents what is supposed to.
• Reliability is the degree to which the observed variable measures the ‘true’ value and is ‘error
free’. It refers to the consistency or repeatability of a measurement.
0.5 Multicollinearity
Every time a variate has two or more variables there is the potential for multicollinearity: the degree
of correlation among the variables in the variate that may result in a confounding effect in the
interpretation of the individual variables of the variate. Multicollinearity makes is more difficult to
determine the unique contribution of each variable to the model, which is what we often want to know.
It is harder because shared variance between the variables obscures these unique effects.
• The primary decision here is whether to use the individual variables or to perform some form
of dimensional reduction, such as exploratory factor analysis. The objective of exploratory
factor analysis is to find these groups of multicollinear variables.
0.6 Managing dependence model
Single equation models provide an approach for specifying a single variate’s relationship with an
outcome variable (e.g. ANOVA/MANOVA and logistic regression).
• The foundation for almost all of the single equation techniques is the general linear model
(GLM). One limiting characteristic of the GLM is the assumption of an error distribution
following the normal distribution. As such, many times we must test the assumption that the
dependent variable follows a normal distribution (and if not transform it).
But there are also multiple equation models that enable the researcher to relate different equations
to one another, even where they are interrelated (i.e. the outcome measure on one equation becomes
the predictor variable in another equation).
0.7 Dependence techniques
If the analysis involves a single dependent variable that is metric, which is presumed to be related to
two or more metric independent variables, the appropriate technique is multiple regression analysis.
When the research problem involves several dependent variables, we look at the type of dependent
and independent variables. If the several dependent variables are metric, and the independent
variables are nonmetric, MANOVA should be selected. If the several dependent variables are
nonmetric, then they can be transformed using dummy variables.
0.8 Interdependence techniques
With interdependence techniques the variables cannot be classified as either dependent or
independent. All variables are analyzed simultaneously in an effort to find an underlying structure to
the entire set of variables or subjects. If it is about the structure among variables, factor analysis is
used.
0.9 Practical significance
A researcher must not only look at the statistical significance of the results but also at their practical
significance. Practical significance (also called managerial significance) asks the question, ‘so what?’.
Even though the result or relationship is significant, other things can be important as well. For example,
whether the predictive ability was poor or not.
2
,0.10 Sample size
Sample size affects all result. For smaller samples, the sophistication and complexity of the multivariate
technique may easily result in either:
• Too little statistical power for the test to realistically identify significant results; or
• Too easily ‘overfitting’ the data such that the results are artificially good because they fit the
sample yet provide no generalizability.
A similar impact occurs for large sample sizes (400+). They can make the statistical tests overly
sensitive. Unequal sample size among groups can also influence the results and require additional
interpretation or analysis.
0.11 Data preparation – missing data
Data examination is when the researcher evaluates the impact of missing data, identifies outliers, and
tests for the assumptions underlying most multivariate techniques.
Missing data primarily result from errors in data collection/data entry or from the omission of answers
by respondents. It is important to look at missing data because it has a practical impact, the reduction
of the sample size available for analysis. It also has a substantive impact, any statistical results based
on data with a nonrandom missing data process could be inaccurate.
• Ignorable missing data: operating at random or explicitly accommodated in the technique
used (for example from taking a sample instead of the entire population, or skipping questions
that are not applicable).
• Missing data that are not ignorable: typically fall into two classes based on their sources.
o Known processes: for example, failure to complete questionnaire.
o Unknown processes: for example, refusal of the respondent to respond to certain
questions because they have a sensitive nature.
The most direct means to assess the extent of missing data is by tabulating (1) the percentage of
variables with missing data for each case and (2) the number of cases with missing data for each
variable. The researcher should look for any nonrandom patterns. They should also determine the
number of cases with no missing data on any of the variables, which will provide the sample size
available for analysis if remedies are not applied.
- Cases or observations with 10% or less missing data are generally acceptable.
- Be sure that the minimum sample with complete data, is sufficient for model estimation.
First the researcher should consider simply deleting offending cases and/or variables with excessive
levels of missing data. If this is not possible, the researcher can use an imputation method.
0.12 Data preparation – outliers
Outliers, or extreme responses, may unduly influence the outcome of any multivariate. They are
observations identifiable as distinctly different from what is ‘normal’. In assessing the impact of outliers,
we must consider the practical and substantive considerations along with whether we should designate
outliers as “good or bad.”
• Practical impact: For example, assume that we sample 20 individuals to determine the average
household income. In our sample we gather responses that range between €20,000 and
€100,000, so that the average is €45,000. But assume that the 21st person has an income of
€1 million. If we include this value in the analysis, the average income increases to more than
€90,000. Obviously, the outlier is a valid case, but what is the better estimate of the average
3
, household income: €45,000 or €90,000? The researcher must assess whether the outlying
value is retained or eliminated due to its undue influence on the results
• Substantiative impact: The outlier must be viewed in the light of how representative it is of
the population. Again, using our example of household income, how representative of the
more wealthy segment is the millionaire? If the researcher feels that it is a small, but viable
segment in the population, then perhaps the value should be retained. If, however, this
millionaire is the only one in the entire population and truly far above everyone else (i.e.,
unique) and represents an extreme value, then it may be deleted.
0.13 Assumptions of multivariate analysis
• Normality: A possible remedy for variables that are not normally distributed, is transforming
the variables.
• Homoscedasticity: refers to the assumption that dependent variables exhibit equal levels of
variance across the range of predictor variables. This is desirable because the variance of the
dependent variable being explained in the dependence relationship should not be
concentrated in only a limited range of the independent values. If the dispersion is unequal,
the relationship is heteroscedastic.
o The Levene’s test is used to assess homoscedasticity for a single metric variable. We
want to fail to reject the 0-hypothesis, so p should be > than 0.05 (alpha).
o Box’s M test is used to assess homoscedasticity for more than one metric variable. We
want to fail to reject the 0-hypothesis, so p should be > than 0.05 (alpha).
A remedy for heteroscedasticity could be data transformations similar to those used to achieve
normality. Heteroscedasticity is often the result of non-normality of one of the variables.
• Linearity: the idea that there is a linear relationship between the independent and dependent
variables.
• Absence of correlated errors: assumption that any prediction errors are uncorrelated with
each other.
0.14 Data transformations
Data transformations alter the characteristics of the original variable. If we want to achieve
normality/homoscedasticity we can use transformations. If the distribution is flat, the most common
transformation is the inverse. For skewed distributions the variable can be transformed by taking the
square root, logarithms, squared, or cubed, or even the inverse.
We can also use data transformations for interpretation reasons.
• Standardization: Transforming data by subtracting the variable mean from each value and
dividing the result by the variable's standard deviation. This process results in a dataset with a
mean of zero and a standard deviation of 1. Commonly achieved using z-scores.
o Z-score of 0 indicates the value equals the mean.
o Positive or negative z-scores represent deviations above or below the mean, measured
in standard deviations (a z-score of 0.8 is 0.8 standard deviations above the mean).
Standardization allows for the direct comparison of variables with different distributions.
• Centering: Related to standardization. Involves subtracting the mean value from each
observation, making values relative to their mean without dividing by the standard deviation.
o Variable-based centering: commonly used in multiple regression to improve the
interpretation of moderation and interaction effects, and potentially reduce
multicollinearity.
4
