Introduction to Research in Marketing
Hoofdstuk 1 - Data exploration and visualization
Total error framework: Next to true value, chances are that there are several types of errors present:
Sampling error
Measurement error
Statistical error
Note: take these errors into account, otherwise your results will be biased and your
recommendations will be wrong!
Sampling error
Population: the whole group of people/companies/events/object over which you want to conclude
something
Sample: a subgroup of the population, in order to generalize the results to the population.
There are different types of sampling errors:
Coverage error: frame population ≠ population
Population: voters of a certain election
Frame population: everyone with a telephone
Sample error: the method of sampling may cause errors, for example choosing random digits
Non-response error: not everyone in the sample population accepts the call (big error!)
Eventually, chances are that the sample respondents differ significantly from the population
How to deal with this in practice?
Post-stratification weights: make your sample closer to your population by using the correct weights
Population is 50% female, but your sample is 80% female.
Simple average: 0.2 x 4.2 + 0.8 x 3.4 = 3.56
Females are overrepresented
Weighted average: 0.5 x 4.2 + 0.5 x 3.4 = 3.8
Example: 20% of the population is highly educated, while 70% of the sample is highly educated
It is important to compare the characteristics (gender/education/etc) of the population to those of
the sample
Measurement error
Measurement scales
Non-metric Metric (continuous)
Nomina Numbers only serve as a Interva Relative positions are
l label/tag l comparable
Mutually exclusive: not at the No natural zero point
same time → Likert scale, temperature (Celsius)
Collectively exhaustive: at least Ratio Most precise scale
1
, one of the categories Absolute zero point (cannot
→ Gender, SNR be negative)
Ordinal Numbers are assigned to → Weight, height, age, temperature
indicate relative position, but (Kelvin)
not the magnitude
→ Gold, Silver, Bronze
Outcomes measure direction, and intensity as well
(strongly agree/somewhat agree)
Outcomes can only measure the direction of the
response (e.g. yes/no)
Note: The right statistical technique depends on what scale is used (metric/non-metric)
Summated scales: abstract concepts (attitudes/beliefs) are measured using more than one question.
Validity: Does it measure what it is supposed to?
→ Do the 8 questions measure the concept of frugality?
Reliability: Do you get the same results every time?
Ideally, you want a measure that is both reliable and valid.
How to deal with this in practice?
Face validity: Do the coefficients make sense? (Effect sizes and signs)
Reliability: How much do these results change if:
Additional control variables are added
Some observations are taken away (outliers)
The model is estimated on a new dataset
Statistical error
Two possible outcomes of hypothesis testing:
H0 is true
Alternative is true
Example:
H0: Not pregnant
H1: Pregnant
Two types of errors:
Type 1: false positive (H0 is incorrectly rejected)
→ You’re pregnant (to a man)
Type 2: false negative (H0 is incorrectly accepted)
→ You’re not pregnant (to a pregnant woman)
P-value: probability of observed data/statistic, given that the null hypothesis is true
Low p-value: data are unlikely according to the null → Reject H0 (Low chance of Type 1 error)
Typically, threshold (α) is set at 0.05 → Reject H0 if p-value < α
2
, In practice: P-value is misused and overused.
→ Consider p-values along with interpretation, power, measurement, sampling and summaries
Exploratory data analysis
Preparation
- Always explore your data before running any model!
→ Missing observations, mutually consistent, valid, etcetera
Visualization
Purpose: exploration of data, understanding data and communicating the results
Hoofdstuk 2 – ANOVA
Steps:
1. Defining the objectives
Objective: to test if there are differences in the mean of a metric (interval/ratio) dependent variable
across different levels of one/more non-metric (nominal/ordinal) independent variables (factors).
One-way ANOVA: 1 factor, 2+ levels
→ The difference of the type of ad (skinny/average/plussize) on the appeal
T-test: a special case of ANOVA, when there are only 2 levels within 1 IV.
→ The difference of the type of ad (naked/not naked) on the appeal
Two-way ANOVA: When there is >1 independent variable
→ The difference of the type of poster (1/2/3) on the appeal, and to what extent does the difference
depend on gender (male/female)?
2 factors, so two-way ANOVA
3x2 ANOVA
2. Designing the ANOVA
2.1 Interactions
Interaction: the effect of one variable on the DV is dependent on
another (moderator)
→ Interaction between poster and gender?
2.2 Covariates
Covariates/control variables: affect DV separately from the
treatment variables
If not included → biased results!
Requirements control variables:
1. Pre-measure: Independent of treatment
2. Limited number
3
Hoofdstuk 1 - Data exploration and visualization
Total error framework: Next to true value, chances are that there are several types of errors present:
Sampling error
Measurement error
Statistical error
Note: take these errors into account, otherwise your results will be biased and your
recommendations will be wrong!
Sampling error
Population: the whole group of people/companies/events/object over which you want to conclude
something
Sample: a subgroup of the population, in order to generalize the results to the population.
There are different types of sampling errors:
Coverage error: frame population ≠ population
Population: voters of a certain election
Frame population: everyone with a telephone
Sample error: the method of sampling may cause errors, for example choosing random digits
Non-response error: not everyone in the sample population accepts the call (big error!)
Eventually, chances are that the sample respondents differ significantly from the population
How to deal with this in practice?
Post-stratification weights: make your sample closer to your population by using the correct weights
Population is 50% female, but your sample is 80% female.
Simple average: 0.2 x 4.2 + 0.8 x 3.4 = 3.56
Females are overrepresented
Weighted average: 0.5 x 4.2 + 0.5 x 3.4 = 3.8
Example: 20% of the population is highly educated, while 70% of the sample is highly educated
It is important to compare the characteristics (gender/education/etc) of the population to those of
the sample
Measurement error
Measurement scales
Non-metric Metric (continuous)
Nomina Numbers only serve as a Interva Relative positions are
l label/tag l comparable
Mutually exclusive: not at the No natural zero point
same time → Likert scale, temperature (Celsius)
Collectively exhaustive: at least Ratio Most precise scale
1
, one of the categories Absolute zero point (cannot
→ Gender, SNR be negative)
Ordinal Numbers are assigned to → Weight, height, age, temperature
indicate relative position, but (Kelvin)
not the magnitude
→ Gold, Silver, Bronze
Outcomes measure direction, and intensity as well
(strongly agree/somewhat agree)
Outcomes can only measure the direction of the
response (e.g. yes/no)
Note: The right statistical technique depends on what scale is used (metric/non-metric)
Summated scales: abstract concepts (attitudes/beliefs) are measured using more than one question.
Validity: Does it measure what it is supposed to?
→ Do the 8 questions measure the concept of frugality?
Reliability: Do you get the same results every time?
Ideally, you want a measure that is both reliable and valid.
How to deal with this in practice?
Face validity: Do the coefficients make sense? (Effect sizes and signs)
Reliability: How much do these results change if:
Additional control variables are added
Some observations are taken away (outliers)
The model is estimated on a new dataset
Statistical error
Two possible outcomes of hypothesis testing:
H0 is true
Alternative is true
Example:
H0: Not pregnant
H1: Pregnant
Two types of errors:
Type 1: false positive (H0 is incorrectly rejected)
→ You’re pregnant (to a man)
Type 2: false negative (H0 is incorrectly accepted)
→ You’re not pregnant (to a pregnant woman)
P-value: probability of observed data/statistic, given that the null hypothesis is true
Low p-value: data are unlikely according to the null → Reject H0 (Low chance of Type 1 error)
Typically, threshold (α) is set at 0.05 → Reject H0 if p-value < α
2
, In practice: P-value is misused and overused.
→ Consider p-values along with interpretation, power, measurement, sampling and summaries
Exploratory data analysis
Preparation
- Always explore your data before running any model!
→ Missing observations, mutually consistent, valid, etcetera
Visualization
Purpose: exploration of data, understanding data and communicating the results
Hoofdstuk 2 – ANOVA
Steps:
1. Defining the objectives
Objective: to test if there are differences in the mean of a metric (interval/ratio) dependent variable
across different levels of one/more non-metric (nominal/ordinal) independent variables (factors).
One-way ANOVA: 1 factor, 2+ levels
→ The difference of the type of ad (skinny/average/plussize) on the appeal
T-test: a special case of ANOVA, when there are only 2 levels within 1 IV.
→ The difference of the type of ad (naked/not naked) on the appeal
Two-way ANOVA: When there is >1 independent variable
→ The difference of the type of poster (1/2/3) on the appeal, and to what extent does the difference
depend on gender (male/female)?
2 factors, so two-way ANOVA
3x2 ANOVA
2. Designing the ANOVA
2.1 Interactions
Interaction: the effect of one variable on the DV is dependent on
another (moderator)
→ Interaction between poster and gender?
2.2 Covariates
Covariates/control variables: affect DV separately from the
treatment variables
If not included → biased results!
Requirements control variables:
1. Pre-measure: Independent of treatment
2. Limited number
3
