SMM494 Analytical Methods for Marketing Week 2 Lecture Notes
Why We Care About Data
Organizations record information as they operate.
This information may be a source of insights relevant for decision making.
Variables that represent human behaviour are connected to other variables,
and these correlations are captured by the data.
We can also predict variables from other variables.
Caveats
o The higher the number of variables we examine, the higher the chance
of a spurious effect.
o Correlation does not imply causation
Even if two variables co-vary, changing one does not
necessarily produce a change in the other.
Correlation is NOT Causation
Correlation tell us that two variables are related but it does not provide an
explanation of why they co-vary.
o It could be sheer coincidence. (Consistency)
o It could be that one factor leads to another. (Responsiveness)
o It could be that there is an external factor affecting this. (No Plausible
Alternative Explanation)
Randomised experiments are the gold standard for establishing causality.
There are many reasons why we might find a correlation between an outcome
and a predictor.
Counterfactual reasoning provides a powerful lens for thinking about these
questions.
Experiments are Causal Tests
To claim that a marketing action has a positive effect on a market outcome, it
needs to be observed that:
o Taking the action increases the outcome.
o Not taking action does not increase the outcome.
o The increase in the outcome is observed after the action is taken.
o There are no external factors to which the increase in the outcome can
be attributed.
Experiments are the most suitable methods to test causality
o Moving from predictive to prescriptive analytics.
Four Important Attributes of Randomised Experiments
o The researcher actively intervenes in the system, actually changing X
(the treatment) and seeing what happens to Y (manipulation of X).
o Because of random assignment, groups are guaranteed to be initially
equivalent, on average, on all observable (and unobservable)
characteristics.
, o The control group provides the ideal counterfactual —our best estimate
of what the treatment group would have looked like if it didn’t receive
the treatment.
o Any difference found in Y must be due to the changing of X (the
treatment) because there’s no other plausible explanation.
Designing an Experiment
Step 1: Formulating a Hypothesis
Step 2: Set up the experiment
o Define manipulation and treatment groups
o Define units of observation
o Set up randomisation or matching
Step 3: Collect Data
Step 4: Evaluate Results
Randomisation
o This is a key aspect of an experiment.
o It ensures that participants in the control and in the treatment groups
are similar.
o This is done by allocating participants randomly between the
experiment conditions.
o If randomisation is not feasible, or groups are not large enough, there
is a need for alternative ways to make groups comparable.
o Matching
Participants are assigned to the conditions based on their
matched levels of relevant attributes.
A before/after experiment may be useful to ensure the
comparability of groups.
A/B Testing
These are experiments in digital contexts
A critical part of digital marketing
o Easy to customise and send individualised messages
o Easy to measure outcomes.
Each participant (visitor) is randomly shown one of different variations of a
message.
A metric of interest is measured and tested across the different variations.
Results are examined.
Examples of applications
o E-Mail Marketing
Testable Features: Subject line, discounts and promotions,
almost endless features of the creative (image, font, colors),
time and day sent.
Measurable Outcomes: Open and click-through (intermediate
variables), products viewed, sales and subscriptions, time on
site, unsubscribing.
Why We Care About Data
Organizations record information as they operate.
This information may be a source of insights relevant for decision making.
Variables that represent human behaviour are connected to other variables,
and these correlations are captured by the data.
We can also predict variables from other variables.
Caveats
o The higher the number of variables we examine, the higher the chance
of a spurious effect.
o Correlation does not imply causation
Even if two variables co-vary, changing one does not
necessarily produce a change in the other.
Correlation is NOT Causation
Correlation tell us that two variables are related but it does not provide an
explanation of why they co-vary.
o It could be sheer coincidence. (Consistency)
o It could be that one factor leads to another. (Responsiveness)
o It could be that there is an external factor affecting this. (No Plausible
Alternative Explanation)
Randomised experiments are the gold standard for establishing causality.
There are many reasons why we might find a correlation between an outcome
and a predictor.
Counterfactual reasoning provides a powerful lens for thinking about these
questions.
Experiments are Causal Tests
To claim that a marketing action has a positive effect on a market outcome, it
needs to be observed that:
o Taking the action increases the outcome.
o Not taking action does not increase the outcome.
o The increase in the outcome is observed after the action is taken.
o There are no external factors to which the increase in the outcome can
be attributed.
Experiments are the most suitable methods to test causality
o Moving from predictive to prescriptive analytics.
Four Important Attributes of Randomised Experiments
o The researcher actively intervenes in the system, actually changing X
(the treatment) and seeing what happens to Y (manipulation of X).
o Because of random assignment, groups are guaranteed to be initially
equivalent, on average, on all observable (and unobservable)
characteristics.
, o The control group provides the ideal counterfactual —our best estimate
of what the treatment group would have looked like if it didn’t receive
the treatment.
o Any difference found in Y must be due to the changing of X (the
treatment) because there’s no other plausible explanation.
Designing an Experiment
Step 1: Formulating a Hypothesis
Step 2: Set up the experiment
o Define manipulation and treatment groups
o Define units of observation
o Set up randomisation or matching
Step 3: Collect Data
Step 4: Evaluate Results
Randomisation
o This is a key aspect of an experiment.
o It ensures that participants in the control and in the treatment groups
are similar.
o This is done by allocating participants randomly between the
experiment conditions.
o If randomisation is not feasible, or groups are not large enough, there
is a need for alternative ways to make groups comparable.
o Matching
Participants are assigned to the conditions based on their
matched levels of relevant attributes.
A before/after experiment may be useful to ensure the
comparability of groups.
A/B Testing
These are experiments in digital contexts
A critical part of digital marketing
o Easy to customise and send individualised messages
o Easy to measure outcomes.
Each participant (visitor) is randomly shown one of different variations of a
message.
A metric of interest is measured and tested across the different variations.
Results are examined.
Examples of applications
o E-Mail Marketing
Testable Features: Subject line, discounts and promotions,
almost endless features of the creative (image, font, colors),
time and day sent.
Measurable Outcomes: Open and click-through (intermediate
variables), products viewed, sales and subscriptions, time on
site, unsubscribing.
