P4 - Experimental Research
Index
Week 1 .......................................................................................................................................................... 3
Preparation – Leary: chapter 1 ..........................................................................................................................3
Lecture 1 – Introduction to experimenting.........................................................................................................5
1.1 Why experimental research? ..................................................................................................................5
1.2 Why experimenting? ...............................................................................................................................6
1.3 Experimental research in marketing .......................................................................................................6
1.4 Correlation & prediction .........................................................................................................................7
1.5 What is experimental research? .............................................................................................................8
1.6 Problem identification – for research questions .....................................................................................9
Preparation – Leary: chapter 2 ........................................................................................................................10
Preparation – Leary: chapter 3 ........................................................................................................................12
Lecture 2 – Experimental Designs ....................................................................................................................15
2.1 Conceptual models ................................................................................................................................15
2.2 Experimental Designs ............................................................................................................................17
2.3 Manipulating the independent variable................................................................................................18
2.4 Reliability vs. Validity.............................................................................................................................21
2.5 Measurement of Dependent Variable ..................................................................................................22
Week 2 ........................................................................................................................................................ 23
Preparation – Leary: chapter 10 ......................................................................................................................23
Preparation – Leary: chapter 11 ......................................................................................................................25
Preparation - Assignment Threats to Internal Validity.....................................................................................27
Lecture 3 – Confounds & ANOVA .....................................................................................................................28
3.1 Internal validity .....................................................................................................................................28
3.2 Threats to internal validity ....................................................................................................................28
3.3 Hypothesis testing .................................................................................................................................30
3.4 Intuition behind ANOVA ........................................................................................................................31
Preparation – Leary: chapter 12 ......................................................................................................................41
Lecture 4 – One-way ANOVA & Two-way ANOVA ...........................................................................................44
4.1 One-way ANOVA ...................................................................................................................................44
4.2 Two-way ANOVA ...................................................................................................................................49
4.3 Two-way ANOVA - How it’s done ..........................................................................................................51
Week 3 ........................................................................................................................................................ 58
Lecture 5 – 3-way ANOVA & ANCOVA .............................................................................................................58
5.1 Practice interactions..............................................................................................................................58
5.2 Three-way ANOVA.................................................................................................................................60
5.3 ANCOVA ................................................................................................................................................65
Lecture 6 – Repeated Measures Mixed Design ................................................................................................70
6.1 Repeated Measures ..............................................................................................................................71
6.2 Mixed Design .........................................................................................................................................78
Week 4 ........................................................................................................................................................ 84
, Lecture 7 - Mediation .......................................................................................................................................84
7.1 Mediation analysis ................................................................................................................................84
7.2 Mediation analysis SPSS using regression .............................................................................................88
7.3 Mediation analysis using process ..........................................................................................................90
7.4 Mediation by experimentation .............................................................................................................95
Preparation – Leary: chapter 9 ........................................................................................................................97
Lecture 8 – Power and Effect Size ..................................................................................................................100
8.1 Hypothesis testing ...............................................................................................................................100
8.2 Power ..................................................................................................................................................101
8.3 Increasing power through sample size (power analysis).....................................................................102
8.4 Increasing power through effect size (improving your design) ...........................................................104
Week 5 .......................................................................................................................................................110
Lecture 9 – Moderation with continuous variables – Spotlight Analysis ........................................................110
9.1 Moderation with continuous variables ...............................................................................................110
9.2 Regression analysis..............................................................................................................................111
9.3 Spotlight analysis .................................................................................................................................119
Lecture 10 ......................................................................................................................................................128
10.1 Replication crisis ................................................................................................................................128
10.2 Flexible data analysis .........................................................................................................................129
10.3 Confound/Covariate revisited ...........................................................................................................130
COURSE ASSESSMENTS
§ 65% for exam
§ 20% for weekly individual assignments
§ 15% for group assignment presentation on March 14th
2
,Week 1
Preparation – Leary: chapter 1
Research in the Behavioral Sciences
1.2 Goals of Behavioral Research
Behavioral research aims to:
§ Describe Behavior – Observing and documenting patterns (e.g., surveys, observational studies).
§ Predict Behavior – Identifying factors that indicate future behaviors (e.g., employment tests
predicting job performance).
§ Explain Behavior – Understanding why behaviors occur using theories and hypotheses.
1.3 Behavioral Science and Common Sense
Science vs. Common Sense: While some psychological findings align with common beliefs, others contradict
popular assumptions → research provides evidence-based conclusions
1.7 Developing Research Hypotheses
§ Deduction = Start with a theory, then derive testable predictions.
§ Induction = Observe patterns, then formulate a general theory.
1.9 Scientific Progress and Replication
Scientific knowledge evolves through:
§ Proof and disproof: Findings are tentative and subject to revision.
§ Replication: Studies must be repeated to confirm reliability.
§ Scientific filter: Poorly conducted studies are discarded over time.
1.10 Strategies of Behavioral Research
Behavioral scientists use four major research strategies:
§ Descriptive Research – Observing and summarizing behavior.
§ Correlational Research – Measuring relationships between variables.
§ Experimental Research – Manipulating variables to establish causality.
§ Quasi-Experimental Research – Studying cause-and-effect when random assignment isn’t possible.
3
,Key terms – Chapter 1
- Applied Research – Research aimed at solving practical problems.
- A Priori Prediction – A hypothesis stated before data collection.
- Basic Research – Research aimed at expanding general knowledge.
- Close Replication – A study that closely follows the original experiment.
- Conceptual Definition – The meaning of a concept at a theoretical level.
- Conceptual Replication – A study that tests the same idea with different methods.
- Correlational Research – Research assessing relationships between variables.
- Deduction – Logical reasoning from general to specific.
- Descriptive Research – Research aimed at describing phenomena.
- Direct Replication – Repeating an experiment exactly as it was done.
- Empirical Generalization – A pattern observed in empirical findings.
- Empiricism – The idea that knowledge comes from experience.
- Evaluation Research – Research assessing program effectiveness.
- Experimental Research – Research manipulating variables to establish cause.
- Falsification – The principle that theories should be testable and refutable.
- File-Drawer Problem – The tendency for non-significant results to remain unpublished.
- Hypothesis – A testable prediction.
- Induction – Reasoning from specific observations to general theories.
- Methodological Pluralism – Using multiple research methods.
- Model – A simplified representation of a theory.
- Null Finding – A result showing no significant effect.
- Operational Definition – How a concept is measured in a study.
- Post Hoc Explanation – An explanation developed after results are known.
- Pseudoscience – Claims that lack empirical support.
- Public Verification – The requirement for findings to be openly shared.
- Quasi-Experimental Research – Research without full experimental control.
- Registered Report – A study reviewed before data collection.
- Strategy of Strong Inference – Designing research to test multiple competing theories.
- Theory – A broad explanation based on empirical evidence.
4
,Lecture 1 – Introduction to experimenting
1.1 Why experimental research?
Example: Nudging healthy choice & disclosure
As we can see in the diagram, placing cookies did not make a difference. However, for fruit, the nudge and
disclosure had a significant impact. Question: Is this conclusion correct based on the data? (Hint: Look at the
baseline bar on the fruit side.)
Answer: This graph shows how nudging and disclosure influence food choices:
Cookies: Nudging slightly reduces consumption (132 vs. 142), but disclosure (137) has little additional effect.
Fruit: Nudging significantly increases consumption (156 vs. 89), with disclosure boosting it further (164).
→ Key takeaway = nudging works for healthy choices (= fruit) but not for unhealthy ones (= cookies), and we
should compare behavior shifts rather than just absolute numbers
Types of marketing research
5
, 1.2 Why experimenting?
In marketing (and other sciences) we seek to:
§ Describe
§ Predict
§ Explain behavior of (market-parties, employees, buyers)
→ Break up phenomena in variables and relations between those variables
1.3 Experimental research in marketing
Examples: A-B testing
Is this a good A-B test?
No, it’s not good. Because they changed
so much the image, the text, the button.
You don’t know what are the elements
that are effective and less effective.
That’s where it goes wrong a lot.
Conversion optimalisation
§ A structured and systematic approach to improving the performance of a website
§ Informed by data insights & psychology
§ Taking the traffic there and making the most of it
Experimenting & consumer insights
When marketing to consumers, it's important to
consider high-frequency vs. low-frequency exercisers.
Should product claims focus on physical benefits like
improving endurance and recovery or on mental
benefits like boosting alertness and making you feel
fit?
You hear these types of claims all the time—whether
in radio ads or on product packaging. But which works
better? That’s what we test in consumer research.
6
Index
Week 1 .......................................................................................................................................................... 3
Preparation – Leary: chapter 1 ..........................................................................................................................3
Lecture 1 – Introduction to experimenting.........................................................................................................5
1.1 Why experimental research? ..................................................................................................................5
1.2 Why experimenting? ...............................................................................................................................6
1.3 Experimental research in marketing .......................................................................................................6
1.4 Correlation & prediction .........................................................................................................................7
1.5 What is experimental research? .............................................................................................................8
1.6 Problem identification – for research questions .....................................................................................9
Preparation – Leary: chapter 2 ........................................................................................................................10
Preparation – Leary: chapter 3 ........................................................................................................................12
Lecture 2 – Experimental Designs ....................................................................................................................15
2.1 Conceptual models ................................................................................................................................15
2.2 Experimental Designs ............................................................................................................................17
2.3 Manipulating the independent variable................................................................................................18
2.4 Reliability vs. Validity.............................................................................................................................21
2.5 Measurement of Dependent Variable ..................................................................................................22
Week 2 ........................................................................................................................................................ 23
Preparation – Leary: chapter 10 ......................................................................................................................23
Preparation – Leary: chapter 11 ......................................................................................................................25
Preparation - Assignment Threats to Internal Validity.....................................................................................27
Lecture 3 – Confounds & ANOVA .....................................................................................................................28
3.1 Internal validity .....................................................................................................................................28
3.2 Threats to internal validity ....................................................................................................................28
3.3 Hypothesis testing .................................................................................................................................30
3.4 Intuition behind ANOVA ........................................................................................................................31
Preparation – Leary: chapter 12 ......................................................................................................................41
Lecture 4 – One-way ANOVA & Two-way ANOVA ...........................................................................................44
4.1 One-way ANOVA ...................................................................................................................................44
4.2 Two-way ANOVA ...................................................................................................................................49
4.3 Two-way ANOVA - How it’s done ..........................................................................................................51
Week 3 ........................................................................................................................................................ 58
Lecture 5 – 3-way ANOVA & ANCOVA .............................................................................................................58
5.1 Practice interactions..............................................................................................................................58
5.2 Three-way ANOVA.................................................................................................................................60
5.3 ANCOVA ................................................................................................................................................65
Lecture 6 – Repeated Measures Mixed Design ................................................................................................70
6.1 Repeated Measures ..............................................................................................................................71
6.2 Mixed Design .........................................................................................................................................78
Week 4 ........................................................................................................................................................ 84
, Lecture 7 - Mediation .......................................................................................................................................84
7.1 Mediation analysis ................................................................................................................................84
7.2 Mediation analysis SPSS using regression .............................................................................................88
7.3 Mediation analysis using process ..........................................................................................................90
7.4 Mediation by experimentation .............................................................................................................95
Preparation – Leary: chapter 9 ........................................................................................................................97
Lecture 8 – Power and Effect Size ..................................................................................................................100
8.1 Hypothesis testing ...............................................................................................................................100
8.2 Power ..................................................................................................................................................101
8.3 Increasing power through sample size (power analysis).....................................................................102
8.4 Increasing power through effect size (improving your design) ...........................................................104
Week 5 .......................................................................................................................................................110
Lecture 9 – Moderation with continuous variables – Spotlight Analysis ........................................................110
9.1 Moderation with continuous variables ...............................................................................................110
9.2 Regression analysis..............................................................................................................................111
9.3 Spotlight analysis .................................................................................................................................119
Lecture 10 ......................................................................................................................................................128
10.1 Replication crisis ................................................................................................................................128
10.2 Flexible data analysis .........................................................................................................................129
10.3 Confound/Covariate revisited ...........................................................................................................130
COURSE ASSESSMENTS
§ 65% for exam
§ 20% for weekly individual assignments
§ 15% for group assignment presentation on March 14th
2
,Week 1
Preparation – Leary: chapter 1
Research in the Behavioral Sciences
1.2 Goals of Behavioral Research
Behavioral research aims to:
§ Describe Behavior – Observing and documenting patterns (e.g., surveys, observational studies).
§ Predict Behavior – Identifying factors that indicate future behaviors (e.g., employment tests
predicting job performance).
§ Explain Behavior – Understanding why behaviors occur using theories and hypotheses.
1.3 Behavioral Science and Common Sense
Science vs. Common Sense: While some psychological findings align with common beliefs, others contradict
popular assumptions → research provides evidence-based conclusions
1.7 Developing Research Hypotheses
§ Deduction = Start with a theory, then derive testable predictions.
§ Induction = Observe patterns, then formulate a general theory.
1.9 Scientific Progress and Replication
Scientific knowledge evolves through:
§ Proof and disproof: Findings are tentative and subject to revision.
§ Replication: Studies must be repeated to confirm reliability.
§ Scientific filter: Poorly conducted studies are discarded over time.
1.10 Strategies of Behavioral Research
Behavioral scientists use four major research strategies:
§ Descriptive Research – Observing and summarizing behavior.
§ Correlational Research – Measuring relationships between variables.
§ Experimental Research – Manipulating variables to establish causality.
§ Quasi-Experimental Research – Studying cause-and-effect when random assignment isn’t possible.
3
,Key terms – Chapter 1
- Applied Research – Research aimed at solving practical problems.
- A Priori Prediction – A hypothesis stated before data collection.
- Basic Research – Research aimed at expanding general knowledge.
- Close Replication – A study that closely follows the original experiment.
- Conceptual Definition – The meaning of a concept at a theoretical level.
- Conceptual Replication – A study that tests the same idea with different methods.
- Correlational Research – Research assessing relationships between variables.
- Deduction – Logical reasoning from general to specific.
- Descriptive Research – Research aimed at describing phenomena.
- Direct Replication – Repeating an experiment exactly as it was done.
- Empirical Generalization – A pattern observed in empirical findings.
- Empiricism – The idea that knowledge comes from experience.
- Evaluation Research – Research assessing program effectiveness.
- Experimental Research – Research manipulating variables to establish cause.
- Falsification – The principle that theories should be testable and refutable.
- File-Drawer Problem – The tendency for non-significant results to remain unpublished.
- Hypothesis – A testable prediction.
- Induction – Reasoning from specific observations to general theories.
- Methodological Pluralism – Using multiple research methods.
- Model – A simplified representation of a theory.
- Null Finding – A result showing no significant effect.
- Operational Definition – How a concept is measured in a study.
- Post Hoc Explanation – An explanation developed after results are known.
- Pseudoscience – Claims that lack empirical support.
- Public Verification – The requirement for findings to be openly shared.
- Quasi-Experimental Research – Research without full experimental control.
- Registered Report – A study reviewed before data collection.
- Strategy of Strong Inference – Designing research to test multiple competing theories.
- Theory – A broad explanation based on empirical evidence.
4
,Lecture 1 – Introduction to experimenting
1.1 Why experimental research?
Example: Nudging healthy choice & disclosure
As we can see in the diagram, placing cookies did not make a difference. However, for fruit, the nudge and
disclosure had a significant impact. Question: Is this conclusion correct based on the data? (Hint: Look at the
baseline bar on the fruit side.)
Answer: This graph shows how nudging and disclosure influence food choices:
Cookies: Nudging slightly reduces consumption (132 vs. 142), but disclosure (137) has little additional effect.
Fruit: Nudging significantly increases consumption (156 vs. 89), with disclosure boosting it further (164).
→ Key takeaway = nudging works for healthy choices (= fruit) but not for unhealthy ones (= cookies), and we
should compare behavior shifts rather than just absolute numbers
Types of marketing research
5
, 1.2 Why experimenting?
In marketing (and other sciences) we seek to:
§ Describe
§ Predict
§ Explain behavior of (market-parties, employees, buyers)
→ Break up phenomena in variables and relations between those variables
1.3 Experimental research in marketing
Examples: A-B testing
Is this a good A-B test?
No, it’s not good. Because they changed
so much the image, the text, the button.
You don’t know what are the elements
that are effective and less effective.
That’s where it goes wrong a lot.
Conversion optimalisation
§ A structured and systematic approach to improving the performance of a website
§ Informed by data insights & psychology
§ Taking the traffic there and making the most of it
Experimenting & consumer insights
When marketing to consumers, it's important to
consider high-frequency vs. low-frequency exercisers.
Should product claims focus on physical benefits like
improving endurance and recovery or on mental
benefits like boosting alertness and making you feel
fit?
You hear these types of claims all the time—whether
in radio ads or on product packaging. But which works
better? That’s what we test in consumer research.
6
