Table of Contents
Note that we skip certain parts, as these were just introduction to, for example, Simbound.
More important, and thus included, are the debriefings.
Lecture 1A: Course Intro, Intro to Display Ads................................................................... 2
1.1 The Basics of Online Display Ads................................................................................ 2
1.2 Targeting Online........................................................................................................... 4
1.3 Takeaways....................................................................................................................5
Lecture 1B: Campaigns KPIs................................................................................................ 6
1.4 KPIs & Business Objectives......................................................................................... 7
1.5 Campaign Metrics: Advantages & Disadvantages....................................................... 7
1.6 Ad blocking.................................................................................................................10
1.7 Are experiments needed to measure ad effectiveness?............................................ 10
1.8 Takeaways..................................................................................................................11
Lecture 2A: Lee Hower Case Study....................................................................................12
2.1 Lee Hower Marketing Analytics’ team calculation...................................................... 12
2.2 Yahoo! Data Science Team Calculation..................................................................... 14
2.3 What Would You Do Without Red Cross Ads?...........................................................15
2.4 Statistics of difference estimators...............................................................................16
2.5 Experimental Analysis: Step by Step......................................................................... 18
2.6 Comparing TOT and ITT estimates............................................................................21
Lecture 2B: Intro to Sponsored Search & SEO................................................................. 22
2.5 Introduction to Sponsored Search..............................................................................22
2.6 How Paid Search Works............................................................................................ 23
2.7 SEO............................................................................................................................28
2.8 Website evaluation..................................................................................................... 30
2.9 Takeaways..................................................................................................................31
Lecture 3B: Lessons From Advertising Research............................................................32
3.1 Display Ad Effectiveness............................................................................................32
3.2 Lessons From 432 Ghost Ad Experiments.................................................................34
3.3 Do We Need Experiments To Measure Ad Effectiveness?........................................ 35
3.4 Takeaways..................................................................................................................36
3.4 Branded Sponsored Search: eBay Experiments........................................................37
3.5 What about the Rest of Branded Search?..................................................................37
3.7 Sponsored Search Takeaways...................................................................................40
Lecture 4A: Adaptive Experimentation and Website Morphing Guest Lecture............. 41
4.1 Adaptive Experimentation: Learning while Earning....................................................41
4.2 Are we all the Same?................................................................................................. 43
Lecture 4B: A/B Test Simulation.........................................................................................45
4.3 The Basics of A/B Testing.......................................................................................... 45
4.4 Benefits of A/B Testing............................................................................................... 46
Lecture 5A: eCommerce......................................................................................................47
5.1 E-commerce Basics................................................................................................... 47
5.2 On-site Marketing....................................................................................................... 48
1
, 5.3 On-site Marketing: Site Design and Optimization.......................................................49
5.4 On-site Marketing: Pop-overs.....................................................................................49
5.4 On-site Marketing: Pricing.......................................................................................... 50
5.5 Takeaways..................................................................................................................52
Lecture 5B: AB Test Debrief and Reviews and Social Care............................................. 53
5.6 Bandit Optimization in Online Decisions.................................................................... 53
5.7 Using A/B Testing and ML for Decision-Making......................................................... 54
5.8 Reviews, Ratings, and User-Generated Content....................................................... 55
5.9 Social Care.................................................................................................................62
Lecture 6A: Guest Lecture + Social Media Analytics....................................................... 63
6.1 Principles of Community Management.......................................................................63
6.2 Monetization............................................................................................................... 64
6.3 Community Building in a Digital World....................................................................... 64
6.4 Online Communities and Online Reviews.................................................................. 65
6.5 Step 1: Set Goals....................................................................................................... 66
6.6 Step 2: Social Media Metrics......................................................................................66
6.7 Step 3: Measuring ROI...............................................................................................67
6.8 Examples of Analytics................................................................................................ 68
6.9 Virality.........................................................................................................................71
6.10 Takeaways................................................................................................................72
Lecture 6B: Content Marketing........................................................................................... 73
6.11 Influencer Marketing................................................................................................. 73
6.12 Content Marketing Primer........................................................................................ 74
6.13 Step 1: Research and planning................................................................................ 76
6.14 Step 2: Content Creation..........................................................................................77
6.15 Step 3: Content Diffusion......................................................................................... 78
6.16 Takeaways................................................................................................................80
Lecture 7A: Guest Lecture + Privacy and AI..................................................................... 81
7.1 Attribution Modelling in Online Marketing...................................................................81
7.2 Privacy and a “Cookie-Less World”............................................................................83
7.3 Online Behavioural Advertising.................................................................................. 84
7.4 Privacy Regulation in US vs EU (GDPR)................................................................... 86
7.5 Takeaways (1)............................................................................................................ 87
7.6 AI tools....................................................................................................................... 87
7.7 Takeaways (2)............................................................................................................ 88
Lecture 7B: Wrap Up and Q&A........................................................................................... 89
7.9 Course Takeaways..................................................................................................... 89
Quiz Answers....................................................................................................................... 92
Campaign KPIs quiz.........................................................................................................92
Content marketing and creator economy......................................................................... 92
Display ad quiz................................................................................................................. 92
Ecommerce...................................................................................................................... 92
Lee Hower........................................................................................................................ 93
Lessons from ad research................................................................................................93
2
, Privacy and AI.................................................................................................................. 93
Reviews/Social Care........................................................................................................ 93
Social media Analytics..................................................................................................... 94
Sponsored Search and Search Engine Optimization (SEO)............................................ 94
Summary Quiz 2023.........................................................................................................94
3
Note that we skip certain parts, as these were just introduction to, for example, Simbound.
More important, and thus included, are the debriefings.
Lecture 1A: Course Intro, Intro to Display Ads................................................................... 2
1.1 The Basics of Online Display Ads................................................................................ 2
1.2 Targeting Online........................................................................................................... 4
1.3 Takeaways....................................................................................................................5
Lecture 1B: Campaigns KPIs................................................................................................ 6
1.4 KPIs & Business Objectives......................................................................................... 7
1.5 Campaign Metrics: Advantages & Disadvantages....................................................... 7
1.6 Ad blocking.................................................................................................................10
1.7 Are experiments needed to measure ad effectiveness?............................................ 10
1.8 Takeaways..................................................................................................................11
Lecture 2A: Lee Hower Case Study....................................................................................12
2.1 Lee Hower Marketing Analytics’ team calculation...................................................... 12
2.2 Yahoo! Data Science Team Calculation..................................................................... 14
2.3 What Would You Do Without Red Cross Ads?...........................................................15
2.4 Statistics of difference estimators...............................................................................16
2.5 Experimental Analysis: Step by Step......................................................................... 18
2.6 Comparing TOT and ITT estimates............................................................................21
Lecture 2B: Intro to Sponsored Search & SEO................................................................. 22
2.5 Introduction to Sponsored Search..............................................................................22
2.6 How Paid Search Works............................................................................................ 23
2.7 SEO............................................................................................................................28
2.8 Website evaluation..................................................................................................... 30
2.9 Takeaways..................................................................................................................31
Lecture 3B: Lessons From Advertising Research............................................................32
3.1 Display Ad Effectiveness............................................................................................32
3.2 Lessons From 432 Ghost Ad Experiments.................................................................34
3.3 Do We Need Experiments To Measure Ad Effectiveness?........................................ 35
3.4 Takeaways..................................................................................................................36
3.4 Branded Sponsored Search: eBay Experiments........................................................37
3.5 What about the Rest of Branded Search?..................................................................37
3.7 Sponsored Search Takeaways...................................................................................40
Lecture 4A: Adaptive Experimentation and Website Morphing Guest Lecture............. 41
4.1 Adaptive Experimentation: Learning while Earning....................................................41
4.2 Are we all the Same?................................................................................................. 43
Lecture 4B: A/B Test Simulation.........................................................................................45
4.3 The Basics of A/B Testing.......................................................................................... 45
4.4 Benefits of A/B Testing............................................................................................... 46
Lecture 5A: eCommerce......................................................................................................47
5.1 E-commerce Basics................................................................................................... 47
5.2 On-site Marketing....................................................................................................... 48
1
, 5.3 On-site Marketing: Site Design and Optimization.......................................................49
5.4 On-site Marketing: Pop-overs.....................................................................................49
5.4 On-site Marketing: Pricing.......................................................................................... 50
5.5 Takeaways..................................................................................................................52
Lecture 5B: AB Test Debrief and Reviews and Social Care............................................. 53
5.6 Bandit Optimization in Online Decisions.................................................................... 53
5.7 Using A/B Testing and ML for Decision-Making......................................................... 54
5.8 Reviews, Ratings, and User-Generated Content....................................................... 55
5.9 Social Care.................................................................................................................62
Lecture 6A: Guest Lecture + Social Media Analytics....................................................... 63
6.1 Principles of Community Management.......................................................................63
6.2 Monetization............................................................................................................... 64
6.3 Community Building in a Digital World....................................................................... 64
6.4 Online Communities and Online Reviews.................................................................. 65
6.5 Step 1: Set Goals....................................................................................................... 66
6.6 Step 2: Social Media Metrics......................................................................................66
6.7 Step 3: Measuring ROI...............................................................................................67
6.8 Examples of Analytics................................................................................................ 68
6.9 Virality.........................................................................................................................71
6.10 Takeaways................................................................................................................72
Lecture 6B: Content Marketing........................................................................................... 73
6.11 Influencer Marketing................................................................................................. 73
6.12 Content Marketing Primer........................................................................................ 74
6.13 Step 1: Research and planning................................................................................ 76
6.14 Step 2: Content Creation..........................................................................................77
6.15 Step 3: Content Diffusion......................................................................................... 78
6.16 Takeaways................................................................................................................80
Lecture 7A: Guest Lecture + Privacy and AI..................................................................... 81
7.1 Attribution Modelling in Online Marketing...................................................................81
7.2 Privacy and a “Cookie-Less World”............................................................................83
7.3 Online Behavioural Advertising.................................................................................. 84
7.4 Privacy Regulation in US vs EU (GDPR)................................................................... 86
7.5 Takeaways (1)............................................................................................................ 87
7.6 AI tools....................................................................................................................... 87
7.7 Takeaways (2)............................................................................................................ 88
Lecture 7B: Wrap Up and Q&A........................................................................................... 89
7.9 Course Takeaways..................................................................................................... 89
Quiz Answers....................................................................................................................... 92
Campaign KPIs quiz.........................................................................................................92
Content marketing and creator economy......................................................................... 92
Display ad quiz................................................................................................................. 92
Ecommerce...................................................................................................................... 92
Lee Hower........................................................................................................................ 93
Lessons from ad research................................................................................................93
2
, Privacy and AI.................................................................................................................. 93
Reviews/Social Care........................................................................................................ 93
Social media Analytics..................................................................................................... 94
Sponsored Search and Search Engine Optimization (SEO)............................................ 94
Summary Quiz 2023.........................................................................................................94
3
