Experimental Research Summary
Lecture 1
Course objectives:
- Learn important concepts, tools, and technical skills needed for planning and
designing laboratory experiments.
- Generate a researchable problem statement together with specific hypotheses that
will be tested in an experiment.
- Learn the create skills of experimentation and how to design and set up your own.
- Get hands-on experience on how to analyze results from an experiment and how to
interpret results obtained.
- Get acquainted with reporting the results of an experiment.
Keep in mind:
- Expect to learn to think critically, construct, analyze designs, and data
- Do NOT expect you’ll be able to effortlessly come up with ideal designs for all
situations
- Experimentation is creative work, where ideal circumstances are never possible;
every design will have limitations.
- Yes: there will be a lot of statistics. Data are essential to experimentation.
Meaning Knowledge Validity Breath
focus focus of
findings
Survey: Technique to gather information - Descriptive External Medium
A: Cross- regarding a variable under study. - Hypothesis validity
sectional Questioning the respondents/ generation
B: Longitudinal population on a specific subject. and testing
C: panel
Experiment: A scientific procedure wherein the - Experimental Internal Narrow
A: Field factor under study is isolated to - Hypothesis validity
B: Lab test a hypothesis. testing
One or more IVs are manipulated,
and any change on one or more
DV is measured, while controlling
for the effect of extraneous
variable.
Experimenting is done to describe, predict and explain behavior. To break up phenomena in
variables and relations between those variables.
Experiments in practice: A-B testing (participants get to see either website version A or B,
conversion rate is measured and compared)
Leads to conversion optimalisation:
- Structured and systematic approach to improving the performance of a website
- Informed by data insights & psychology
- Taking the traffic there and making the most of it
,Experiments to gain customer insights:
- What product claims work best
- For high vs. low frequency exercise consumers
Behavioral research:
1. Descriptive research (thoughts, feelings, ideas, behaviors)
2. Correlational research: identifying relationships between different variables
(measuring thoughts, feelings, behavior)
Correlational research:
- Measures the association between 2 variables (-1 negative association, 1 positive
association).
- What is the direction of the correlation? (watching aggressive movies leads to
aggressive behavior or the other way around?)
- Spurious correlations (in a rural area there is a higher stork density and a higher birth
rate, but storks don’t lead to a higher birth rate) (Is the number of murders
committed caused by the number of ice creams eaten?)
From description to prediction:
1. Description: careful observation
2. Correlation: relationship between observed variables
3. Experimental: testing causality, A B?
Experimental research setting:
- Field experimentation (real life setting, mundane reality, less control)
- Laboratory experimentation (more control, better able to manipulate, no natural
setting)
Experimental research crux:
- Experimentation is the only type of research that can demonstrate that a change in
one variable causes a predictable change in another variable
- Most difficult: making sure that a change in Y was not caused by something else than
X.
Reliability vs. validity:
, Experimentation & causation:
1. Needs to be correlation between 2 variables
2. Asymmetrical direction
3. A (cause) -- > B (effect)
4. Change in A is accompanied by a change in B
5. No alternative explanation for the change in B than the change in A (other possible
causes controlled for)
Experimental research essence:
- Test specific hypotheses about relationship between cause and effect via controlled
(laboratory) conditions.
a. The effect of an IV on a DV
b. Manipulating the IV
c. Measure effects on the DV
d. Control other influences (high internal but low external validity)
Importance of randomization
- Use all sorts of people with all sorts of characteristics
- Potential confounds are under control
- So: change in Y can be attributed to X
Lecture 2:
Experimental research: main steps
1. Theoretical framework
a. Problem identification
b. Hypothesis formulation
2. Experimental design
a. Manipulation of IVs
b. Measurement of DV
c. Control for confounds
3. Data analyses
a. Get familiar with your data
b. Checks
c. Conduct main analyses
d. Conduct follow up analyses
Problem identification; sources:
- Real life experiences
- Previous research and theory
1. Conflicting findings
2. Boundary conditions find explanation for observed effects
3. Propositions of a theory that are not tested yet
4. Applying a theory to a consumer setting
Problem identification = creativity:
- Look for the exception rather than the rule
- Introspective self-analysis
- Look for the opposite problem
Lecture 1
Course objectives:
- Learn important concepts, tools, and technical skills needed for planning and
designing laboratory experiments.
- Generate a researchable problem statement together with specific hypotheses that
will be tested in an experiment.
- Learn the create skills of experimentation and how to design and set up your own.
- Get hands-on experience on how to analyze results from an experiment and how to
interpret results obtained.
- Get acquainted with reporting the results of an experiment.
Keep in mind:
- Expect to learn to think critically, construct, analyze designs, and data
- Do NOT expect you’ll be able to effortlessly come up with ideal designs for all
situations
- Experimentation is creative work, where ideal circumstances are never possible;
every design will have limitations.
- Yes: there will be a lot of statistics. Data are essential to experimentation.
Meaning Knowledge Validity Breath
focus focus of
findings
Survey: Technique to gather information - Descriptive External Medium
A: Cross- regarding a variable under study. - Hypothesis validity
sectional Questioning the respondents/ generation
B: Longitudinal population on a specific subject. and testing
C: panel
Experiment: A scientific procedure wherein the - Experimental Internal Narrow
A: Field factor under study is isolated to - Hypothesis validity
B: Lab test a hypothesis. testing
One or more IVs are manipulated,
and any change on one or more
DV is measured, while controlling
for the effect of extraneous
variable.
Experimenting is done to describe, predict and explain behavior. To break up phenomena in
variables and relations between those variables.
Experiments in practice: A-B testing (participants get to see either website version A or B,
conversion rate is measured and compared)
Leads to conversion optimalisation:
- Structured and systematic approach to improving the performance of a website
- Informed by data insights & psychology
- Taking the traffic there and making the most of it
,Experiments to gain customer insights:
- What product claims work best
- For high vs. low frequency exercise consumers
Behavioral research:
1. Descriptive research (thoughts, feelings, ideas, behaviors)
2. Correlational research: identifying relationships between different variables
(measuring thoughts, feelings, behavior)
Correlational research:
- Measures the association between 2 variables (-1 negative association, 1 positive
association).
- What is the direction of the correlation? (watching aggressive movies leads to
aggressive behavior or the other way around?)
- Spurious correlations (in a rural area there is a higher stork density and a higher birth
rate, but storks don’t lead to a higher birth rate) (Is the number of murders
committed caused by the number of ice creams eaten?)
From description to prediction:
1. Description: careful observation
2. Correlation: relationship between observed variables
3. Experimental: testing causality, A B?
Experimental research setting:
- Field experimentation (real life setting, mundane reality, less control)
- Laboratory experimentation (more control, better able to manipulate, no natural
setting)
Experimental research crux:
- Experimentation is the only type of research that can demonstrate that a change in
one variable causes a predictable change in another variable
- Most difficult: making sure that a change in Y was not caused by something else than
X.
Reliability vs. validity:
, Experimentation & causation:
1. Needs to be correlation between 2 variables
2. Asymmetrical direction
3. A (cause) -- > B (effect)
4. Change in A is accompanied by a change in B
5. No alternative explanation for the change in B than the change in A (other possible
causes controlled for)
Experimental research essence:
- Test specific hypotheses about relationship between cause and effect via controlled
(laboratory) conditions.
a. The effect of an IV on a DV
b. Manipulating the IV
c. Measure effects on the DV
d. Control other influences (high internal but low external validity)
Importance of randomization
- Use all sorts of people with all sorts of characteristics
- Potential confounds are under control
- So: change in Y can be attributed to X
Lecture 2:
Experimental research: main steps
1. Theoretical framework
a. Problem identification
b. Hypothesis formulation
2. Experimental design
a. Manipulation of IVs
b. Measurement of DV
c. Control for confounds
3. Data analyses
a. Get familiar with your data
b. Checks
c. Conduct main analyses
d. Conduct follow up analyses
Problem identification; sources:
- Real life experiences
- Previous research and theory
1. Conflicting findings
2. Boundary conditions find explanation for observed effects
3. Propositions of a theory that are not tested yet
4. Applying a theory to a consumer setting
Problem identification = creativity:
- Look for the exception rather than the rule
- Introspective self-analysis
- Look for the opposite problem
