Methodology 3
TENTAMEN/EXAM: 27 March 2020
INDEX
1. Introduction & Null hypothesis 2
2. T-tests, degrees of freedom 6
3. Power, effect size, non-parametric tests 8
4. ANOVA 1 11
5. ANOVA 2 14
6. Confidence intervals, correlation and regression 18
7. Multiple Regression Analysis (MRA) 20
This summary includes (almost) everything from the lectures and the chapters from the book.
DISCLAIMER
This summary is made by a student!
Studying from it and relying on it for 100% is your own responsibility.
THANKS & GOOD LUCK!!! J
J YOU CAN DO IT!!! CA
, 2
Introduction, null hypothesis
Greer & Mulhern, Ch. 6, 9
Example drunk driving and driving performance. Different questions to ask:
- What is the incidence of drunk-driving? Descriptive research
- Is there a relationship between alcohol dose and driving performance? Relational research
- Does alcohol dose influence driving performance? Experimental research
Is there a relationship between alcohol dose and driving performance? Relational research
- Participants: 30 car drivers
- Two measurements per participants:
o Permillage alcohol (breathalyzer test)
o Deviation from the ideal track (# steering wheel reversals)
- Present in scatter plot
o Each data point = 1 participant, two measurements
o When we generally look at the data, we see an increasing line à
regression line
o Can calculate correlation coefficient (r): strength of linear relationship
- Limitation of this design: we can never find out whether it is a causal relationship
Does alcohol dose influence driving performance? Experimental research
- Hypothesis: Alcohol reduces driving performance
o Hypothesis implies/specifies a causal relationship. To test it, we must perform an experiment.
o The researcher..
§ Manipulates independent variable(s)
§ Controls other variables (keeping constant; randomization; counterbalancing)
§ Measures effect on dependent variable(s)
o If correctly carried out, an experiment has high internal validity
§ Means that we can substantiate our causal claim: we can attribute an effect that we see on
the dependent variable to the independent variable
§ No confounds
- Choosing variables/measurements
o Choice of independent variable: alcohol dose, e.g. with two levels: 0 or 1 permillage
o Choice of dependent variable: deviation from ideal track
o For those variables, we need operational definitions: what do we mean when we talk about
someone’s driving performance?
§ In our study, straightforward for independent variable: permillage
§ Less straightforward for dependent variable
• Representativeness for the construct driving performance
• Also has to be sensitive to the effect of alcohol dose that you use in the experiment
- Drawing the sample
o Population of interest = e.g. Dutch citizens with driver’s license
o How to draw sample
§ Random (or stratified) sample
§ Convenience sample
o Here we draw a convenience sample. Why? We are interested in general relationship between the
variables and not in absolute population parameters
- Assigning to conditions
o Between subjects design (independent groups design)
§ Independent groups of participants for each condition
§ Random assignment to the groups
, 3
o Within subjects design
§ Each participant contributes to each condition à so no group differences! Because everyone
is in both conditions
§ Random assignment to order of conditions (A-B; B-A)
- Results
o Shown in scatter plot: each point corresponds to one participant
§ The two dots that are not filled in with the line in between the points are
the means of each group
o Not very convenient, we are not interested in individual data but in the mean: is
it on average the case that there is a relationship?
§ Individual data may lead to clutter, especially in complex designs.
o Take out individual data points: show only means
§ This now shows the effect caused by alcohol dose on deviation
§ Disadvantages: loss of information on variability within groups…
• Why is this relevant?
o Enter statistics
§ Descriptive statistics
• E.g. we found a correlation of .65 in the sample – we found an effect of alcohol dose in
the sample
§ We are actually more interested in inferential statistics
• Is there a positive correlation in the population? Or was the r = .65 a coincidence
• Is there actually an effect of alcohol dose on deviation in the population, or was the
observed effect a coincidence?
Inferential statistics
- E.g. if the means were based on only 2 observations, the 0 permillage group would be better than the 1
pmg group (left graph). However, if we add one observation that is a bit extreme, we find that possibly
the 0 pmg group is worse than the 1 pmg group (right graph).
- So we need additional observations… how many???
- What can we say about the population on the basis of our sample data?
o Inconvenient truth = we can never be sure that an
observed effect in the sample also exists in the
population. Our finding can always be just a
coincidence!!!
o What we do to tackle this = express our uncertainty in
terms of probabilities
o Which probabilities?
§ Attempt 1: calculating the probability of the
observed data – however, we cannot do this, because
for this we would need to know the population
parameters!
• Formulating a hypothesis about a population and test it:
§ Attempt 2: Possible hypotheses about effect sizes in populations
• E.g.: H: r (alcohol dose, steering error) = .80
• Then collect data and calculate the probability of the data given that H applies: p(data|H)
• But we can only do this if there is any motivation for e.g. saying r = .80
• Disadvantages
o We often have no clue about effect sizes
o We often just want to know if there is an effect at all; effect size is of later concern
§ Attempt 3: Proof by contradiction.
TENTAMEN/EXAM: 27 March 2020
INDEX
1. Introduction & Null hypothesis 2
2. T-tests, degrees of freedom 6
3. Power, effect size, non-parametric tests 8
4. ANOVA 1 11
5. ANOVA 2 14
6. Confidence intervals, correlation and regression 18
7. Multiple Regression Analysis (MRA) 20
This summary includes (almost) everything from the lectures and the chapters from the book.
DISCLAIMER
This summary is made by a student!
Studying from it and relying on it for 100% is your own responsibility.
THANKS & GOOD LUCK!!! J
J YOU CAN DO IT!!! CA
, 2
Introduction, null hypothesis
Greer & Mulhern, Ch. 6, 9
Example drunk driving and driving performance. Different questions to ask:
- What is the incidence of drunk-driving? Descriptive research
- Is there a relationship between alcohol dose and driving performance? Relational research
- Does alcohol dose influence driving performance? Experimental research
Is there a relationship between alcohol dose and driving performance? Relational research
- Participants: 30 car drivers
- Two measurements per participants:
o Permillage alcohol (breathalyzer test)
o Deviation from the ideal track (# steering wheel reversals)
- Present in scatter plot
o Each data point = 1 participant, two measurements
o When we generally look at the data, we see an increasing line à
regression line
o Can calculate correlation coefficient (r): strength of linear relationship
- Limitation of this design: we can never find out whether it is a causal relationship
Does alcohol dose influence driving performance? Experimental research
- Hypothesis: Alcohol reduces driving performance
o Hypothesis implies/specifies a causal relationship. To test it, we must perform an experiment.
o The researcher..
§ Manipulates independent variable(s)
§ Controls other variables (keeping constant; randomization; counterbalancing)
§ Measures effect on dependent variable(s)
o If correctly carried out, an experiment has high internal validity
§ Means that we can substantiate our causal claim: we can attribute an effect that we see on
the dependent variable to the independent variable
§ No confounds
- Choosing variables/measurements
o Choice of independent variable: alcohol dose, e.g. with two levels: 0 or 1 permillage
o Choice of dependent variable: deviation from ideal track
o For those variables, we need operational definitions: what do we mean when we talk about
someone’s driving performance?
§ In our study, straightforward for independent variable: permillage
§ Less straightforward for dependent variable
• Representativeness for the construct driving performance
• Also has to be sensitive to the effect of alcohol dose that you use in the experiment
- Drawing the sample
o Population of interest = e.g. Dutch citizens with driver’s license
o How to draw sample
§ Random (or stratified) sample
§ Convenience sample
o Here we draw a convenience sample. Why? We are interested in general relationship between the
variables and not in absolute population parameters
- Assigning to conditions
o Between subjects design (independent groups design)
§ Independent groups of participants for each condition
§ Random assignment to the groups
, 3
o Within subjects design
§ Each participant contributes to each condition à so no group differences! Because everyone
is in both conditions
§ Random assignment to order of conditions (A-B; B-A)
- Results
o Shown in scatter plot: each point corresponds to one participant
§ The two dots that are not filled in with the line in between the points are
the means of each group
o Not very convenient, we are not interested in individual data but in the mean: is
it on average the case that there is a relationship?
§ Individual data may lead to clutter, especially in complex designs.
o Take out individual data points: show only means
§ This now shows the effect caused by alcohol dose on deviation
§ Disadvantages: loss of information on variability within groups…
• Why is this relevant?
o Enter statistics
§ Descriptive statistics
• E.g. we found a correlation of .65 in the sample – we found an effect of alcohol dose in
the sample
§ We are actually more interested in inferential statistics
• Is there a positive correlation in the population? Or was the r = .65 a coincidence
• Is there actually an effect of alcohol dose on deviation in the population, or was the
observed effect a coincidence?
Inferential statistics
- E.g. if the means were based on only 2 observations, the 0 permillage group would be better than the 1
pmg group (left graph). However, if we add one observation that is a bit extreme, we find that possibly
the 0 pmg group is worse than the 1 pmg group (right graph).
- So we need additional observations… how many???
- What can we say about the population on the basis of our sample data?
o Inconvenient truth = we can never be sure that an
observed effect in the sample also exists in the
population. Our finding can always be just a
coincidence!!!
o What we do to tackle this = express our uncertainty in
terms of probabilities
o Which probabilities?
§ Attempt 1: calculating the probability of the
observed data – however, we cannot do this, because
for this we would need to know the population
parameters!
• Formulating a hypothesis about a population and test it:
§ Attempt 2: Possible hypotheses about effect sizes in populations
• E.g.: H: r (alcohol dose, steering error) = .80
• Then collect data and calculate the probability of the data given that H applies: p(data|H)
• But we can only do this if there is any motivation for e.g. saying r = .80
• Disadvantages
o We often have no clue about effect sizes
o We often just want to know if there is an effect at all; effect size is of later concern
§ Attempt 3: Proof by contradiction.
