Summary Significance and Power Notes

Significance and Power

Part one

Null hypothesis significance testing
 Rules of significance testing
o 1) Assume the null hyp is true
o 2) Fit a statistical model to the data (this represents the alternative
hypothesis) – how well can the variants in our data be explained
o




o Systematic variance e.g., relationship bet two variables
o Unsystematic variance e.g. a variance in data set that can’t be explained by
the relationship e.g. maybe explained by extraneous variables
o 3) Calculate probability of getting that model or test statistic assuming that
null hyp is still true (aka our p value)
o If p<.05, our model fits the data well and we can gain confidence in the
alternative hypothesis
o But this isn’t the whole story as we only have a probability we need to figure
out how important the p value is and what sort of factors can affect this
o How important the p value is can be established by calculating levels of
power

Part two

What affects power?
 3 mian factors
o Effect size
o No of ppts in a study
o Size of our alpha level
o Other factors – variability, design, test choice, tails of a test
o Knowing these 3 factors were able to figure out the fourth one
 Effect size
o An objective and standardised measure of the magnitude of an effect
o Larger value = bigger effect size
o Depends on test conducted
 Cohen’s d (for t tests)
 Pearson’s r (correlation)
 Partial eta squared (ANOVA)
 Number of participants
o In conjunction and dependent on effect size
o Larger effect size = fewer ppts needed to get a ‘real’ effect
o Smaller effect size = more ppts needed to detect a ‘real’ effect