Reading 1: Field Everything you never wanted to know about statistics Chap 2:
BUILDING STATISTICAL MODELS:
o Real-world phenomenon = stuff that actually exists
Collect data from real world to test hypothesis
Leads to building statistical model of the phenomenon of interest
(E.g., collecting data about old bridges to build new one building a small
version/model to test it)
o The model needs to be an accurate representation of the real world to draw
correct conclusions = fit of the model
Harder in psychology (> engineering)
Good moderate poor fit
POPULATIONS AND SAMPLES:
o Want the model to apply to whole population (e.g., all humans)
Larger model/sample leads to generalizable result
o Types of statistical models = linear models
Need to check how well a model fits the given data
Plot data first (e.g., check if collected data is linear)
STATISTICAL MODELS:
o Outcome = (model) + error (= tests like ANOVA)
o Variables within the model (e.g., mean, median)
o Parameter b predictor variable Xi entity I
Outcome = (bXi) + error
Outcome = (b1X1i + b2X2i) + error
o We can predict values of an outcome based on some kind of model
Aim of stats is to get a model of what general population could look like
estimate (hypothetical value)
o Error = deviance, residual, deviation -> outcome - model
Total error = sum of errors
More spread data (i.e., not close to mean) = less representative of reality
o Degree of freedom -> number of observations that are free to vary
N-1 to divide SS
o Method of least squares = minimizing sum of squared error
GOING BEYOND THE DATA:
o Sampling variation = samples vary cause they have different members not
always representative of general population
Sampling distribution = distribution of sample means
o Standard error of the mean (SE)/standard error = sd of sample mean
Central limit theorem = the larger the sample, the more normal the
distribution
1
, o Confidence intervals = 95% z-score = 1.96 and -1.96
2.58 for 99% CI
Two different CI -> 1) both samples contain population mean but come
from different populations (most likely)
2) same population but one is in the 5% out of the CI
USING STATISTICAL MODELS TO TEST RESEARCH QUESTIONS:
o Null hypothesis significance testing (NHST) = p-value OR competing hypotheses
H0 = null hypothesis (there is no effect) H1 = alternative hypothesis
o P-value = probability of getting that model if H0 was true (p<0.05 = more likely
that H1 is correct)
Can never be fully sure that either H is correct (just probabilities)
o Bigger test = less likely to occur by chance
o H can be directional -> one-tailed VS non-directional -> two-tailed
One directional can be dangerous
o Type I error = -level) p=5%
Type II error = believe there is no effect when there is ( -level) p=20%
o Inflates error rates (multiple testing) -> Familywise/experimentwise error rate = 1-(0.95)^n
n=number of tests carries out
Bonferroni correction => use p
o Statistical power -> for type II
Power = ability of a test to find an effect -> opposite of so power = 1-
necessary for given power (often want of .05 and of .8
o Margin of error (MOE) = half the length of CI (e.g., CI of 10 has MOE of 5)
Moderate overlap means half the MOE equates standard p=.05
Same overlap of ¼ of CI means .05 no overlap means p<.05
o Larger sample size leads to smaller CI -> also means small differences can have big
-> significance of test directly linked to sample size small differences can
be seen as sig in large sample + big differences as unsig in small sample
o Problems with NHST => p-value affected by sample size (large sample can make
small effects seem important)
If p>.05 we can reject H1 but not accept H0
can just say H0 is highly unlikely
Encourages all-or-none thinking
o Need to look at CI > just p-value
2
,MODERN APPROACHES TO THEORY TESTING:
o Effect sizes => size of an effect (manipulation or strength of a relationship)
Objective/standardized measure of magnitude (so we can compare studies)
o Sd as a measure or error/noise -> dividing by it leads to score expressed in sd units
(e.g., z-score)
= = mean1 mean2 / s
In case the groups have different sd -> take the one from the
control/baseline group OR pool the sd of both groups (changes
meaning of d comparing means against all background noise, not
just noise in normal cases)
-> Not affected by sample size -value) but bigger sample is still better
o -> r=.10 (small) r=.30 (medium) r=.50 (large)
Always lies between 0-1
But d is better when groups are incompatible
o Meta-analyses => use average of effect sizes (use weighted average higher
weight for effect sizes of high precision)
REPORTING STATISTICAL MODELS:
o E.g., M = xx, 95% CI [yy, zz].
3
, o Best to give exact p-value for both non-sig & sig outcomes (unless p<.001)
Since p depends on sample size -> should also give effect size
p = xx, d = yy
SUMMARY:
4
BUILDING STATISTICAL MODELS:
o Real-world phenomenon = stuff that actually exists
Collect data from real world to test hypothesis
Leads to building statistical model of the phenomenon of interest
(E.g., collecting data about old bridges to build new one building a small
version/model to test it)
o The model needs to be an accurate representation of the real world to draw
correct conclusions = fit of the model
Harder in psychology (> engineering)
Good moderate poor fit
POPULATIONS AND SAMPLES:
o Want the model to apply to whole population (e.g., all humans)
Larger model/sample leads to generalizable result
o Types of statistical models = linear models
Need to check how well a model fits the given data
Plot data first (e.g., check if collected data is linear)
STATISTICAL MODELS:
o Outcome = (model) + error (= tests like ANOVA)
o Variables within the model (e.g., mean, median)
o Parameter b predictor variable Xi entity I
Outcome = (bXi) + error
Outcome = (b1X1i + b2X2i) + error
o We can predict values of an outcome based on some kind of model
Aim of stats is to get a model of what general population could look like
estimate (hypothetical value)
o Error = deviance, residual, deviation -> outcome - model
Total error = sum of errors
More spread data (i.e., not close to mean) = less representative of reality
o Degree of freedom -> number of observations that are free to vary
N-1 to divide SS
o Method of least squares = minimizing sum of squared error
GOING BEYOND THE DATA:
o Sampling variation = samples vary cause they have different members not
always representative of general population
Sampling distribution = distribution of sample means
o Standard error of the mean (SE)/standard error = sd of sample mean
Central limit theorem = the larger the sample, the more normal the
distribution
1
, o Confidence intervals = 95% z-score = 1.96 and -1.96
2.58 for 99% CI
Two different CI -> 1) both samples contain population mean but come
from different populations (most likely)
2) same population but one is in the 5% out of the CI
USING STATISTICAL MODELS TO TEST RESEARCH QUESTIONS:
o Null hypothesis significance testing (NHST) = p-value OR competing hypotheses
H0 = null hypothesis (there is no effect) H1 = alternative hypothesis
o P-value = probability of getting that model if H0 was true (p<0.05 = more likely
that H1 is correct)
Can never be fully sure that either H is correct (just probabilities)
o Bigger test = less likely to occur by chance
o H can be directional -> one-tailed VS non-directional -> two-tailed
One directional can be dangerous
o Type I error = -level) p=5%
Type II error = believe there is no effect when there is ( -level) p=20%
o Inflates error rates (multiple testing) -> Familywise/experimentwise error rate = 1-(0.95)^n
n=number of tests carries out
Bonferroni correction => use p
o Statistical power -> for type II
Power = ability of a test to find an effect -> opposite of so power = 1-
necessary for given power (often want of .05 and of .8
o Margin of error (MOE) = half the length of CI (e.g., CI of 10 has MOE of 5)
Moderate overlap means half the MOE equates standard p=.05
Same overlap of ¼ of CI means .05 no overlap means p<.05
o Larger sample size leads to smaller CI -> also means small differences can have big
-> significance of test directly linked to sample size small differences can
be seen as sig in large sample + big differences as unsig in small sample
o Problems with NHST => p-value affected by sample size (large sample can make
small effects seem important)
If p>.05 we can reject H1 but not accept H0
can just say H0 is highly unlikely
Encourages all-or-none thinking
o Need to look at CI > just p-value
2
,MODERN APPROACHES TO THEORY TESTING:
o Effect sizes => size of an effect (manipulation or strength of a relationship)
Objective/standardized measure of magnitude (so we can compare studies)
o Sd as a measure or error/noise -> dividing by it leads to score expressed in sd units
(e.g., z-score)
= = mean1 mean2 / s
In case the groups have different sd -> take the one from the
control/baseline group OR pool the sd of both groups (changes
meaning of d comparing means against all background noise, not
just noise in normal cases)
-> Not affected by sample size -value) but bigger sample is still better
o -> r=.10 (small) r=.30 (medium) r=.50 (large)
Always lies between 0-1
But d is better when groups are incompatible
o Meta-analyses => use average of effect sizes (use weighted average higher
weight for effect sizes of high precision)
REPORTING STATISTICAL MODELS:
o E.g., M = xx, 95% CI [yy, zz].
3
, o Best to give exact p-value for both non-sig & sig outcomes (unless p<.001)
Since p depends on sample size -> should also give effect size
p = xx, d = yy
SUMMARY:
4
