Statistic Chapter 16
Bootstrap Methods and Permutation Tests
T procedures are useful in practice because they are robust
Inference about spread based on Normal distributions is not
robust and therefore of little use in practice
“what would happen if we applied this method many times?”
bootstrap interval and permutation tests are conceptually
simple because they appeal directly to the basis of all
inference: the sampling distribution that shows what would
happen if we took very many samples under the same
conditions
Software
bootstrapping and permutation test are feasible in practice
only with software that automates the heavy computation that
these methods require
16.1 The Bootstrap Idea
The big idea: resampling and the bootstrap distribution
a sampling distribution is based on many random samples
from the population
the bootstrap is a way of finding the sampling distribution
from just one sample
o Step 1: Resampling – create many resamples by
repeatedly sampling with replacement from this one
random sample. Each resample is the same size as the
original random sample
o Sampling with replacement means that after we
randomly draw an observation from the original sample,
we put it back before drawing the next observation
o Step 2: Bootstrap distribution – the bootstrap
distribution of a statistic collects its values from the
many resamples; gives information about the sampling
distribution
the bootstrap idea – the original sample represents the
population from which it was drawn. Thus, resamples from this
original sample represent what we would get if we took many
samples from the population. The bootstrap distribution of a
statistic, based on the resamples, represents the sampling
distribution of the statistic
Shape: the bootstrap distribution is nearly Normal – the
bootstrap distribution shape is close to the shape we expect
the sampling distribution to have (central limit theorem)
Center: the bootstrap distribution is centered close to the
mean of the original sample; the mean of the bootstrap
, distribution has little bias as an estimator of the mean of the
original sample
Spread: bootstrap standard error s/ √ n
The great advantage of the resampling idea is that it often
works even when theory fails
Statistical theory tells us if the population has a Normal
distribution, then the sampling distribution of x bar is also
Normal
o If the population is not Normal but our sample is large
we can use the central limit theorem
The bootstrap idea: we avoid taking many samples by instead
taking many resamples from a single sample
Thinking about the bootstrap idea
The resampled observations are not used as if they were new
data
The bootstrap distribution of the resample means is used only
to estimate how the sample mean of one actual sample size
would vary because of random sampling
Using the data for two purposes – to estimate a parameter
and also to estimate the variability of the estimate
B – number of resamples; x́ * - mean of the resamples
1
o mean boot = ∑ x́ *
B
o
√
SE boot =
1
B−1
∑ ( x́ ¿ −meanboot ) 2
bootstrap standard error – the bootstrap standard error
SE boot of a statistic is the standard deviation of the bootstrap
distribution of that statistic
we don’t appeal to the central limit theorem or other theory to
tell us that a sampling distribution is roughly Normal. We look
at the bootstrap distribution to see if it is roughly Normal (or
not).
The bootstrap allows us to calculate standard errors for
statistics for which we don’t have formulas and to check
Normality for statistics that theory doesn’t easily handle
The plug-in principle – to estimate a parameter, a quantity
that describes the population, use the statistic that is the
corresponding quantity for the sample
The plug-in principle tells us to estimate a population mean by
the sample mean and a population standard deviation by the
sample standard deviation
The bootstrap idea itself is a form of the plug-in principle:
substitute the data for te population and the draw samples to
mimic the process of building a sampling distributions
Using software
Bootstrap Methods and Permutation Tests
T procedures are useful in practice because they are robust
Inference about spread based on Normal distributions is not
robust and therefore of little use in practice
“what would happen if we applied this method many times?”
bootstrap interval and permutation tests are conceptually
simple because they appeal directly to the basis of all
inference: the sampling distribution that shows what would
happen if we took very many samples under the same
conditions
Software
bootstrapping and permutation test are feasible in practice
only with software that automates the heavy computation that
these methods require
16.1 The Bootstrap Idea
The big idea: resampling and the bootstrap distribution
a sampling distribution is based on many random samples
from the population
the bootstrap is a way of finding the sampling distribution
from just one sample
o Step 1: Resampling – create many resamples by
repeatedly sampling with replacement from this one
random sample. Each resample is the same size as the
original random sample
o Sampling with replacement means that after we
randomly draw an observation from the original sample,
we put it back before drawing the next observation
o Step 2: Bootstrap distribution – the bootstrap
distribution of a statistic collects its values from the
many resamples; gives information about the sampling
distribution
the bootstrap idea – the original sample represents the
population from which it was drawn. Thus, resamples from this
original sample represent what we would get if we took many
samples from the population. The bootstrap distribution of a
statistic, based on the resamples, represents the sampling
distribution of the statistic
Shape: the bootstrap distribution is nearly Normal – the
bootstrap distribution shape is close to the shape we expect
the sampling distribution to have (central limit theorem)
Center: the bootstrap distribution is centered close to the
mean of the original sample; the mean of the bootstrap
, distribution has little bias as an estimator of the mean of the
original sample
Spread: bootstrap standard error s/ √ n
The great advantage of the resampling idea is that it often
works even when theory fails
Statistical theory tells us if the population has a Normal
distribution, then the sampling distribution of x bar is also
Normal
o If the population is not Normal but our sample is large
we can use the central limit theorem
The bootstrap idea: we avoid taking many samples by instead
taking many resamples from a single sample
Thinking about the bootstrap idea
The resampled observations are not used as if they were new
data
The bootstrap distribution of the resample means is used only
to estimate how the sample mean of one actual sample size
would vary because of random sampling
Using the data for two purposes – to estimate a parameter
and also to estimate the variability of the estimate
B – number of resamples; x́ * - mean of the resamples
1
o mean boot = ∑ x́ *
B
o
√
SE boot =
1
B−1
∑ ( x́ ¿ −meanboot ) 2
bootstrap standard error – the bootstrap standard error
SE boot of a statistic is the standard deviation of the bootstrap
distribution of that statistic
we don’t appeal to the central limit theorem or other theory to
tell us that a sampling distribution is roughly Normal. We look
at the bootstrap distribution to see if it is roughly Normal (or
not).
The bootstrap allows us to calculate standard errors for
statistics for which we don’t have formulas and to check
Normality for statistics that theory doesn’t easily handle
The plug-in principle – to estimate a parameter, a quantity
that describes the population, use the statistic that is the
corresponding quantity for the sample
The plug-in principle tells us to estimate a population mean by
the sample mean and a population standard deviation by the
sample standard deviation
The bootstrap idea itself is a form of the plug-in principle:
substitute the data for te population and the draw samples to
mimic the process of building a sampling distributions
Using software
