Statistic 1b 6.4
6.4 Power and Inference as a Decision
The reject-or-not view is very important for planning studies
and for understanding statistical decision theory
Power
Fixed level ∝ significance test are closely related to
confidence intervals – in fact, two-sided test can be carried
out directly from a confidence interval
The significance level, like the CI, says how reliable the
method is in repeated use
Power – the probability that a fixed level ∝ significance test
will reject the NH when a particular alternative value of the
parameter is true
Calculation of power p.386
o 1) State the NH and AH, the particular alternative we
want to detect, and the significance level ∝
o 2) Find the values of x́ that will lead you to reject the
NH
o 3) Calculate the probability of observing these values of
x́ when the alternative is true
Increasing the power
increase ∝
consider a particular alternative that is farther away from μ0
; values of μ that are far away from μ0 are easier to detect
increase the sample size; more data provides more
information about μ
decrease σ ; has the same effect as increasing the sample
size
power calculations are important in planning studies
o a significance test low power will unlikely show a
significant effect
o failure to reject the NH when using tests of low power is
not evidence that the NH is true
Inference as decision
the AH enter the test only to help us see what outcomes count
against the NH
acceptance sampling –
Two Types of error
the NH has no longer the sample status (the statement we try
to find evidence against) that it had in test of significance
Type I error - if we reject the NH (accept AH) when in fact the
NH is true
Type II error – if we accept the NH (reject AH) when in fact the
AH is true
P.390 Figure 6.17 & 6.18
, Error probabilities
Statistical inference is based on probability
We can never be for sure, BUT by random sampling and the
laws of probability, we can say what the probabilities of both
kinds of errors are
Significance test with a fixed level of alpha give a rule for
decision making
We can describe the performance of a test by the probabilities
of Type I and Type II errors
The probability of a Type I error is the probability of rejecting
the NH when it is really true
Significance and Type I error – the significance level ∝ of
any fixed level test is the probability of a Type I error
o ∝ is the probability that the test will reject the null
hypothesis when the NH is true
the probability of a Type II error is the probability that the test
will fail to reject the NH when μ has this alternative value
power and Type II error – the power of a fixed level test to
detect a particular alternative is 1 minus the probability of a
Type II error for that alternative
the difference b/w two hypotheses lies in the reasoning that
motivates the calculations
the two types of error and their probabilities give another
interpretation of the significance test and power of a test
calculations of power are done to test the sensitivity of the
test
The common practice of testing hypotheses
1. State the NH and AH just as in a test of significance
2. Think of the problem as a decision problem, so that the
probabilities of Type I and Type II error are relevant
3. Because of Step 1, Type I errors are more serious. Choose an
∝ (significance level) and consider only tests with
probability of Type I error no greater than ∝
4. Among theses tests, select one that makes the probability of a
Type II error as small as possible (that is, power as larger as
possible). If this probability is too large, you will have to take a
larger sample to reduce the chance of an error
An alternative to significance testing regards the NH and AH
as two statements of equal status that we must decide b/w.
This decision theory point of view regards statistical inference
in general as giving rules for making decisions in the presence
of uncertainty
6.4 Power and Inference as a Decision
The reject-or-not view is very important for planning studies
and for understanding statistical decision theory
Power
Fixed level ∝ significance test are closely related to
confidence intervals – in fact, two-sided test can be carried
out directly from a confidence interval
The significance level, like the CI, says how reliable the
method is in repeated use
Power – the probability that a fixed level ∝ significance test
will reject the NH when a particular alternative value of the
parameter is true
Calculation of power p.386
o 1) State the NH and AH, the particular alternative we
want to detect, and the significance level ∝
o 2) Find the values of x́ that will lead you to reject the
NH
o 3) Calculate the probability of observing these values of
x́ when the alternative is true
Increasing the power
increase ∝
consider a particular alternative that is farther away from μ0
; values of μ that are far away from μ0 are easier to detect
increase the sample size; more data provides more
information about μ
decrease σ ; has the same effect as increasing the sample
size
power calculations are important in planning studies
o a significance test low power will unlikely show a
significant effect
o failure to reject the NH when using tests of low power is
not evidence that the NH is true
Inference as decision
the AH enter the test only to help us see what outcomes count
against the NH
acceptance sampling –
Two Types of error
the NH has no longer the sample status (the statement we try
to find evidence against) that it had in test of significance
Type I error - if we reject the NH (accept AH) when in fact the
NH is true
Type II error – if we accept the NH (reject AH) when in fact the
AH is true
P.390 Figure 6.17 & 6.18
, Error probabilities
Statistical inference is based on probability
We can never be for sure, BUT by random sampling and the
laws of probability, we can say what the probabilities of both
kinds of errors are
Significance test with a fixed level of alpha give a rule for
decision making
We can describe the performance of a test by the probabilities
of Type I and Type II errors
The probability of a Type I error is the probability of rejecting
the NH when it is really true
Significance and Type I error – the significance level ∝ of
any fixed level test is the probability of a Type I error
o ∝ is the probability that the test will reject the null
hypothesis when the NH is true
the probability of a Type II error is the probability that the test
will fail to reject the NH when μ has this alternative value
power and Type II error – the power of a fixed level test to
detect a particular alternative is 1 minus the probability of a
Type II error for that alternative
the difference b/w two hypotheses lies in the reasoning that
motivates the calculations
the two types of error and their probabilities give another
interpretation of the significance test and power of a test
calculations of power are done to test the sensitivity of the
test
The common practice of testing hypotheses
1. State the NH and AH just as in a test of significance
2. Think of the problem as a decision problem, so that the
probabilities of Type I and Type II error are relevant
3. Because of Step 1, Type I errors are more serious. Choose an
∝ (significance level) and consider only tests with
probability of Type I error no greater than ∝
4. Among theses tests, select one that makes the probability of a
Type II error as small as possible (that is, power as larger as
possible). If this probability is too large, you will have to take a
larger sample to reduce the chance of an error
An alternative to significance testing regards the NH and AH
as two statements of equal status that we must decide b/w.
This decision theory point of view regards statistical inference
in general as giving rules for making decisions in the presence
of uncertainty
