Summary Statistics 1B

Statistic 1b 6.4

6.4 Power and Inference as a Decision

 The reject-or-not view is very important for planning studies
and for understanding statistical decision theory
Power
 Fixed level ∝ significance test are closely related to
confidence intervals – in fact, two-sided test can be carried
out directly from a confidence interval
 The significance level, like the CI, says how reliable the
method is in repeated use
 Power – the probability that a fixed level ∝ significance test
will reject the NH when a particular alternative value of the
parameter is true
 Calculation of power p.386
o 1) State the NH and AH, the particular alternative we
want to detect, and the significance level ∝
o 2) Find the values of x́ that will lead you to reject the
NH
o 3) Calculate the probability of observing these values of
x́ when the alternative is true
Increasing the power
 increase ∝
 consider a particular alternative that is farther away from μ0
; values of μ that are far away from μ0 are easier to detect
 increase the sample size; more data provides more
information about μ
 decrease σ ; has the same effect as increasing the sample
size
 power calculations are important in planning studies
o a significance test low power will unlikely show a
significant effect
o failure to reject the NH when using tests of low power is
not evidence that the NH is true
Inference as decision
 the AH enter the test only to help us see what outcomes count
against the NH
 acceptance sampling –
Two Types of error
 the NH has no longer the sample status (the statement we try
to find evidence against) that it had in test of significance
 Type I error - if we reject the NH (accept AH) when in fact the
NH is true
 Type II error – if we accept the NH (reject AH) when in fact the
AH is true
 P.390 Figure 6.17 & 6.18

, Error probabilities
 Statistical inference is based on probability
 We can never be for sure, BUT by random sampling and the
laws of probability, we can say what the probabilities of both
kinds of errors are
 Significance test with a fixed level of alpha give a rule for
decision making
 We can describe the performance of a test by the probabilities
of Type I and Type II errors
 The probability of a Type I error is the probability of rejecting
the NH when it is really true
 Significance and Type I error – the significance level ∝ of
any fixed level test is the probability of a Type I error
o ∝ is the probability that the test will reject the null
hypothesis when the NH is true
 the probability of a Type II error is the probability that the test
will fail to reject the NH when μ has this alternative value
 power and Type II error – the power of a fixed level test to
detect a particular alternative is 1 minus the probability of a
Type II error for that alternative
 the difference b/w two hypotheses lies in the reasoning that
motivates the calculations
 the two types of error and their probabilities give another
interpretation of the significance test and power of a test
 calculations of power are done to test the sensitivity of the
test
The common practice of testing hypotheses
1. State the NH and AH just as in a test of significance
2. Think of the problem as a decision problem, so that the
probabilities of Type I and Type II error are relevant
3. Because of Step 1, Type I errors are more serious. Choose an
∝ (significance level) and consider only tests with
probability of Type I error no greater than ∝
4. Among theses tests, select one that makes the probability of a
Type II error as small as possible (that is, power as larger as
possible). If this probability is too large, you will have to take a
larger sample to reduce the chance of an error
 An alternative to significance testing regards the NH and AH
as two statements of equal status that we must decide b/w.
This decision theory point of view regards statistical inference
in general as giving rules for making decisions in the presence
of uncertainty