PORTAGE STATISTICS CNSL 503
MODULE 3 STATS QUESTIONS AND
CORRECT ANSWERS
Hypothesis testing ANSW✅✅the formal process by which sample data is used to evaluate a
statistical hypothesis or a claim about a population (make inferences about a population)
-based on probability (it's not black and white), making a best guess based on evidence from our
sample
What are the four steps of hypothesis testing? ANSW✅✅1 State the hypotheses (Ho and Ha)
Ho= There is no relationship between test scores and tutoring. (M= 75)
Ha= There is a relationship between test scores and tutoring. (M=X 75, M>75)
2. Establish the decision criteria (set the alpha level or significance level as 0.05)
3. Collect data and compute sample statistics as well as convert sample statistic into test statistic
(e.g. z-score)
4. Make a decision (if p-value <= alpha level, reject the null hypothesis, the result is statistically
significant.)
(if p value>= alpha level, fail to reject the null hypothesis. there is no significant treatment effect).
significance level ANSW✅✅the probability the researcher defines as "very unlikely" in a
hypothesis test
If a sample mean falls in the critical region, it is sufficiently unlikely to be the same as the untreated
population
what action? ANSW✅✅Reject Ho
, If a sample mean does not fall in the critical region, it is likely to be the same as the untreated
population ANSW✅✅Fail to reject Ho
T/F Hypothesis testing is Based on the logic of falsification
T/F Hypothesis testing can prove claims. ANSW✅✅T.
Based on the logic of falsification (trying to reject the null hypothesis)
False
Never ever say "prove" during hypothesis testing
Combating Problems with Hypothesis Testing ANSW✅✅Supplement hypothesis testing with
additional measures such as effect size and confidence intervals
Problems with Hypothesis Testing ANSW✅✅1. Conventional levels of alpha (.05, .01, or .001) are
arbitrary
We just somehow decided, but probabilities exist on a continuum
2. A larger sample size is more likely to achieve statistical significance than a smaller sample size
Statistical significance does not always have meaningful significance
Even small effects can be statistically significant with a large enough sample size
3 There is too much reliance on the p-value as the sole measure upon which conclusions are made
Conclusions should also be drawn on past evidence, validity of assumptions made and so forth
4 People (including researchers!) misinterpret the p-value and the results of hypothesis testing
5 There is a bias towards publishing statistically significant results over non-significant results
This leads to an incomplete and biased picture of the findings
Alpha level (a)/ significance level ANSW✅✅the maximum probability that one would make a Type
I error if the null hypothesis is true
P-value ANSW✅✅the probability that we have made a type I error if the null hypothesis is true
MODULE 3 STATS QUESTIONS AND
CORRECT ANSWERS
Hypothesis testing ANSW✅✅the formal process by which sample data is used to evaluate a
statistical hypothesis or a claim about a population (make inferences about a population)
-based on probability (it's not black and white), making a best guess based on evidence from our
sample
What are the four steps of hypothesis testing? ANSW✅✅1 State the hypotheses (Ho and Ha)
Ho= There is no relationship between test scores and tutoring. (M= 75)
Ha= There is a relationship between test scores and tutoring. (M=X 75, M>75)
2. Establish the decision criteria (set the alpha level or significance level as 0.05)
3. Collect data and compute sample statistics as well as convert sample statistic into test statistic
(e.g. z-score)
4. Make a decision (if p-value <= alpha level, reject the null hypothesis, the result is statistically
significant.)
(if p value>= alpha level, fail to reject the null hypothesis. there is no significant treatment effect).
significance level ANSW✅✅the probability the researcher defines as "very unlikely" in a
hypothesis test
If a sample mean falls in the critical region, it is sufficiently unlikely to be the same as the untreated
population
what action? ANSW✅✅Reject Ho
, If a sample mean does not fall in the critical region, it is likely to be the same as the untreated
population ANSW✅✅Fail to reject Ho
T/F Hypothesis testing is Based on the logic of falsification
T/F Hypothesis testing can prove claims. ANSW✅✅T.
Based on the logic of falsification (trying to reject the null hypothesis)
False
Never ever say "prove" during hypothesis testing
Combating Problems with Hypothesis Testing ANSW✅✅Supplement hypothesis testing with
additional measures such as effect size and confidence intervals
Problems with Hypothesis Testing ANSW✅✅1. Conventional levels of alpha (.05, .01, or .001) are
arbitrary
We just somehow decided, but probabilities exist on a continuum
2. A larger sample size is more likely to achieve statistical significance than a smaller sample size
Statistical significance does not always have meaningful significance
Even small effects can be statistically significant with a large enough sample size
3 There is too much reliance on the p-value as the sole measure upon which conclusions are made
Conclusions should also be drawn on past evidence, validity of assumptions made and so forth
4 People (including researchers!) misinterpret the p-value and the results of hypothesis testing
5 There is a bias towards publishing statistically significant results over non-significant results
This leads to an incomplete and biased picture of the findings
Alpha level (a)/ significance level ANSW✅✅the maximum probability that one would make a Type
I error if the null hypothesis is true
P-value ANSW✅✅the probability that we have made a type I error if the null hypothesis is true
