LS 40 UCLA Study Guide
Question 1.
What are the 4 steps to the bootstrapping process for constructing a 99% confidence interval of a sample median?
ANSWER
1. For each simulation 10,000: -sample dataset with replacement using the same sample size as original sample.
- calculate and store median of resampled datasets.
2. Sort bootstrap medians
3. Select values at 0.5% and 99.% of the data which is the 99% CI.
4. Calculate the 99% pivotal confidence interval.
Question 2.
List 5 problems with simply just saying the results of a study are "p<.05"? (Hint: SPEPQ)
ANSWER
1. 0.05 is not *s*trong enough; use a lower alpha value (0.01 or 0.001.
2. "p<x" is a binary, yes or no question. The researcher should give the *p* value.
3. This gives no *e*ffect size.
4. This gives no sense of *p*recision, even if effect size is given.
5. The p-value is the answer to the wrong *q*uestion.
Question 3.
Why is 0.05 not a strong enough alpha level? How can we fix this? (Problem #1 with "p<0.05)
ANSWER
Alpha = 0.05 means there is a 1 in 20 chance that studies will have a false positive. Decrease this chance by lowering alpha level.
Question 4.
Why can't we just say "p<x" and give a binary answer? (Problem #2 with p<0.05)
ANSWER
Don't try to conceal the data. Give the specific p-value.
Question 5.
What is an example of effect size? Why should we give effect size? (Problem #3 with p<0.05)
ANSWER
ex. effect size is the difference in medians Again, don't conceal the data. If the effect size not biologically or practically significant, then the research is
pointless. ex. aspirin
Question 6.
Why do we need to give precision? (Problem #4 with p<0.05)
ANSWER
We do not know the uncertainty in data around the effect size due to sample size and variation in data. Confidence intervals combine effect size and
uncertainty.
, Question 7.
Why is p-value the answer to the wrong question? (Problem #5 with p<0.05)
ANSWER
P-value tells us the probability of having a sample at least as extreme as our data by chance, IF the null hypothesis is true. p(e | H0) What we want is the
probability that the null hypothesis is wrong given our evidence. p(H0 | e)
Question 8.
What are 3 features of good plots? (Hint: VRD)
ANSWER
1. Shows the *v*ariation with the 99% CI, which is labeled.
2. Shows the *r*aw data in the dot plot.
3. Shows the *d*istribution of the data in the histogram with KDE.
Question 9.
What are 5 reasons why dynamic plunger plots suck? (Hint: ESSHT)
ANSWER
1. Does not show type of *e*rror of the error bar. (SD? SE? CI?)
2. Only shows error bar up. We have to assume error is *s*ymmetric which is not always true.
3. No *s*ample size indicated.
4. *H*ides the distribution of data.
5. Doesn't tell which *t*est method was used.
Question 10.
Limitations of Student's T-Test
ANSWER
Only for bell-shaped distributions. Only for difference in means. Only for A and B that have the same variance and same sample size.
Question 11.
Limitations of Welch's T-Test
ANSWER
Only valid for difference of means and a normally distributed A and B
Question 12.
Why is it better to use a computer for simulations?
ANSWER
A computer can do a huge amount of simulations in a very short amount of time.
Question 13.
Why is it important to run a lot of simulations (10,000)?
ANSWER
We get a more precise p-value.
, Question 14.
Pivotal Confidence Interval calculations
ANSWER
2*Mobs - Mupper 2*Mobs + Mlower Mobs is the observed median of the data.
Question 15.
Pivotal Confidence Interval interpretation
ANSWER
We are 99% confident that the interval (, ) contains the true difference.
Question 16.
Why do we use pivotal confidence inter-vals?
ANSWER
The PCI tells us about the true value of the parameter, whereas the percentile CI only tells us about the simulations.
Question 17.
Confidence interval calculations
ANSWER
1. Simulate 10,000 times.
2. Calculate Di for each simulation.
3. Sort Di values.
4. Find the cutoffs for the 0.5%ile and the 99.5%ile.
5. If Dobs is outside those cutoffs, say it is statistically significantly large
Question 18.
Percentile Confidence Interval interpretation
ANSWER
99% of the simulated differences in __ are in the interval (, ).
Question 19.
Why do we use confidence intervals?
ANSWER
Confidence intervals tells us the uncertainty and effect size.
Question 20.
What simulation do you use when you compare a proportion based on an observed sample to a known proportion?
ANSWER
Monte Carlo Simulations
Question 1.
What are the 4 steps to the bootstrapping process for constructing a 99% confidence interval of a sample median?
ANSWER
1. For each simulation 10,000: -sample dataset with replacement using the same sample size as original sample.
- calculate and store median of resampled datasets.
2. Sort bootstrap medians
3. Select values at 0.5% and 99.% of the data which is the 99% CI.
4. Calculate the 99% pivotal confidence interval.
Question 2.
List 5 problems with simply just saying the results of a study are "p<.05"? (Hint: SPEPQ)
ANSWER
1. 0.05 is not *s*trong enough; use a lower alpha value (0.01 or 0.001.
2. "p<x" is a binary, yes or no question. The researcher should give the *p* value.
3. This gives no *e*ffect size.
4. This gives no sense of *p*recision, even if effect size is given.
5. The p-value is the answer to the wrong *q*uestion.
Question 3.
Why is 0.05 not a strong enough alpha level? How can we fix this? (Problem #1 with "p<0.05)
ANSWER
Alpha = 0.05 means there is a 1 in 20 chance that studies will have a false positive. Decrease this chance by lowering alpha level.
Question 4.
Why can't we just say "p<x" and give a binary answer? (Problem #2 with p<0.05)
ANSWER
Don't try to conceal the data. Give the specific p-value.
Question 5.
What is an example of effect size? Why should we give effect size? (Problem #3 with p<0.05)
ANSWER
ex. effect size is the difference in medians Again, don't conceal the data. If the effect size not biologically or practically significant, then the research is
pointless. ex. aspirin
Question 6.
Why do we need to give precision? (Problem #4 with p<0.05)
ANSWER
We do not know the uncertainty in data around the effect size due to sample size and variation in data. Confidence intervals combine effect size and
uncertainty.
, Question 7.
Why is p-value the answer to the wrong question? (Problem #5 with p<0.05)
ANSWER
P-value tells us the probability of having a sample at least as extreme as our data by chance, IF the null hypothesis is true. p(e | H0) What we want is the
probability that the null hypothesis is wrong given our evidence. p(H0 | e)
Question 8.
What are 3 features of good plots? (Hint: VRD)
ANSWER
1. Shows the *v*ariation with the 99% CI, which is labeled.
2. Shows the *r*aw data in the dot plot.
3. Shows the *d*istribution of the data in the histogram with KDE.
Question 9.
What are 5 reasons why dynamic plunger plots suck? (Hint: ESSHT)
ANSWER
1. Does not show type of *e*rror of the error bar. (SD? SE? CI?)
2. Only shows error bar up. We have to assume error is *s*ymmetric which is not always true.
3. No *s*ample size indicated.
4. *H*ides the distribution of data.
5. Doesn't tell which *t*est method was used.
Question 10.
Limitations of Student's T-Test
ANSWER
Only for bell-shaped distributions. Only for difference in means. Only for A and B that have the same variance and same sample size.
Question 11.
Limitations of Welch's T-Test
ANSWER
Only valid for difference of means and a normally distributed A and B
Question 12.
Why is it better to use a computer for simulations?
ANSWER
A computer can do a huge amount of simulations in a very short amount of time.
Question 13.
Why is it important to run a lot of simulations (10,000)?
ANSWER
We get a more precise p-value.
, Question 14.
Pivotal Confidence Interval calculations
ANSWER
2*Mobs - Mupper 2*Mobs + Mlower Mobs is the observed median of the data.
Question 15.
Pivotal Confidence Interval interpretation
ANSWER
We are 99% confident that the interval (, ) contains the true difference.
Question 16.
Why do we use pivotal confidence inter-vals?
ANSWER
The PCI tells us about the true value of the parameter, whereas the percentile CI only tells us about the simulations.
Question 17.
Confidence interval calculations
ANSWER
1. Simulate 10,000 times.
2. Calculate Di for each simulation.
3. Sort Di values.
4. Find the cutoffs for the 0.5%ile and the 99.5%ile.
5. If Dobs is outside those cutoffs, say it is statistically significantly large
Question 18.
Percentile Confidence Interval interpretation
ANSWER
99% of the simulated differences in __ are in the interval (, ).
Question 19.
Why do we use confidence intervals?
ANSWER
Confidence intervals tells us the uncertainty and effect size.
Question 20.
What simulation do you use when you compare a proportion based on an observed sample to a known proportion?
ANSWER
Monte Carlo Simulations
