Chapter 1. Cautions about Significance Tests

Correct. We have evidence against the null hypothesis whenever P-value is smaller than α.

Incorrect. We have evidence against the null hypothesis whenever P-value is smaller than α.

2

Try again.

Correct. A skeptical doctor wants strong evidence for St. John’s Wort so would set α very small, like 0.01.

Incorrect. A skeptical doctor wants strong evidence for St. John’s Wort so would set α very small, like 0.01.

2

Try again.

Correct. If the consequences of rejecting a correct null hypothesis are very serious, set α very small, like 0.01.

Incorrect. If the consequences of rejecting a correct null hypothesis are very serious, set α very small, like 0.01.

2

Try again.

Correct. If the consequences of failing to reject an incorrect null hypothesis are very serious, set α on the large side, like 0.10.

Incorrect. If the consequences of failing to reject an incorrect null hypothesis are very serious, set α on the large side, like 0.10.

2

Try again.

Correct. If chlorine is not added when it should be, you increase the risk of spreading disease.

Incorrect. If chlorine is not added when it should be, you increase the risk of spreading disease.

2

Try again.

Correct. If chlorine is added when it shouldn't be, you increase the risk of skin irritation.

Incorrect. If chlorine is added when it shouldn't be, you increase the risk of skin irritation.

2

Try again.

Correct. This is a subjective question. Most would agree that “not adding chlorine when you should” is the most serious consequence.

Incorrect. This is a subjective question. Most would agree that “not adding chlorine when you should” is the most serious consequence.

2

Try again.

Correct. Selecting α of 0.01 leads us to reject H₀ less often which lessens the risk of skin irritation, but increases the risk of disease. Selecting α of 0.10 leads us to reject H₀ more often and add chlorine more often than perhaps we need to. This lessens the risk of disease, but increases the risk of skin irritation.

Incorrect. Selecting α of 0.01 leads us to reject H₀ less often which lessens the risk of skin irritation, but increases the risk of disease. Selecting α of 0.10 leads us to reject H₀ more often and add chlorine more often than perhaps we need to. This lessens the risk of disease, but increases the risk of skin irritation.

2

Try again.

Correct. Since P-value = 0.049 < α = 0.05, we reject H₀.

Incorrect. Since P-value = 0.049 < α = 0.05, we reject H₀.

2

Try again.

Correct. Since P-value = 0.051 < α = 0.05, we fail to reject H₀.

Incorrect. Since P-value = 0.051 < α = 0.05, we fail to reject H₀.

2

Try again.

Correct. α should be used as a guide for rejecting and failing to reject H₀.

Incorrect. α should be used as a guide for rejecting and failing to reject H₀.

2

Try again.

Correct. This is true because P-value for a two-sided test is the area in both tails whereas P-value for a one-sided test is the area in only one tail.

Incorrect. This is true because P-value for a two-sided test is the area in both tails whereas P-value for a one-sided test is the area in only one tail.

2

Try again.

Correct. As \(\overline{x}\) gets farther and farther from \(\mu _{0} \), P-value gets smaller and smaller because the tail area gets smaller and smaller.

Incorrect. As \(\overline{x}\) gets farther and farther from \(\mu _{0} \), P-value gets smaller and smaller because the tail area gets smaller and smaller.

2

Try again.

Correct. If sample size is too small, \( \frac{ \sigma }{ \sqrt{n} } \) may be so large that the difference \(\overline{x} - \mu _{0} \) does not have a small P-value when H₀ is incorrect and thus is not declared to be statistically significant.

Incorrect. If sample size is too small, \( \frac{ \sigma }{ \sqrt{n} } \) may be so large that the difference \(\overline{x} - \mu _{0} \) does not have a small P-value when H₀ is incorrect and thus is not declared to be statistically significant.

2

Try again.

Correct. Check for practical significance whenever sample size is really large.

Incorrect. Check for practical significance whenever sample size is really large.

2

Try again.

Correct. Since P-value = 0.0004 < α = 0.05, these results are statistically significant.

Incorrect. Since P-value = 0.0004 < α = 0.05, these results are statistically significant.

2

Try again.

Correct. Statistical significance and practical significance are not the same.

Incorrect. Statistical significance and practical significance are not the same.

2

Try again.

Correct. A one point difference is not worth investing time and money to improve the average score of rural students.

Incorrect. A one point difference is not worth investing time and money to improve the average score of rural students.

2

Try again.

Correct. A difference that is not big enough to matter is not practically significant.

Incorrect. A difference that is not big enough to matter is not practically significant.

2

Try again.

Correct. For statistical significance, check to see if P-value is smaller than α.

Incorrect. For statistical significance, check to see if P-value is smaller than α.

2

Try again.

Correct. For practical significance, check to see if the difference \(\overline{x} - \mu _{0} \) is big enough to matter.

Incorrect. For practical significance, check to see if the difference \(\overline{x} - \mu _{0} \) is big enough to matter.

2

Try again.

Correct. This is exactly what we mean by “multiple analyses.”

Incorrect. This is exactly what we mean by “multiple analyses.”

2

Try again.

Correct. When performing multiple analyses, more than one null hypothesis is tested using the same data. Since they are all tested using the same data, more of these get rejected than should be.

Incorrect. When performing multiple analyses, more than one null hypothesis is tested using the same data. Since they are all tested using the same data, more of these get rejected than should be.

2

Try again.

Correct. When the sample size is very large, a difference may be declared as statistically significant when it is not practically significant.

Incorrect. When the sample size is very large, a difference may be declared as statistically significant when it is not practically significant.

2

Try again.

Correct. The sample size may be too small to detect real differences.

Incorrect. The sample size may be too small to detect real differences.

2

Try again.