# Chapter 1. Cautions about Significance Tests

true
Stat Tutor
true
true

0:48

### Question 1.1

U3KY5UCJOSvK8fI3Daf3H/lgpk5L4bt6OkcTlDuMqT8JKxomEH+iAhTYcMZDMxcAWS4NvHnnZmFeNUhUfFJyHrTDtL1NRHAuXdyfyzy5GNmEq4jFOunur+VfHKvUw22UBkYriYyWbZuAvPgZRXs8RtTZFluJd0Qooh+5nGtJRMSQPvGX2VVKOuhzm14=
Correct. We have evidence against the null hypothesis whenever P-value is smaller than α.
Incorrect. We have evidence against the null hypothesis whenever P-value is smaller than α.
2
Try again.

1:54

### Question 1.2

Correct. A skeptical doctor wants strong evidence for St. John’s Wort so would set α very small, like 0.01.
Incorrect. A skeptical doctor wants strong evidence for St. John’s Wort so would set α very small, like 0.01.
2
Try again.

2:58

### Question 1.3

zbvINBVCcfPvX9bmSsoONn6VyUgNgDyVGJe119MiUeyX+lzdF/qcYghBRUKvlE3YDre+fMmWRyAokCyKP5ryAvx9w2NF8e/Jz+QQrN444s0zxhonevZW86IiVjnkGKZ/EhWYpwhZHc55haggcegFLdpnYntcYpi3t+4q5wXcsMv88/VI
Correct. If the consequences of rejecting a correct null hypothesis are very serious, set α very small, like 0.01.
Incorrect. If the consequences of rejecting a correct null hypothesis are very serious, set α very small, like 0.01.
2
Try again.

### Question 1.4

h7k3KNnZQyL4IYqGEjioCcgbDg7pmUOxK6lVgSdN9TxHiRUiAnbG+X8fVPtkbsrZD3dvcfsgho30VLi1UaCBD2+OBn/OpTVEZcwxPzPJ/TF0i2V94lQmfHvAvpX4GapsKhy8LhEfb94Y0FyxmTvTsxubunqHQcoAwoaU3bOyXqzEGuZ3EldOiSMZ2CYvRwIz
Correct. If the consequences of failing to reject an incorrect null hypothesis are very serious, set α on the large side, like 0.10.
Incorrect. If the consequences of failing to reject an incorrect null hypothesis are very serious, set α on the large side, like 0.10.
2
Try again.

3:56

### Question 1.5

CSTimphTkCUpPet3WerETLg98JsVL+BhZjbARdhWeenMHxPdcdVnc69gqi4TooWIDU9g8QBg23yUEM54wbdiBFHG1LLGbRXFGqn55ah8gTuM5Dxmi+ztqZPrHxtJkNPEtbtOwwR6uQrQBlE/iKPzcBRRVe7oQECiPvaxs4SUWDM+yvg7UjobmKVvpHcaikEl
Correct. If chlorine is not added when it should be, you increase the risk of spreading disease.
Incorrect. If chlorine is not added when it should be, you increase the risk of spreading disease.
2
Try again.

### Question 1.6

bg/sv2NgIGATAXyvHbG63spxWR55K6/Pv3xN4+sL4sOuT9nu+3o0Alu5bkE327p8HHeJQAtQmcVXHgLwjsRvC6rm+4JqiUmDy6mntwfWDhYIj4RSVoYWLDNGmAOgk4/VdruNH0P3A/so1ujm/IsVqRlw9KLbN6Iy2EhzvH6qe3FhxbQNAFMQNM867vs4h52F+RyS5g==
Correct. If chlorine is added when it shouldn't be, you increase the risk of skin irritation.
Incorrect. If chlorine is added when it shouldn't be, you increase the risk of skin irritation.
2
Try again.

### Question 1.7

D19w6HNnZxB/51SeCqgtvMkDCVOQMF8CCop40UTIIiJ0NnHpPtKtdb6dmCe6ia01HSQ+3E/IdOMkJA2BWi75q8SLbNAqLsySUPggZWUFZZehPqMh0nWYm/9uHV8Tq7F3w1vnO11eMwrzectE2k9WmPHqDe9b1oUP17rkCfk6MO+JQ7QJY7SoC56huTepBZS56Afnix3Ba4nt10tT2NIYG4S33H0K8eenXFQHylvOMZvNr8sMxIKw2eEAT/phxS2zZoHOAfw8QVMhA7vu6rInQA==
Correct. This is a subjective question. Most would agree that “not adding chlorine when you should” is the most serious consequence.
Incorrect. This is a subjective question. Most would agree that “not adding chlorine when you should” is the most serious consequence.
2
Try again.

4:38

### Question 1.8

EfCWoYQnnYgZGiJnCIyWORh2qhOmfiUhHjom9+S8HfV9CD0RtaGRsyRRpZvu6ewxaJB472hJnbBd/bktY68O2wIyQaImdH6HirRa4rOz7MDC/QDJQDT6OS/dhIP9eHCUC4N/0WUB8BtxxUiEEbXtzd9A8bw5vLUrVnaeAkFQYv6+6X72DbDfACabY8SY0+F8
Correct. Selecting α of 0.01 leads us to reject H0 less often which lessens the risk of skin irritation, but increases the risk of disease. Selecting α of 0.10 leads us to reject H0 more often and add chlorine more often than perhaps we need to. This lessens the risk of disease, but increases the risk of skin irritation.
Incorrect. Selecting α of 0.01 leads us to reject H0 less often which lessens the risk of skin irritation, but increases the risk of disease. Selecting α of 0.10 leads us to reject H0 more often and add chlorine more often than perhaps we need to. This lessens the risk of disease, but increases the risk of skin irritation.
2
Try again.

6:32

### Question 1.9

Correct. Since P-value = 0.049 < α = 0.05, we reject H0.
Incorrect. Since P-value = 0.049 < α = 0.05, we reject H0.
2
Try again.

### Question 1.10

GeIONAgV1MkM6oGakMkQxgjvGSpkkdL9DsxKWZ91Wfni2y22DOBKPKEtLn0QJFronAKJ1h+hHXA4WtZml53/Ff2E5preDBpZZcV1yLYmyJB7zNO7cB17BeATdMY=
Correct. Since P-value = 0.051 < α = 0.05, we fail to reject H0.
Incorrect. Since P-value = 0.051 < α = 0.05, we fail to reject H0.
2
Try again.

### Question 1.11

pu4s+TNSy82qQOnqAa7zE/RBJV5pyBXdlpJ2G6hccrMyP9fGHESiqdWJ4eynUoB1D/ibH064WkJLAm1jzlNxs901FUOr+Krgj6jzNaK9jkTlxbpfVArvBe+7RoFN/6rupz0T9t6o+IA81RzJDy7hPyH09nw=
Correct. α should be used as a guide for rejecting and failing to reject H0.
Incorrect. α should be used as a guide for rejecting and failing to reject H0.
2
Try again.

7:28

### Question 1.12

bRsJRm3jC/V6fLPvRjmtB/UWryUy4PSvCXonnR8gPyWjCdMLfdPgUgeT8UBGLOMACxn0ju8CbNOtVaSOlM6yq8uBhzrN7K5xvHac6hebelk71RTtOVcu0JCF8Iyx7/s4rymbMSjfzxtqKhrSZzrSBIK34dyknOdzpdhbiA==
Correct. This is true because P-value for a two-sided test is the area in both tails whereas P-value for a one-sided test is the area in only one tail.
Incorrect. This is true because P-value for a two-sided test is the area in both tails whereas P-value for a one-sided test is the area in only one tail.
2
Try again.

9:16

### Question 1.13

79IZ+x04VC9DDBG5GnJkxBx5CVK9RfM34cN403JddmCcXlN+9c/5b5a7zILeNyK5BRpa8962dQF8bJYJZ7krQsb1Ox/G8uoa2DpMNm6IcylH9MrOKDHDz7byLXfhyMj50ylfvUxX/HCtZVWJY6aIn1/cmgZpwNsbmxCjJpVk8J25OBlMYNxbj5azFK/K1goLsoXDRZiQ1wy/h/yjlsFFiZu41Jpdbl4O8UtFyD2ATuY/3h3f54sh69F8Zp3c1sepg/lEoxCYe77fL64NQ7o1XyGvsX9ub6y5uEGGdem3Rr26OqliTJrKkg==
Correct. As $$\overline{x}$$ gets farther and farther from $$\mu _{0}$$, P-value gets smaller and smaller because the tail area gets smaller and smaller.
Incorrect. As $$\overline{x}$$ gets farther and farther from $$\mu _{0}$$, P-value gets smaller and smaller because the tail area gets smaller and smaller.
2
Try again.

### Question 1.14

dv1hKyQRn/GzCStNGEGSHAw8llwIu0kh9imj+bcN1CPcNtz39YDsUVKjYO7iCrrjecsiQrSub+f6AC9AcCw/HEb6drl7/QGhsG6FP1HzUhmDbJ16POEda1suc20P6no/Le4CZXeyEVfpRgb2fLV684Qbw1sBilv8Hwpf0oDH2T+58A1w
Correct. If sample size is too small, $$\frac{ \sigma }{ \sqrt{n} }$$ may be so large that the difference $$\overline{x} - \mu _{0}$$ does not have a small P-value when H0 is incorrect and thus is not declared to be statistically significant.
Incorrect. If sample size is too small, $$\frac{ \sigma }{ \sqrt{n} }$$ may be so large that the difference $$\overline{x} - \mu _{0}$$ does not have a small P-value when H0 is incorrect and thus is not declared to be statistically significant.
2
Try again.

9:46

### Question 1.15

Byn3dSnN8r/Ac4cN1n0AQ7ZieXa//Q5jg/HFuya3xhaQd2RJvC4Zx5QBt0AhiS8i4gvHMdug5uQOUaJj7X7kyvRwikOc+UuHmbLq8OeDBkKEOoS8TGpAyG+etKT6srD11xWG/MPPMQFKRg5iNd1o4Lh7OaPga3cc9DzoZ34NGZva05CE2vHH1CHz+SOvX5XdidhfonbdP0P5325HNvmIvtR4TovomOOR6n1JIP1j9wivywShNTTUpdjRJD2qfGTSNFBEQAs83eqnBQoUoFyqXF5jDZQ=
Correct. Check for practical significance whenever sample size is really large.
Incorrect. Check for practical significance whenever sample size is really large.
2
Try again.

10:39

### Question 1.16

ZGQFXebch3RcWvCRmOGuM6Uq+THlgcKX0giiR0pegoxKDE0028tHlx65Iup+mwvFUkiex8n931QtyV2AB5pLEGHIXXGLI8l8GahBCQhEMK1sKVqhxZQoNx+f7x1wLL1hk6UIiqVHKIxIqvX1
Correct. Since P-value = 0.0004 < α = 0.05, these results are statistically significant.
Incorrect. Since P-value = 0.0004 < α = 0.05, these results are statistically significant.
2
Try again.

11:02

### Question 1.17

P+UxGZO0RzgCevUPcUeWi8uf4oxNKUHWzFALX097frxKxs1eRm7HNTdWhH6PT9aSZj+m5AeW5F9yHxsNB5LkAtEuDW4+1ZfgDqpQqV44TByll5uS
Correct. Statistical significance and practical significance are not the same.
Incorrect. Statistical significance and practical significance are not the same.
2
Try again.

11:34

### Question 1.18

Correct. A one point difference is not worth investing time and money to improve the average score of rural students.
Incorrect. A one point difference is not worth investing time and money to improve the average score of rural students.
2
Try again.

13:32

### Question 1.19

g3lrDjNd7eWAIE1k8TDAutGwPFF3ZrSnNk9GNszApxTgSDHq9DGZHw6vBB3jXXWVgEl9v42UhDZQH9JSFs2pk7O1BylHpOmiznPGZ/EfcFcbBIaH7p1HgbuinfMCKDRXe4qBEaeExE7rDOJA6Q/js/qbtxWt4/SNKaFwdQi3l51KVzreAn/q95UE0Pcdeg458X1N0q/NVDUl445t5aIGmiotjOHNTQo0GrB17+K0mXz6LmU7KjQX7UdJQxU=
Correct. A difference that is not big enough to matter is not practically significant.
Incorrect. A difference that is not big enough to matter is not practically significant.
2
Try again.

14:30

### Question 1.20

Correct. For statistical significance, check to see if P-value is smaller than α.
Incorrect. For statistical significance, check to see if P-value is smaller than α.
2
Try again.

### Question 1.21

k0YnhB0CEYqrRcxTaWBvn9N11ETollHWFExjw3dj28bAByDuBFb4dHJdwWbq7ocQcMuW4jXSCAM7QcP17JFZNJb4pcHqLjLY5bjeeahkY2F2fsbOoTA6p8c0I581zd5epbDv0Z26CafSJKAuIJAI/6EkiQsjtYt4Vqy8qFUHiHhZzvHNdtFqbvGo/egPjbQf8kgBhE6+2mhUGscu0kXVPd9AqC50aHAxL+ysFA==
Correct. For practical significance, check to see if the difference $$\overline{x} - \mu _{0}$$ is big enough to matter.
Incorrect. For practical significance, check to see if the difference $$\overline{x} - \mu _{0}$$ is big enough to matter.
2
Try again.

15:34

### Question 1.22

TM8MnbxYvh/z9nAUfUh2XuF9GRvAFacW5/x5wc1z2995uf7QK6Cqf3vRMdr53zYiKAi7aF1B17foJaEH4NhigSZD/Wk5ajM8cinshtwV+giTlvmxCdKUsvE5sKP1iQpoYhc/ndMzCyxy3ogTqZH1cg==
Correct. This is exactly what we mean by “multiple analyses.”
Incorrect. This is exactly what we mean by “multiple analyses.”
2
Try again.

19:55

### Question 1.23

Correct. When performing multiple analyses, more than one null hypothesis is tested using the same data. Since they are all tested using the same data, more of these get rejected than should be.
Incorrect. When performing multiple analyses, more than one null hypothesis is tested using the same data. Since they are all tested using the same data, more of these get rejected than should be.
2
Try again.

22:06

### Question 1.24

ubXDRbTkonjQJom9NbuWfOgmqjOKDJNxRQ55uXytSWEf6eW1HVbQpY2ynsqIXZTzS4qHdc/LHkhuu3tkdGA0qK4cCwJdtaYiAYQsJkW40bA7JxvFhKG2L+hWMJleBtN+sd14lBLHHKgZCgL1yqlAN2Z7ktSkwtOuHlEnFdE4sM0u6VpMCiUvIu0Y4RWi+lOeV9OGAzn24jQ1baHjSojGkIBC5oC9+BnfkBVeUX8f/2M03Qy/sryvKa9PFp/U/+QE0R1bjnd06xbC9WkQ/o0RzSnDvsbl9Q7c
Correct. When the sample size is very large, a difference may be declared as statistically significant when it is not practically significant.
Incorrect. When the sample size is very large, a difference may be declared as statistically significant when it is not practically significant.
2
Try again.

### Question 1.25

cGxjEapSr5gzjOF/r/5ZCcIZg4fWBF3JFna2cYZm/U1n0mcz9SNJGyYSr3o8cvTZL8gdgHow5MO0zDSEX/qGoFcAOYGF/JdUUTL6wndUwwpjQsv+BNIVfwELXsIHRTHXwvZjKlTpvAzc9fHKRP/OA3FM0aXnli1I50VXg0qFxyCeRrFuIEevOCiCE967J44Kr5EGAhqDSJtrh/+lYITKfhC2CSdxZD5MJBxfrwiUZFFpxFHu1NYumE5vUQs/5SjJlseyLgv7T40zCa3yuc6JZZdKeN3zZrUW
Correct. The sample size may be too small to detect real differences.
Incorrect. The sample size may be too small to detect real differences.
2
Try again.