Question 6.101

6.101 Turning insignificance in significance.

Every user of statistics should understand the distinction between statistical significance and practical importance.

342

A sufficiently large sample will declare very small effects statistically signifcant. Consider the following randomly generated digits used to form observation pairs:

1 7 9 4 6 4 6 5 0 1
0 0 4 3 7 5 5 2 4 5

Read the 10 ordered pair values into statistical software. We will want to test the significance of the observed correlation. Excel doesn’t provide that capability.

  1. Make a scatterplot of the data and describe what you see.
  2. Compute and report the sample correlation. Software will report the -value for testing the null hypothesis that the true population correlation is 0. What is the -value? Is it consistent with what you observed in part (a)?
  3. Copy and paste the 10 ordered pair values into the same two columns to create two replicates of the original data set. Your sample size is now . Produce a scatterplot and compare it with part (a). Has the sample correlation changed? What is the -value now?
  4. Add more replicates to the two columns so that you can get -values for , 40, 50, and 60. Using these values along with what was found in parts (b) and (c), make a table of the -values versus . Describe what is happening with the -values as increases. Has the correlation changed with the increase in ?
  5. Keep replicating until you get the -value becomes less than 0.05. What is the value of ?
  6. Briefly discuss the general lesson learned with this exercise.

6.101

(a) There seems to be no relationship between and . (b) . . Yes, there is no significant correlation between and . (c) The plot is identical. The correlation is the same, . The -value is smaller (). (d) The correlation has not changed, but the -value gets smaller as increases.

-value
10 0.07565 0.8355
20 0.07565 0.7513
30 0.07565 0.6911
40 0.07565 0.6427
50 0.07565 0.6016
60 0.07565 0.5657

(e) . (f) Even with no relationship and a very small correlation, a big enough sample size can show statistical significance, warning us to make sure the effect is worth our attention rather than just “trusting” the statistics.

signif