Examples 2 and 3 suggest that we should not rely on significance alone in understanding a statistical study. In Example 3, just knowing that the sample proportion was $\widehat{p}$ = 0.507 helps a lot. You can decide whether this deviation from one-half is large enough to interest you. Of course, $\widehat{p}$ = 0.507 isn’t the exact truth about the coin, just the chance result of Count Buffon’s tosses. So a confidence interval, whose width shows how closely we can pin down the truth about the coin, is even more helpful. Here are the 95% confidence intervals for the true probability of a head p, based on the two sample sizes in Example 3. You can check that the method of Chapter 21 gives these answers.

The confidence intervals make clear what we know (with 95% confidence) about the true p. The interval for 4040 tosses includes 0.5, so we are not confident that the coin is unbalanced. For 100,000 tosses, however, we are confident that the true p lies between 0.504 and 0.510. In particular, we are confident that it is not 0.5.