EXAMPLE 4 Does your “sign” affect your health?

In June 2015, The Washington Post published an article that reported on researchers who examined the records of patients treated at the Columbia University Medical Center between 1900 and 2000. According to the article, the researchers examined the records of 1.75 million patients, and “using statistical analysis, [the researchers] combed through 1,688 different diseases and found 55 that had a correlation with birth month, including ADHD, reproductive performance, asthma, eyesight and ear infections.” Believers in astrology are vindicated! There is a relationship between your “sign” and your health. Or is there?

Before you base future decisions about your sign and your health on these findings, recall that results significant at the 5% level occur five times in 100 in the long run, even when H₀ is true. When you make dozens of tests at the 5% level, you expect a few of them to be significant by chance alone. That means that we would expect 5% of the 1,688 different diseases, or about 84 diseases, to show some relationship with birth month. Finding that 55 of the diseases had some relationship with birth month is actually less than we would expect if there was no relationship. The results are not surprising. Running one test and reaching the α = 0.05 level is reasonably good evidence that you have found something. Running several dozen tests and reaching that level once or twice is not.