1.16: Statistics can help us in making decisions.

In Section 1-13 we saw that researchers repeatedly found, through experimentation, that megadoses of vitamin C do not reduce cancer risk. If you put yourself in the researchers’ shoes, you might wonder how you’d figure out that the vitamin C did not reduce cancer risk. Perhaps there were 100 individuals in the group receiving megadoses of vitamin C: some of them developed cancer and some of them did not. And among the 100 subjects in the group not receiving the megadoses, some of them developed cancer and some did not. How do you decide whether the vitamin C actually had an effect? This knowledge comes from a branch of mathematics called statistics, a set of analytical and mathematical tools designed to help researchers gain understanding from the data they gather. To understand statistics, let’s start with a simple situation.

Suppose you measured the height of two people. One is a woman who is 5 feet 10 inches tall. The other is a man who is 5 feet 6 inches tall. If these were your only two observations of human height, you might conclude that female humans are taller than males. But suppose you measured the height of 100 women and 100 men chosen randomly from a population. Then you can say, “for the 100 men, the average height is 5 feet 9.5 inches, and for the 100 women, the average height is 5 feet 4 inches.” Better still, the data can illuminate for you not only the average but also some measure of how much variation there is from one individual to another. Statistical analysis can tell you not only that the average height for a man in this study is 5 feet 9.5 inches but that two-thirds of the men are between 5 feet 6.5 inches tall (3 inches less than the average) and 6 feet 0.5 inches tall (3 inches more than the average).

You will often see this type of range stated as “5 feet 9.5 inches ± 3 inches” (“plus or minus 3 inches”). Similarly, the data might show for the female subjects “5 feet 6 inches ± 3 inches,” indicating that two-thirds of the women are between 5 feet 3 inches and 5 feet 9 inches tall. As we discussed in Section 1-11, on experimental design, and as this example shows, larger numbers of participants are better than smaller numbers if you want to draw general conclusions about natural phenomena such as the height of men and women (FIGURE 1-20).

Using data to describe the characteristics of individuals participating in a study is useful, but often we want to know whether data support (or do not support) a hypothesis. If the scientific method is to be effective in helping us understand the world, it must help us make wise decisions about concrete things. For example, suppose we want to know whether having access to a textbook helps a student perform better in a biology class. Statistics can help us answer this question. After conducting a study, let’s say we find that students who had access to a textbook scored an average of 81% ± 8% on their exams, while those who did not scored an average of 76% ± 7%.

In this example, it is difficult to distinguish between the two possible conclusions.

Possibility 1: Students having access to a textbook do perform better in biology classes. In other words, our sampling of this class reveals a true relationship between the two variables, textbook access and class performance.

Possibility 2: Students having access to a textbook do not perform better in biology classes. The variation in the scores for the two groups may be too large to allow us to conclude that there’s any effect of having access to a textbook. Instead, the difference in average scores for the two groups might mean that more of the high-performing students just happened, through random chance, to be in the group given access to a textbook.

But what if the students with access to a textbook scored, on average, 95% ± 5%, while those without access scored only 60% ± 5%? In this case, we would be much more confident that there is a significant effect of having access to a textbook (FIGURE 1-21), because even with the large variation in the scores seen in each group, the two averages are still very different from each other.

Statistical methods help us to decide between these two possibilities and, equally important, to state how confident we are that one or the other is true. The greater the difference between two groups (95% vs. 60% is a greater difference than 81% vs. 76%), and the smaller the variation in each group (± 5% in the 95% and 60% groups, vs. ± 8% and ± 7% in the 81% and 76% groups), the more confident we are of the conclusion that there is a significant effect of the treatment (having access to a textbook, in this case). In other words, in the case where the groups of students scored 95% or 60%, depending on whether they did or did not have access to a textbook, it is possible that having a textbook did not actually improve performance and that this observed difference was just the result of chance. But this conclusion is very, very unlikely.

Statistics can also help us identify relationships (or lack of relationships) between variables. For example, we might note that when there are more firefighters at a fire, the fire is larger and causes more damage. This is a positive correlation, meaning that when one variable (the number of firefighters) increases, so does the other (the severity of the fire). Should we conclude that firefighters make fires worse? No. While correlations can reveal relationships between variables, they don’t tell us how the variables are related or whether change in one variable causes change in another. (You may have heard or read the phrase “correlation is not causation,” which refers to this sort of situation.) Before drawing any conclusions about more firefighters causing larger fires, we need to know about the type of fire and its size when the firefighters arrived, because those factors will significantly influence the ultimate amount of damage. To estimate the effect of the number of firefighters on the amount of damage, we would need to compare the amount of damage from fires of similar size that are fought by different numbers of firefighters.

Ultimately, statistical analyses can help us organize and summarize the observations that we make and the evidence we gather in an experiment. These analyses can then help us decide whether any differences we measure between experimental and control groups are likely to be the result of the treatment, and how confident we can be in that conclusion.