5.2 Probability

You have probably heard phrases such as “the margin of error” or “plus or minus 3 percentage points,” especially during an election season. These are another way of saying, “We’re not 100% sure that we can believe our own results.” This could make you cynical about statistics—you do all this work, and then you still don’t know if you can trust your data.

You would be more justified, however, in celebrating statistics for being so truthful—statistics allows us to quantify the uncertainty. And let’s be honest—most of life is filled with uncertainty, whether it’s the lack of clouds over Nagasaki, the results you get from a given face moisturizer, or the most efficient way for a worker to perform a task. Probability is central to inferential statistics because our conclusions about a population are based on data collected from a sample rather than on anecdotes and testimonials.

Coincidence and Probability

Probability and statistical reasoning can save us from ourselves when, for example, we are confronted with eerie coincidences. Two personal biases get intertwined in our thinking so that we say with genuine astonishment, “Wow! What are the chances of that?” Confirmation bias is our usually unintentional tendency to pay attention to evidence that confirms what we already believe and to ignore evidence that would disconfirm our beliefs. It is a confirmation bias when an athlete attributes her team’s wins to her lucky earrings, ignoring any losses while wearing them or any wins while wearing other earrings. Confirmation biases closely follow illusory correlations. Illusory correlation is the phenomenon of believing one sees an association between variables when no such association exists. An athlete with a confirmation bias attributing wins to her lucky earrings now believes an illusory correlation. We invite illusory correlations into our lives whenever we ignore the gentle, restraining logic of statistical reasoning.

Paul the Octopus Amazingly, Paul the German Octopus accurately predicted the outcome of all of Germany’s World Cup matches in 2010 (Shenker, 2010). By the championship game, he was famous, with a BBC News headline announcing: “German ‘psychic’ octopus predicts victory for Spain.” Before each match, Paul was presented with two boxes of food, each marked by a national flag. His picks were the boxes from which he chose food first. Paul was famous because he did so well with his picks; he wouldn’t have been in the news otherwise! But once he was famous, people focused only on his successes, not his failures. Indeed, he only correctly picked four of Germany’s six matches in the earlier Euro 2008 tournament—just one more correct pick than chance would predict. Moreover, it’s not at all surprising that there will occasionally be an animal with a lucky string like this. As University of Cambridge statistics professor David Spiegelharter pointed out to the BBC News, Paul’s feat is not that remarkable if you consider all of the animals who have tried and failed to predict sports wins over the years.

AFP/Getty Images

For example, the science show Radiolab told a remarkable story of coincidence (Abumrad & Krulwich, 2009). A 10-year-old girl named Laura Buxton released a red balloon from her hometown in the north of England. “Almost 10,” Laura corrected the host. Laura had written her address on the balloon as well as an entreaty: “Please return to Laura Buxton.” The balloon traveled 140 miles to the south of England and was found by a neighbor of another 10-year-old girl, also named Laura Buxton! The second Laura wrote to the first, and they arranged to meet. They both showed up to their meeting wearing jeans and pink sweaters. They were both the same height, had brown hair; and owned a black Labrador retriever, a gray rabbit, and a brown guinea pig with an orange spot. In fact, each brought her guinea pig to the meeting. At the time of the radio broadcast, they were 18 years old and friends. “Maybe we were meant to meet,” one of the Laura Buxtons speculated. “If it was just the wind, it was a very, very lucky wind,” said the other.

The chances seem unbelievably slim, but confirmation bias and illusory correlations both play a role here—and probability helps us understand why such coincidences happen. The thing is, coincidences are not unlikely. We notice and remember strange coincidences but do not notice the uncountable times in which there are not unlikely occurrences—the background, so to speak. We remember the story of the woman who won the lottery twice but forget the many times we bought lottery tickets and lost—and the millions of people who did the same.

The Radiolab story described this phenomenon as the “blade of grass paradox.” Imagine a golfer hitting a ball that flies way down the fairway and lands on a blade of grass. The radio show imagines the blade of grass saying: “Wow. What are the odds that that ball, out of all the billions of blades of grass…just landed on me?” Yet we know that there’s almost a 100% chance that some blade of grass was going to be crushed by that ball. It just seems miraculous to the individual blade of grass—or to the lottery winner.

Let’s go back to our story about the Laura Buxtons. A statistician pointed out that the details were “manipulated” to make for a better story. The host had remembered that they were both 10 years old, yet the first Laura reminded him she was still 9 at the time (“almost 10”). The host also admitted there were many discrepancies—one’s favorite color was pink and one’s was blue, and they had opposite academic interests—biology, chemistry, and geography for one and English, history, and classical civilization for the other. Further, it was not the second Laura Buxton who found the balloon; rather, it was her neighbor. The similarities make a better story. When you add in confirmation bias and illusory correlation, these become the WOW! details that we remember—but it’s still just another blade of grass. Still not convinced? Read on.

Expected Relative-Frequency Probability

When we discuss probability in everyday conversation, we tend to think of what statisticians call personal probability: a person’s own judgment about the likelihood that an event will occur; also called subjective probability. We might say something like “There’s a 75% chance I’ll finish my paper and go out tonight.” We don’t mean that the chance we’ll go out is precisely 75%. Rather, this is our rating of our confidence that this event will occur. It’s really just our best guess.

Mathematicians and statisticians, however, use the word probability a bit differently. Statisticians are concerned with a different type of probability, one that is more objective. In a general sense, probability is the likelihood that a particular outcome—out of all possible outcomes—will occur. For example, we might talk about the likelihood of getting heads (a particular outcome) if we flip a coin 10 times (all possible outcomes). We use probability because we usually have access only to a sample (10 flips of a coin) when we want to know about an entire population (all possible flips of a coin).

Language Alert! In statistics, we are interested in an even more specific definition of probability—expected relative-frequency probability, the likelihood of an event occurring, based on the actual outcome of many, many trials. When flipping a coin, the expected relative-frequency probability of heads, in the long run, is 0.50. Probability refers to the likelihood that something would take place, and frequency refers to how often a given outcome (e.g., heads or tails) occurs out of a certain number of trials (e.g., coin flips). Relative indicates that this number is relative to the overall number of trials, and expected indicates that it’s what we would anticipate, which might be different from what actually happens.

Determining Probabilities To determine the probability of heads, we would have to conduct many trials (coin flips), record the outcomes (either heads or tails), and determine the proportion of successes (in this case, heads).

© Eyebyte/Alamy

In reference to probability, the term trial refers to each occasion that a given procedure is carried out. For example, each time we flip a coin, it is a trial. Outcome refers to the result of a trial. For coin-flip trials, the outcome is either heads or tails. Success refers to the outcome for which we’re trying to determine the probability. If we are testing for the probability of heads, then success is heads.

People often confuse the terms probability, proportion, and percentage. Probability, the concept of most interest to us right now, is the proportion that we expect to see in the long run. Proportion is the number of successes divided by the number of trials. In the short run, in just a few trials, the proportion might not reflect the underlying probability. A coin flipped six times might have more or fewer than three heads, leading to a proportion of heads that does not parallel the underlying probability of heads. Both proportions and probabilities are written as decimals.

Percentage is simply probability or proportion multiplied by 100. A flipped coin has a 0.50 probability of coming up heads and a 50% chance of coming up heads. You are probably already familiar with percentages, so simply keep in mind that probabilities are what we would expect in the long run, whereas proportions are what we observe.

One of the central characteristics of expected relative-frequency probability is that it only works in the long run. This is an important aspect of probability, and it is referred to as the law of large numbers. Think of the earlier discussion of random assignment in which we used a random numbers generator to create a series of 0’s and 1’s to assign participants to levels of the independent variable. In the short run, over just a few trials, we can get strings of 0’s and 1’s and often do not end up with half 0’s and half 1’s, even though that is the underlying probability. With many trials, however, we’re much more likely to get close to 0.50, or 50%, of each, although many strings of 0’s or 1’s would be generated along the way. In the long run, the results are quite predictable.

Independence and Probability

Gambling and Misperceptions of Probability Many people falsely believe that a slot machine that has not paid off in a long time is “due.” A person may continue to feed coins into it, expecting an imminent payout. Of course, the slot machine itself, unless rigged, has no memory of its previous outcomes. Each trial is independent of the others.

© Photofusion Picture Library/Alamy

Language Alert! To avoid bias, statistical probability requires that the individual trials be independent, one of the favorite words of statisticians. Here we use independent to mean that the outcome of each trial must not depend in any way on the outcome of previous trials. If we’re flipping a coin, then each coin flip is independent of every other coin flip. If we’re generating a random numbers list to select participants, each number must be generated without thought to the previous numbers. In fact, this is exactly why humans can’t think randomly. We automatically glance at the previous numbers we have generated in order to best make the next one “random.” Chance has no memory, and randomness is, therefore, the only way to assure that there is no bias.