5.1 Randomness and Probability

The simplest example of a random phenomenon, one most people understand intuitively, is tossing a coin. At the beginning of football games—in high school, college, and the National Football League, the captain of the visiting team chooses Heads or Tails. The home team is assigned the other choice. A coin is then tossed, and the team whose choice lands face up selects from one of three options: to start on offensive or defense, to choose a particular goal to defend, or to defer the choice until the second half of the game. The purpose of this exercise, done thousands of times every year, is to ensure fairness.

Why do players, fans, and officials all consider this fair? Because first, the result of the toss is unknown ahead of time, and second, the result is equally likely to be heads or tails. This is exactly what we mean by a random phenomenon—we don’t know how it will turn out on any given occasion, but we can describe what happens in the long run. The long-term behavior of the phenomenon is described by a number called its probability.

Most people’s intuition tells them that if an event is random then there should not be streaks in the data. You can test this claim by asking a friend if she thinks that it would be more likely to get HTTTHTHTHH or HHHHHHHHHH when flipping a coin ten times. It turns out (and we’ll see why later) that both outcomes have the same likelihood of occurring. Yet most people tend to say that the first situation is more likely because it looks more random.

However, if an experiment is truly random and is conducted many times, streaks will appear. In fact, one should question the true randomness of an experiment if streaks are not present. If you’re a sports fan or even just one who listens to the news, you’ve no doubt heard about hot (or cold) streaks in sports. The Cincinnati Reds player, Pete Rose, had a 44 game hitting streak during the late 1970s. It’s interesting to note, though, that given that he was a .303 hitter, there was less than a one in a million chance that he’d have such a streak. Perhaps Rose’s hitting streak had more to do with randomness than the fact that he was a truly exceptional baseball player?

5.1.1 What Is Probability?

Probability measures the long-term behavior of random phenomena by describing the proportion of times that events occur in repeated trials. Before we look at probability in specific settings, we need to define some terminology.

• A probability experiment is the process of observing the outcome of a random phenomenon. Tossing a coin is a probability experiment.
• A trial is a single observation of the phenomenon. Tossing a coin once is a trial.
• An outcome is any possible result of the experiment. For tossing a coin, the outcomes are Heads or Tails.
• An event is a collection of one or more outcomes in an experiment. When tossing a coin three times, getting a Head twice is an event.

Here’s another example. Suppose the probability experiment involves selecting the "shuffle" option on your iPod. An outcome is the title of the song that your iPod first selects. An event may be the titles of the first five songs that your iPod selected.

Probability obeys a number of mathematical rules, some quite simple and some extremely complicated. The three most basic rules governing probability are the following:

As we consider more probabilities later in this chapter, we will look at these rules in a more algebraic form, but it may be easier for you to remember them in the English form given above.

When we talk about tossing a coin, we say that the probability of Heads is ½. How does this statement relate to the definition of probability and to the rules just stated?

Saying that the probability of Heads is ½ means that if you toss a coin a large number of times, roughly half of the tosses will result in Heads and half in Tails. Similarly, if you have five hundred songs stored on your iPod (and if the shuffle feature is truly random) then there is a 1/500 chance that any given song will be selected to be the first song.

Further, we see that the probability of the coin landing Heads, ½, is a number between 0 and 1, that the probability of getting Heads added to the probability of getting Tails (also ½ for a fair coin) is equal to 1, and that the probability of getting a Head is 1 minus the probability of not getting a Head (getting a Tail, in this case). So our statement about the probability of Heads being ½ obeys the three basic rules of probability.

Saying that the probability of getting a Head is ½ does not mean that whenever we toss a coin, half of the outcomes will be Heads and half will be Tails. In fact, you frequently must toss a coin a very large number of times to get a percentage of heads close to 50%. In an experiment in which a coin was tossed ten times, the outcomes were Heads, Tails, Tails, Tails, Tails, Tails, Heads, Tails, Tails, Tails, so the percentage of Heads was only 20%. But when the coin was tossed 10 more times, the total number of heads (in the 20 trials) was 8, or 40%.

Of course, physically tossing a coin a large number of times gets tedious after a while. Luckily, statistical software allows us to simulate tossing a coin. Table 5.1 shows one such simulation. Six different samples of size 25 were taken from the population {Heads, Tails}. Since samples were taken with replacement, for each selection in each sample, Heads and Tails were equally likely—just like doing an actual coin toss.

Like the physical tossing discussed earlier, this simulation has some seemingly surprising runs of the same outcome. Sample 6, for instance, has eight Heads in a row. However, this comes as no surprise to a statistician, who expects to see such behavior. It is important to remember than on any given coin toss (or selection if we are simulating), Heads and Tails are equally likely, and that the probability of ½ for Heads refers only to long-term behavior—the results from many, many coin tosses.

The Probability applet allows you to observe the Law of Large Numbers in action. This law states that as we observe a random experiment over more and more trials, the percentage of time that a particular outcome occurs gets closer and closer to the percentage that we expect. If we toss a fair coin many, many times, we will see that eventually the percentage of heads gets close to 50%. It is not uniformly close to 50%, sometimes being somewhat bigger, sometimes somewhat smaller. But when the number of tosses gets very, very big, the percentage gets quite close to 50%.

This law reinforces our definition of probability as a measure of long-term behavior. In the short run, or over any particular string of tosses, what will happen with the coin is unknown. But if we persist in tossing the coin over and over and over, sooner or later, the percentage of heads approaches 50%.

5.1.2 Types of Probability

In the Law of Large Numbers, we talk about the percentage of time a certain outcome is actually observed getting closer to the “percentage we expect” as the number of trials increases. In the case of tossing a coin, we expect to get heads 50% of the time, because there are two possible outcomes, and each is equally likely. We don’t need to flip the coin to figure out what should happen. So when we say that the probability of getting Heads is ½, we are using theoretical or classical probability.

Suppose we are interested in determining a probability for which theoretical methods are not appropriate, such as the probability of Heads for an unfair coin. In this case, we might repeatedly toss the coin and record the fraction of outcomes that were Heads. This fraction represents the approximate probability of Heads for this coin. We call this type of probability empirical probability, because it is determined by experimentation. Empirical probability is also used in settings when approximate probabilities are found based on data from previous results, surveys, or observational studies. Using your favorite team’s won-lost record to approximate the probability that it will win its next game is another example of empirical probability.

And what about the question at the beginning of the chapter—when a meteorologist says there is a 70% chance of rain, what does that mean? There is no theoretical model that applies here, and we cannot perform an experiment to find an approximate probability. The 70% chance of rain is determined by the meteorologist’s experience with conditions like the current ones (or by similar analysis conducted using technology). This type of probability is said to be subjective. While in ordinary English “subjective” generally refers to opinions with no basis in fact, in this setting, it means that neither theory nor experiment can provide an estimate of the desired probability. Although subjective probability is approximate, when used correctly, it provides a measure of likelihood that can be most useful. In the case of the weather report, it helps you decide whether to toss your rain jacket into your backpack or not.

5.1.3 Probability and Unlikely Events

When talking about probability, it is important to remember two critical points. First, we use probability to describe the likelihood of random phenomena and events—events that are determined in the short run by chance but that in the long run have a pattern that we can recognize and describe numerically. Second, even for a random phenomenon, once we know what actually happened, it makes no sense to talk about probability. Probability is used to describe the likelihood of something happening that is currently unknown.

Given those constraints, recall that our first rule of probability states that every probability is a number between 0 and 1 inclusive. What does it mean for the probability of an event to be 0? What does it mean for the probability of an event to be 1?

A probability of 0 means that the indicated event cannot occur. It is impossible. If you roll a standard six-sided die, the probability that you will roll a 7 is 0, because the faces of the die represent only the numbers 1, 2, 3, 4, 5, and 6. A probability of 1 means that the indicated event must occur; it is certain to happen. If you roll a standard six-sided die, the probability that you will roll a number less than 10 is 1, because all of the possible outcomes are less than 10.

The closer an event’s probability is to 0, the less likely it is to occur. Similarly, the closer an event’s probability is to 1, the more likely it is to occur.

But we have to be careful about the assumptions we make. Very unlikely (but not impossible) events do occur. According to the National Weather Service, the probability that any particular person in the U. S. will be struck by lightning in a given year is quite low, ranging from 1/700,000 to 1/4,000,000, depending on how the value is calculated. But over the past 30 years, an average of 62 people have been killed by lightning each year, and hundreds more have been injured. Even though the probability of being struck by lightning is quite small, that probability doesn’t mean much to you if you are one of the victims.

Likewise, just because an event is very likely (but not certain) to occur does not mean it will. If your college is one that reports on professors’ pass rates, and you find that 78% of your statistics professor’s students get a C or better, it would be reasonable to conclude that the probability that a randomly selected student will get such a grade is 0.78. But that is no guarantee that you will do so!

Human beings have very poor intuition, in general, when it comes to matters of probability. If you were to offer to buy a lottery ticket for a friend, \and gave him the choice of two tickets, one with the numbers 1, 2, 3, 4, 5, and one with the numbers 3, 17, 24, 38, 41, chances are he would select the second one, because it would seem to him more likely to win.

In fact, lottery numbers are chosen at random, and each set of five numbers has the same chance of being selected—each drawing represents a simple random sample. It’s just that 1, 2, 3, 4, 5 doesn’t look random to us, and choosing such a ticket just doesn’t feel right.

In the radio story above, “Math Guy” Keith Devlin discusses just how poor intuition is, even in professional mathematicians. This is further demonstrated by the controversy caused by an “Ask Marilyn” column. Craig Whitaker of Columbia, Maryland, asked about the Let’s Make a Deal game show and "switching doors." Marilyn vos Savant, the columnist, answered that the best strategy is to switch doors, because it increases the probability of winning. Her reply generated letters of criticism from many people, including college and university faculty members. She then explained her answer in further detail and challenged her readers to experiment for themselves and record their results. The experiments supported Marilyn’s contention. You can read Marilyn’s explanation, along with both critical and supporting comments, at her website.

The statistical inference that we study later in this course requires that we evaluate results in terms of how likely or unlikely they are. We want to be sure that it is sound reasoning and not gut feeling that informs our decision making. It is important to get a good understanding of the basic concepts of probability, so that we can correctly apply statistical processes and sensibly report our conclusions.