5 Probability

5.3 Conditional Probability

OBJECTIVES By the end of this section, I will be able to …

Calculate conditional probabilities.
Explain independent and dependent events.
Solve problems using the Multiplication Rule and recognize the difference between sampling with replacement and sampling without replacement.
Approximate probabilities for dependent events.
Apply Bayes' Rule to solve probability problems.

1 Introduction to Conditional Probability

As we progress through this book, you will notice a recurring theme: the more information available, the better. Very often, when we are investigating the probability of a certain event , we learn that another event has occurred. If events and are related, then the occurrence of event often influences the probability that event will occur.

271

EXAMPLE 18 Having more information often affects the probability of an event

Table 19 contains the Academy Award winners for Best Actress and Best Actor for 2009–2014, along with their ages at the time of the award.

Table 5.52: Table 19 Oscar-winning actresses and actors, and their ages

Year	Best actress	Film	Age	Best actor	Film	Age
2009	Kate Winslet	The Reader	33	Sean Penn	Milk	48
2010	Sandra Bullock	The Blind Side	45	Jeff Bridges	Crazy Heart	60
2011	Natalie Portman	Black Swan	29	Colin Firth	The King's Speech	50
2012	Meryl Streep	Iron Lady	62	Jean Dujardin	The Artist	39
2013	Jennifer Lawrence	Silver Linings Playbook	22	Daniel Day- Lewis	Lincoln	55

Table 5.52: Source: www.oscars.org.

Table 20 is a contingency table, summarizing the information in Table 19, providing the counts of the performers' genders and whether the performer was under age 40.

Table 5.53: Table 20 Contingency table of Oscar-winning performers

	Female	Male	Total
Under age 40	3	1	4
Age 40 or older	2	4	6
Total	5	5	10

Now, if we choose a performer at random from Table 20, the probability of choosing a female is . But what if we were given the extra information that the performer is age 40 or older? How does this extra information affect the probability of selecting a female?

Solution

Notice that when we are given that the person is age 40 or older, we may restrict our attention to the performers who are age 40 or older in Table 20 (highlighted). In other words, this extra information reduces the number of possible outcomes in the sample space from the 10 performers to the 6 performers who are age 40 or older. Of these six performers, two of them are female. Thus, the probability of selecting a female, given that the performer is age 40 or older, is . The extra information we were given changed the probability of selecting a female, from to .

The extra information about a related event changed the probability of the event of interest. This type of probability is an example of what is called conditional probability.

Note: The vertical line | is read as the word “given.” Whatever follows the word “given” is assumed to have occurred already.

For two related events and , the probability of given is called a conditional probability and is denoted as , or .

272

Thus, if we let represent the event that a female is selected and represent the event that the performer is age 40 or older, then

Figure 17 can help us visualize how conditional probability works. The idea is that, once event A has occurred, the only chance for event to occur is in the overlap—the intersection . Therefore, the conditional probability that will occur, given that event has already taken place, is found by taking the ratio .

Figure 5.19: FIGURE 17 How conditional probability works.

Developing Your Statistical Sense

The difference between

Students sometimes confuse the meanings of . The two are very different.

For , neither nor has occurred. We don’t know if either will or will not occur. Instead, we need to determine the probability that events and both occurred. Look at Figure 17a. For finding , we don’t know if our randomly chosen performer is female or is age 40 or older. Two of our 10 performers are both female and age 40 or older: Sandra Bullock and Meryl Streep. Thus, .
For , we assume that the event A has already occurred and now need to find the probability of event , given event . There is no probability involved with the event , because it has already occurred. Look at Figure 17b. For , we are given that the performer is age 40 or older; we know this event occurred. Therefore, we can restrict our attention solely to performers age 40 or older (event ). Of the six performers age 40 or older, two are female. Thus, .

In general, the formula for computing conditional probability is given as follows.

Calculating Conditional Probability

The conditional probability that will occur, given that event has already taken place, equals

273

EXAMPLE 19 Calculating conditional probability

Marketing companies are looking to statistical analysis and data mining in an effort to increase the customer response to their promotions. Therefore, these marketing companies are looking to hire more statistics-savvy students. Table 21 is adapted from a study on direct mail marketing. It contains the numbers of custom-ers who either responded or did not respond to a direct mail marketing campaign, along with whether they had a credit card on file with the company. The two events are:

Table 5.54: Table 21 Credit card status and marketing response

Response	Credit card on file?
Response	No	Yes	Total
Did not respond	161	79	240
Did respond	17	31	48
Total	178	110	288

Table 5.54: Source: Data Mining and Predictive Analytics, by Daniel Larose and Chantal Larose, Wiley Interscience, 2015.

Find the probability that a randomly chosen customer responded to the marketing campaign.
Calculate the probability that a randomly chosen customer both responded to the marketing campaign and has a credit card on file.
Find the conditional probability that a randomly selected customer responded, given that the customer has a credit card on file.

Solution

. There are customers who did respond, and there are customers in this experiment. Thus,
Here, we are looking for , which is the intersection between and . Earlier, we learned that this is represented by the cell at the intersection of the “Did respond” row and the “Credit card yes” column. In Table 21, this cell contains such customers. Thus, the probability that a randomly chosen customer both responded to the marketing campaign and has a credit card on file .
We will use because, in this example, it is easier to work directly with the numbers of outcomes instead of the probabilities. From (b), . Also, there are customers in total who had a credit card on file. Therefore,

That is, the probability that a randomly chosen customer responded to the direct mail marketing campaign, given that the customer had a credit card on file, is 0.2818

NOW YOU CAN DO

Exercises 9–32.

274

YOUR TURN #12

Using Table 21, find the probability that a randomly selected customer

Did not respond to the marketing campaign.
Did not respond to the marketing campaign, given that the customer did not have a credit card on file.

(The solutions are shown in Appendix A.)

What Do These Numbers Mean?

Conditional Probability

Conditional probabilities can often be interpreted as percentages of some subset of a population. For example, the conditional probability that a customer responded, given that the customer has a credit card on file, may be interpreted as the percent-age of customers with credit cards who responded.

2 Independent Events

Having a credit card on file increased the probability of a customer responding from 0.1667 to 0.2818, so we can say that the probability of responding depends in part on whether the customer has a credit card on file. In other words, the events and are dependent events.

On the other hand, if the probability of responding had been unaffected by whether the customer had a credit card on file, then we would have said that and were independent events. That is, and would have been independent events had equaled . In general, if the occurrence of an event does not affect the probability of a second event, then the two events are independent.

Events and are independent if

Equivalently, using symbols, events and are independent if

Otherwise, the events are said to be dependent.

Developing Your Statistical Sense

Andrew Luck, quarterback for the Indianapolis Colts.

Dependent and Independent Events

It may help to think of it this way: Andrew Luck is a superb quarterback for the Indianapolis Colts. Let = the event the Colts win, and let be the event that Andrew Luck gets injured. The Colts are certainly better with their great quarter-back than without him. So is greater than , because their chances of winning decrease if Luck gets injured. Because , the two events and are dependent. This makes sense, because the probability of the Colts winning is dependent on whether Andrew Luck gets injured. Now, do the same calculations—this time letting event be the event that the water boy for the Colts gets injured. It is not likely that the water boy getting injured will affect the Colts’ chances of winning, so that . Thus, the events = Colts win-ning and = water boy gets injured are independent.

Alternatively, in Step 1 you can find () and in Step 2 you can find . Then compare these two quantities for Step 3.

Strategy for Determining Whether Two Events Are Independent

Find .
Find .
Compare the two probabilities. If they are equal, then and are independent events. Otherwise, and are dependent events.

275

EXAMPLE 20 Determining whether two events are independent

Table 22 contains a contingency table summarizing the gender and survival status of the passengers and crew of RMS Titanic.

Table 5.55: Table 22 Contingency table of gender and survival status of passengers and crew of RMS Titanic. Notice that when we are given that the person is a female, we may restrict our attention to the females in the table.

	Female	Male	Total
Did not survive	126	1364	1490
Survived	344	367	711
Total	470	1731	2201

Table 5.55: Source: Report on the Loss of the “Titanic” (S.S.). British Board of Trade Inquiry Report (reprint), Allan Sutton Publishing, Gloucester, United Kingdom, 1990.

Suppose our experiment is to select one person at random from the passengers and crew. Define the following events:

Determine whether events and are independent.

Solution

We use the strategy for determining whether two events are independent.

Step 1 Find . There are 470 females out of a total of 2201 passengers and crew. Thus, .
Step 2 We need to find , which is the probability that the person survived, given the person was female. We are told that the person was female, so we may restrict our attention to the females in the table. In other words, our sample space is reduced when we know that the person was a female. Of the 470 females, 344 survived, so that .
Step 3 Because, , we conclude that and are dependent events.

NOW YOU CAN DO

Exercise 33–52.

YOUR TURN #13

In Example 20, determine whether being a male and not surviving are independent.

(The solution is shown in Appendix A.)

Developing Your Statistical Sense

Don’t Confuse Independent Events and Mutually Exclusive Events

It is important to stress the difference between independent events and mutually exclusive events. Mutually exclusive events have no outcomes in common. For two events to be independent means that the occurrence of one does not affect the probability of the other. The concepts are different.

EXAMPLE 21 Gambler's Fallacy

Suppose we have tossed a fair coin 10 times and have observed heads come up every time. Find the probability of tails on the next toss.

Solution

We have observed an unusual number of heads, so we might think that the probability of tails on the next toss is increased. However, the short answer is “Not so.” Successive tosses of a fair coin are independent because the coin has no memory of its previous tosses. Thus, what happened on the first 10 tosses has no effect on the next toss. Probability theory tells us that, in the long run, the proportion of heads and tails will eventually even out if the coin is fair. Therefore, the probability of tails on the next toss is 0.5. This is an example of the Gambler’s Fallacy.

276

3 Multiplication rule

Just as the Addition Rule is used to find probabilities of unions of events, the Multiplication rule is used to find probabilities of intersections of events. Recall the formula for the conditional probability of event given event :

We solve for ( and ) by multiplying each side by :

Similarly, consider the conditional probability of event given event :

Solving for gives us a second equation for :

The two equations for lead directly to the Multiplication Rule.

Multiplication Rule

EXAMPLE 22 Multiplication Rule

According to the Pew Internet and American Life Project,¹¹ 35% of American adults have cell phones with apps, but only 68% of those who have apps on their cell phones actually use the apps. Define the following events:

Find .
Find ( given ), the probability that an American adult uses the apps, given that he or she has a cell phone with apps.
Use the multiplication rule to calculate , the probability that an American adult has a cell phone with apps and uses the apps on his or her cell phone.

Solution

According to the study, 35% of American adults have a cell phone with apps. So = 0.35.
The research says that 68% of those who have apps actually use them, so .
Using the Multiplication Rule, we have

The probability that an American adult has a cell phone with apps and uses them is 0.238.

NOW YOU CAN DO

Exercises 53–58.

277

YOUR TURN #14

Suppose our experiment is to toss a fair die twice. Find the probability of rolling two sixes in a row.

(The solution is shown in Appendix A.)

By the definition of independent events (page 274), when events and are independent, . Using these identities, we can formulate a special case of the Multiplication Rule. Using , we can write the Multiplication Rule as

Equivalently, the Multiplication Rule also states that , but if and are independent, , so, again, .

Multiplication Rule for Two Independent Events

If and are any two independent events, .

EXAMPLE 23 Multiplication Rule for two independent events

Successive spins of the roulette wheel are independent because the wheel does not remember its result from the previous spin. Suppose the experiment is to spin the roulette wheel two times and to bet on red each time. There are 18 red numbers out of a total of 38 num-bers on the wheel. What is the probability that red will occur twice in succession?

Solution

Define the following events:

We have the probability of each event given as:

Then, because the successive spins are independent, the Multiplication Rule for Two Independent Events tells us:

NOW YOU CAN DO

Exercises 59–66.

YOUR TURN #15

For the roulette experiment in Example 23, what is the probability of not getting red on the first spin and not getting red on the second spin?

(The solution is shown in Appendix A.)

Sampling With and Without Replacement

The relationship between two events can be determined by the way the samples are chosen. Two methods of choosing samples are sampling with replacement and sam-pling without replacement.

In sampling with replacement, the randomly selected unit is returned to the population after being selected. When sampling with replacement, it is possible for the same unit to be sampled more than once.

In sampling without replacement, the randomly selected unit is not returned to the population after being selected. When sampling without replacement, it is not possible for the same unit to be sampled more than once.

278

EXAMPLE 24 Sampling with replacement

We draw a card at random from a shuffled deck, observe the card, and return it to the deck. The deck is then reshuffled, and we draw another card at random. What is the probability that both cards we select will be aces?

Solution

Define the following events:

We want to find ( and ), the probability of observing an ace on the first draw and an ace on the second draw. From the Multiplication Rule, . To find , recall that there are 4 aces in the deck of 52 cards. It is reasonable to assume that all cards are equally likely to be selected, so using the classical method, . Similarly, .

Next, we need to find , the probability of observing an ace on the second draw, given that we observe an ace on the first draw. Because the deck of 52 cards has not changed (except for shuffling), there are still 52 cards—4 of which are aces. Therefore, . Thus, the probability that both cards we select will be aces is .

Note that . Thus, by the alternative method for deter-mining independence, and are independent events when sampling with replacement.

NOW YOU CAN DO

Exercises 67–70.

YOUR TURN #16

For the experiment in Example 24, what is the probability of observing hearts on the first draw and observing hearts on the second draw, if we sample with replacement?

(The solution is shown in Appendix A.)

We can generalize this result as follows.

When sampling with replacement, successive draws can be considered independent.

EXAMPLE 25 Sampling without replacement

Suppose we alter the experiment in Example 24 as follows: We draw a card at random from a shuffled deck, hold onto the card (do not replace it) while the deck is reshuffled, and then select another card at random. What is the probability that both cards we select will be aces?

Solution

Define events and as in Example 24. Again we use the Multiplication Rule to find . The difference in this experiment comes when finding , the probability of observing an ace on the second draw given an ace on the first draw. Once we select the first ace, we do not replace it in the deck. Therefore, when the deck is reshuffled, it has only 51 cards left, only 3 of which are aces. The classical method then gives the probability of observing an ace on the second draw:

Thus, the probability that both cards we select will be aces is

279

This probability is somewhat less than the probability that both cards will be aces when sampling with replacement. Note that here we found that was not equal to . Thus, by the alternative method for determining independence, and are not independent events; they are dependent events.

NOW YOU CAN DO

Exercises 71–74.

YOUR TURN #17

For the experiment in Example 25, what is the probability of observing hearts on the first draw and observing hearts on the second draw, if we sample without replacement?

(The solution is shown in Appendix A.)

We can generalize this result as follows.

When sampling without replacement, successive draws should be considered dependent.

Note that the Multiplication Rule for Independent Events provides us with an alternative method for determining whether two events are indeed independent.

Alternative Method for Determining Independence

If , then events and are independent.

If , then events and are dependent.

EXAMPLE 26 Determining independence using the alternative method

We return to the direct mail marketing data from Example 19, reproduced here in Table 23. Use the alternative method for determining independence to determine whether the following two events are independent:

Table 5.56: Table 23 Credit card status and marketing response

Response	Credit card on file?
Response	No	Yes	Total
Did not respond	161	79	240
Did respond	17	31	48
Total	178	110	288

Table 5.56: Source: Daniel Larose and Chantal Larose, Data Mining and Predictive Analytics (Wiley Interscience, 2015).

Solution

Using Table 23, we may find the following probabilities:

Because , we have ; therefore, and are dependent.

NOW YOU CAN DO

Exercises 79–84.

280

The next example illustrates the relationship between mutually exclusive events and independence.

EXAMPLE 27 Conditional probability for mutually exclusive events

Suppose two events and are mutually exclusive, with .

Find .
Are events and independent or dependent?

Solution

Because and are mutually exclusive, . Then

That is, if event A has occurred, then event B cannot occur. This is a natural consequence of events and being mutually exclusive.

What Results Might We Expect?

Two events are independent if the occurrence of one does not affect the probability that the other will occur. However, as we saw in (a), if event occurs, then the probability that event will occur is 0. Thus, we would expect events and to be dependent.
We are given that . Thus, the product is also greater than 0. However, from (a), . Thus, , and from the alternative method for determining independence, we conclude that events and are dependent.

In other words, if two events are mutually exclusive, then they are dependent.

NOW YOU CAN DO

Exercises 85–92.

We can extend the Multiplication Rule to cover independent events.

Multiplication Rule for n Independent Events

If , , , … are independent events, then

EXAMPLE 28 Multiplication rule for independent events

According to the National Health Interview Survey, 24% of Americans ages 18–44 smoke tobacco. In a random sample of Americans ages 18–44, find the probability that all three smoke.

Solution

The U.S. Census Bureau estimates that over 100 million Americans are ages 18–44. We will shortly see that, because the sample of 3 is so small compared to the population of over 100 million, it is reasonable to assume that the successive draws are independent. Let denote the event that the th American in the 18–44 age group smokes.

281

Then, using the Multiplication Rule for Independent Events,

NOW YOU CAN DO

Exercises 93–100.

YOUR TURN #18

Refer to Example 28. In a random sample of Americans ages 18–44, find the probability that all 10 smoke.

(The solution is shown in Appendix A.)

Next, we turn to solving an “at least” problem. This involves a technique that uses complements in order to give us a nice shortcut for calculating a probability.

EXAMPLE 29 Solving an “at least” problem

Using information in Example 28, find the probability that, in a random sample of 5 Americans ages 18–44, at least 1 of them smokes.

Solution

The phrase “at least” means that one or more of the five Americans smoke, so the probability we are looking for is:

Now, each of these events is mutually exclusive, meaning that our sample can’t yield exactly 1 American who smokes and exactly 2 Americans who smoke. Thus, by the Addition Rule for Mutually Exclusive Events, the above probability equals:

Calculating all these probabilities would take a while. So, we can use the probability of the complement we learned earlier to get us a shortcut. Note that “at least 1 American smokes” is the complement of “no Americans smoke.” Then, because the complement rule for probability is: , we get:

NOW YOU CAN DO

Exercises 101–104.

YOUR TURN #19

For the experiment in Example 29, find the probability that, in a random sample of four Americans ages 18–44, at least one of them smokes.

(The solution is shown in Appendix A.)

282

4 Approximating Probabilities for Dependent Events

When the sample size is small compared to the population size, we can estimate the probability of a dependent event as if it were independent. The question is: How small is a small sample? We will use the following 1% Guideline.

The 1% Guideline

Suppose successive draws, such as those for a random sample, are being made from a population. If the sample size is no larger than 1% of the size of the population, then the probability of dependent successive draws from the population may be approximated using the assumption that the draws are independent.

EXAMPLE 30 Applying the 1% Guideline

The Gardasil vaccine

In the Section 5.2 exercises, we found the probability that a randomly chosen patient completed the treatment to be . Suppose we take a random sample of size patients.

Calculate the probability that both patients complete the treatment, using sampling without replacement.
Confirm that the 1% Guideline applies.
Approximate the probability that both patients complete the treatment using the 1% Guideline.
Find the approximation error by comparing your answers from (a) and (c).

Solution

Define the following events:

We have . Then, given that the first patient completed the treatment, we have, by sampling without replacement, . Then the Multiplication Rule gives us .
The sample of size 2 represents of the population. Thus, the 1% Guideline applies, and we may treat the successive draws as independent.
Thus, we can use the Multiplication Rule for Independent Events to solve this problem.
The approximation error, the difference between (a) and (c), is . For most applications, this small approximation error is acceptable.

The 1% Guideline is also helpful when we do not know the size of the population, but may presume that the population is very large compared to the sample size.

NOW YOU CAN DO

Exercises 75–78

5 Bayes’ Rule

When new information becomes available, our estimates of probability need to be revised to reflect the presence of the new information. An English Presbyterian Minister, Thomas Bayes (1702–1761) developed a rule, called Bayes’ rule, for updating previous probabilities using new information to arrive at revised probabilities.

283

Bayes' Rule

For any two events and ,

We apply Bayes’ Rule in the following example.

EXAMPLE 31 Bayes' Rule

The Gardasil vaccine

Table 24 contains a contingency table (crosstabulation) of the variables com-pleted and insurance type for the data set Gardasil.¹²

Table 5.57: Table 24 Contingency table of the variables completed and insurance type

			Insurance type
		Hospital- based	Medical assistance	Military	Private payer	Total
Completed	No	45	220	209	470	944
	Yes	39	55	122	253	469
	Total	84	275	331	723	1413

Define the following events:

Use Bayes’ Rule to find the probability that a randomly selected patient used military insurance, given that he or she completed the treatment.

Solution

We are interested in the probability . Substituting for and for in the formula for Bayes’ Rule, we obtain:

From Table 24, we have . The event consists of all other insurance types besides military. So we have , and . Thus,

Note that this result is confirmed by the direct calculation of using Table 24, which is the single cell of 122 military completers divided by the total number of completers.

NOW YOU CAN DO

Exercises 105–108.

YOUR TURN #20

Using Table 24 and Bayes’ Rule, find the probability that a randomly selected patient used hospital-based insurance, given that he or she completed the treatment.

(The solution is shown in Appendix A.)

284

EXAMPLE 32 The ELISA test for the presence of HIV

The ELISA test is used to screen blood for the presence of HIV. Like most diagnostic procedures, the test is not foolproof.

When a blood sample contains HIV, the ELISA test will give a positive result 99.6% of the time. That is, the false-negative rate, the percentage of tests returning a negative result when the HIV virus is actually present, is .
When the blood does not contain HIV, the ELISA test will give a negative result 98% of the time. That is, the false-positive rate, the percentage of tests returning a positive result when the HIV virus is not actually present, is .

A positive result means that the test says that the person has the HIV infection. A negative result means that the test says that the person does not have the virus. The prevalence rate for HIV in the general population is 0.5%. That is, 5 of 1000 persons in the general population have HIV.

Suppose we have samples of blood from 100,000 randomly chosen people.

Problem 1. How many people in the sample of 100,000 have HIV? How many do not?

Solution

The prevalence rate of 0.5% means that people in the sample have HIV. The remainder—99,500—do not.

Problem 2. A positive result is given 99.6% of the time for blood containing HIV. For the 500 people with HIV, how many positive results will the ELISA test return? How many of the 500 people with HIV will receive a negative result?

Solution

The ELISA test will return a positive result for of the 500 people. Thus, two people who actually have HIV will receive a test result indicating that they do not have the virus.

Problem 3. A negative result is given 98% of the time for blood without HIV. For the 99,500 people without HIV, how many negative results will the ELISA test return? Positive results?

Solution

The ELISA test will return a negative result for of the 99,500 people without HIV. The remaining 2%, or 1990 people, will receive positive ELISA test results, even though they do not have the virus.

We can use the counts we found to fill in Table 25.

Table 5.58: Table 25 ELISA test contingency table

	In reality
ELISA test results	Person has HIV	Person does not have HIV	Total
Positive	498	1,990	2,488
Negative	2	97,510	97,512
Total	500	99,500	100,000

285

We will use the information in the ELISA test contingency table to solve Problems 4 and 5. If a person is chosen at random from the sample of 100,000, define the following events:

: Person has HIV.
: Person does not have HIV.
Pos: ELISA test returned positive results.
Neg: ELISA test returned negative results.

Problem 4. What is the probability that a randomly chosen person actually does have HIV, given that the ELISA results are negative? In other words, find .

Solution

Problem 5. What is the probability that a randomly chosen person actually does not have HIV, given that the ELISA test results are positive? In other words, find .

Solution

Developing Your Statistical Sense

Which Error Is More Dangerous?

In Problems 4 and 5, we examined the probabilities of the two ways that the ELISA test can be wrong. Which error do you think is more dangerous? represents the probability that HIV is present, even though the ELISA test says otherwise. represents the probability that HIV is not present, even though the ELISA test says it is present. The designers of the ELISA test worked hard to reduce the false-negative rate to as low a level as possible. They rightly considered that it is the more dangerous type of error because of the epidemic nature of the illness. A person who receives a false-negative ELISA result could spread the infection further. Therefore, the designers tried to keep this probability as low as they could.

There is a price to be paid, however, which is the high false-positive rate, , a very high 80%. Thus, if a random person receives a positive ELISA test result, the probability that the person does not have HIV is 80%. When the ELISA test comes back positive, a second batch of tests that have a more reasonable false-positive rate is usually administered.