311

In the previous section, we discussed the probability distribution of the sample mean, which meant a focus on population values that were quantitative. We will now shift our focus to population values that are categorical. Counts and proportions are common discrete statistics that describe categorical data.

In Section 5.1 (pages 283–287), we discussed the use of simulation to study the sampling distribution of the sample proportion. In this section, we will use probability theory to more precisely describe the sampling distributions of the sample count and proportion. Let’s start with an example.

When there are only two possible outcomes for a random variable, we can summarize the results by giving the count for one of the possible outcomes. We let n represent the sample size, and we use X to represent the random variable that gives the count for the outcome of interest.

In our example, we chose the random variable X to be the number of parents who think that long working hours make it difficult to spend enough time with their children. We could have chosen X to be the number of parents who do not think that long working hours make it difficult to spend enough time with their children. The choice is yours. Often, we make the choice based on how we would like to describe the results in a summary. Which choice do you prefer in this case?

When a random variable has only two possible outcomes, it is more common to use the sample proportion as the summary rather than the count X.

312

Notice that this summary takes into account the sample size n. We need to know n in order to properly interpret the meaning of the random variable X. For example, the conclusion we would draw about parent opinions in this survey would be quite different if we had observed X = 755 from a sample twice as large, n = 2012.

Just like the sample mean, sample counts and sample proportions are commonly used statistics, and understanding their sampling distributions is important for statistical inference. These statistics, however, are discrete random variables, so their sampling distributions introduce us to a new family of probability distributions.

The binomial distributions for sample counts

The distribution of a count X depends on how the data are produced. Here is a simple but common situation.

Think of tossing a coin n times as an example of the binomial setting. Each toss gives either heads or tails, and the outcomes of successive tosses are independent. If we call heads a success, then p is the probability of a head and remains the same as long as we toss the same coin. The number of heads we count is a random variable X. The distribution of X (and, more generally, the distribution of the count of successes in any binomial setting) is completely determined by the number of observations n and the success probability p.

313

The binomial distributions are an important class of discrete probability distributions. Later in this section, we will learn how to assign probabilities to outcomes and how to find the mean and standard deviation of binomial distributions. The most important skill for using binomial distributions is the ability to recognize situations to which they do and do not apply. This can be done by checking all the facets of the binomial setting.

314

Binomial distributions in statistical sampling

The binomial distributions are important in statistics when we wish to make inferences about the proportion p of “successes” in a population. Here is a typical example.

Choosing an SRS from a population is not quite a binomial setting. Removing one record in Example 5.19 changes the proportion of bad records in the remaining population, so the state of the second record chosen is not independent of the first. Because the population is large, however, removing a few items has a very small effect on the composition of the remaining population. Successive inspection results are very nearly independent. The population proportion of misclassified records is

If the first record chosen is bad, the proportion of bad records remaining is 799/9999 = 0.079908. If the first record is good, the proportion of bad records left is 800/9999 = 0.080008. These proportions are so close to 0.08 that, for practical purposes, we can act as if removing one record has no effect on the proportion of misclassified records remaining. We act as if the count X of misclassified sales records in the audit sample has the binomial distribution B(150, 0.08).

Populations like the one described in Example 5.19 often contain a relatively small number of items with very large values. For this example, these values would be very large sale amounts and likely represent an important group of items to the auditor. An SRS taken from such a population will likely include very few items of this type. Therefore, it is common to use a stratified sample in settings like this. Strata are defined based on dollar value of the sale, and within each stratum, an SRS is taken. The results are then combined to obtain an estimate for the entire population.

315

Finding binomial probabilities

We will later give a formula for the probability that a binomial random variable takes any of its values. In practice, you will rarely have to use this formula for calculations because some calculators and most statistical software packages will calculate binomial probabilities for you.

If you do not have suitable computing facilities, you can still shorten the work of calculating binomial probabilities for some values of n and p by looking up probabilities in Table C in the back of this book. The entries in the table are the probabilities P(X = k) of individual outcomes for a binomial random variable X.

316

The values of p that appear in Table C are all 0.5 or smaller. When the probability of a success is greater than 0.5, restate the problem in terms of the number of failures. The probability of a failure is less than 0.5 when the probability of a success exceeds 0.5. When using the table, always stop to ask whether you must count successes or failures.

Binomial mean and standard deviation

If a count X has the B(n, p) distribution, what are the mean μ_X and the standard deviation σ_X? We can guess the mean. If we expect 46% of the students to have fallen asleep in class due to poor sleep, the mean number in 10 students should be 46% of 10, or 4.6. That’s μ_X when X has the B(10, 0.46) distribution.

Intuition suggests more generally that the mean of the B(n, p) distribution should be np. Can we show that this is correct and also obtain a short formula for the standard deviation? Because binomial distributions are discrete probability distributions, we could find the mean and variance by using the definitions in Section 4.4. Here is an easier way.

A binomial random variable X is the count of successes in n independent observations that each have the same probability p of success. Let the random variable S_i indicate whether the ith observation is a success or failure by taking the values S_i = 1 if a success occurs and S_i = 0 if the outcome is a failure. The S_i are independent because the observations are, and each S_i has the same simple distribution:

318

From the definition of the mean of a discrete random variable, we know that the mean of each S_i is

μ_S = (1)(p) + (0)(1 − p) = p

Similarly, the definition of the variance shows that . Because each S_i is 1 for a success and 0 for a failure, to find the total number of successes X we add the S_i’s:

X = S₁ + S₂ + · · · + S_n

Apply the addition rules for means and variances to this sum. To find the mean of X we add the means of the S_i’s:

μ_X = μ_S1 + μ_S2 + · · · + μ_Sn

= nμ_S = np

Similarly, the variance is n times the variance of a single S, so that . The standard deviation σ_X is the square root of the variance. Here is the result.

319

Sample proportions

What proportion of a company’s sales records have an incorrect sales tax classification? What percent of adults favor stronger laws restricting firearms? In statistical sampling, we often want to estimate the proportion p of “successes” in a population. Our estimator is the sample proportion of successes:

Be sure to distinguish between the proportion and the count X. The count takes whole-number values between 0 and n, but a proportion is always a number between 0 and 1. In the binomial setting, the count X has a binomial distribution. The proportion does not have a binomial distribution. We can, however, do probability calculations about by restating them in terms of the count X and using binomial methods. In Example 5.12 (page 303), we took a similar approach for the sum, restating the problem in terms of the sample mean and then using the Normal distribution to calculate the probability.

As a first step, find the mean and standard deviation of a sample proportion. We know the mean and standard deviation of a sample count, so apply the rules from Section 4.4 for the mean and variance of a constant times a random variable. Here is the result.

Let’s now use these formulas to calculate the mean and standard deviation for Example 5.24.

The fact that the mean of is p states in statistical language that the sample proportion in an SRS is an unbiased estimator of the population proportion p. When a sample is drawn from a new population having a different value of the population proportion p, the sampling distribution of the unbiased estimator changes so that its mean moves to the new value of p. We observed this fact empirically in Section 5.1 and have now verified it from the laws of probability.

321

The variability of about its mean, as described by the variance or standard deviation, gets smaller as the sample size increases. So a sample proportion from a large sample will usually lie quite close to the population proportion p. We observed this in the simulation experiment on page 285 in Section 5.1. Now we have discovered exactly how the variability decreases: the standard deviation is . Similar to what we observed in the previous section, the in the denominator means that the sample size must be multiplied by 4 if we wish to divide the standard deviation in half.

Normal approximation for counts and proportions

Using simulation, we discovered in Section 5.1 that the sampling distribution of a sample proportion is close to Normal. Now we know that the distribution of is that of a binomial count divided by the sample size n. This seems at first to be a contradiction. To clear up the matter, look at Figure 5.16. This is a probability histogram of the exact distribution of the proportion of very satisfied shoppers , based on the binomial distribution B(2500, 0.6). There are hundreds of narrow bars, one for each of the 2501 possible values of . Most have probabilities too small to show in a graph. The probability histogram looks very Normal! In fact, both the count X and the sample proportion are approximately Normal in large samples.

We also know this to be true as a result of the central limit theorem discussed in the previous section (page 298). Recall that we can consider the count X as a sum

X = S₁ + S₂ + · · · + S_n

of independent random variables S_i that take the value 1 if a success occurs on the ith trial and the value 0 otherwise. The proportion of successes can then be thought of as the sample mean of the S_i and, like all sample means, is approximately Normal when n is large. Given that is approximately Normal, the count will also be approximately Normal because it is just a constant n times , an approximately Normal random variable.

322

These Normal approximations are easy to remember because they say that and X are Normal, with their usual means and standard deviations. Whether or not you use the Normal approximations should depend on how accurate your calculations need to be. For most statistical purposes, great accuracy is not required. Our “rule of thumb” for use of the Normal approximations reflects this judgment.

The accuracy of the Normal approximations improves as the sample size n increases. They are most accurate for any fixed n when p is close to 0.5, and least accurate when p is near 0 or 1. You can compare binomial distributions with their Normal approximations by using the Normal Approximation to Binomial applet. This applet allows you to change n or p while watching the effect on the binomial probability histogram and the Normal curve that approximates it.

Figure 5.17 summarizes the distribution of a sample proportion in a form that emphasizes the big idea of a sampling distribution. Just as with Figure 5.11 (page 304), the general framework for constructing a sampling distribution is shown on the left.

The sampling distribution of is shown on the right. Keep this figure in mind as you move toward statistical inference.

The distribution of the count of bad records in a sample of 15 is distinctly non-Normal, as Figure 5.15 (page 316) showed. When we increase the sample size to 150, however, the shape of the binomial distribution becomes roughly Normal. Figure 5.19 displays the probability histogram of the binomial distribution with the density curve of the approximating Normal distribution superimposed. Both distributions have the same mean and standard deviation, and both the area under the histogram and the area under the curve are 1. The Normal curve fits the histogram reasonably well. Look closely: the histogram is slightly skewed to the right, a property that the symmetric Normal curve can’t match.

325

The continuity correction

Figure 5.20 illustrates an idea that greatly improves the accuracy of the Normal approximation to binomial probabilities. The binomial probability is the area of the histogram bars for values 0 to 10. The bar for actually extends from 9.5 to 10.5. Because the discrete binomial distribution puts probability only on whole numbers, the probabilities and are the same. The Normal distribution spreads probability continuously, so these two Normal probabilities are different. The Normal approximation is more accurate if we consider to extend from 9.5 to 10.5, matching the bar in the probability histogram.

The event includes the outcome . Figure 5.20 shades the area under the Normal curve that matches all the histogram bars for outcomes 0 to 10, bounded on the right not by 10, but by 10.5. So is calculated as . On the other hand, excludes the outcome , so we exclude the entire interval from 9.5 to 10.5 and calculate from the Normal table. Here is the result of the Normal calculation in Example 5.27 improved in this way:

≐ P(Z ≤ −0.45) = 0.3264

The improved approximation misses the binomial probability by only 0.012. Acting as though a whole number occupies the interval from 0.5 below to 0.5 above the number is called the continuity correctioncontinuity correction to the Normal approximation. If you need accurate values for binomial probabilities, try to use software to do exact calculations. If no software is available, use the continuity correction unless n is very large. Because most statistical purposes do not require extremely accurate probability calculations, we do not emphasize use of the continuity correction.

326

Binomial formula

We can find a formula for the probability that a binomial random variable takes any value by adding probabilities for the different ways of getting exactly that many successes in n observations. Here is the example we will use to show the idea.

Because the method doesn’t depend on the specific example, we will use “S” for success and “F” for failure. In Example 5.28, “S” would stand for type O blood. Do the work in two steps.

Step 1: Find the probability that a specific two of the five tries give successes—say, the first and the third. This is the outcome SFSFF. The multiplication rule for independent events tells us that

P(SFSFF) = P(S)P(F)P(S)P(F)P(F)

= (0.25)(0.75)(0.25)(0.75)(0.75)

= (0.25)²(0.75)³

Step 2: Observe that the probability of any one arrangement of two S’s and three F’s has this same probability. That’s true because we multiply together 0.25 twice and 0.75 three times whenever we have two S’s and three F’s. The probability that X = 2 is the probability of getting two S’s and three F’s in any arrangement whatsoever. Here are all the possible arrangements:

SSFFF SFSFF SFFSF SFFFS FSSFF

FSFSF FSFFS FFSSF FFSFS FFFSS

There are 10 of them, all with the same probability. The overall probability of two successes is, therefore,

P(X = 2) = 10(0.25)²(0.75)³ = 0.2637

The pattern of this calculation works for any binomial probability. To use it, we need to be able to count the number of arrangements of k successes in n observations without actually listing them. We use the following fact to do the counting.

327

The formula for binomial coefficients uses the factorialfactorial notation. The factorial n! for any positive whole number n is

n! = n × (n − 1) × (n − 2) × · · · × 3 × 2 × 1

Also, 0! = 1. Notice that the larger of the two factorials in the denominator of a binomial coefficient will cancel much of the n! in the numerator. For example, the binomial coefficient we need for Example 5.28 is

This agrees with our previous calculation.

The notation is not related to the fraction . A helpful way to remember its meaning is to read it as “binomial coefficient n choose k.” Binomial coefficients have many uses in mathematics, but we are interested in them only as an aid to finding binomial probabilities. The binomial coefficient counts the number of ways in which k successes can be distributed among n observations. The binomial probability is this count multiplied by the probability of any specific arrangement of the k successes. Here is the formula we seek.

Here is an example of the use of the binomial probability formula.

328

The Poisson distributions

A count X has a binomial distribution when it is produced under the binomial setting. If one or more facets of this setting do not hold, the count X will have a different distribution. In this subsection, we discuss one of these distributions.

Frequently, we meet counts that are open-ended; that is, they are not based on a fixed number of n observations: the number of customers at a popular café between 12:00 P.M. and 1:00 P.M.; the number of dings on your car door; the number of reported pedestrian/bicyclist collisions on campus during the academic year. These are all counts that could be 0, 1, 2, 3, and so on indefinitely.

The Poisson distribution is another model for a count and can often be used in these open-ended situations. The count represents the number of events (call them “successes”) that occur in some fixed unit of measure such as a period of time or region of space. The Poisson distribution is appropriate under the following conditions.

329

For binomial distributions, the important quantities were n, the fixed number of observations, and p, the probability of success on any given observation. For Poisson distributions, the only important quantity is the mean number of successes μ occurring per unit of measure.

Similar to the binomial, Poisson probability calculations are rarely done by hand if the event includes numerous possible values for X. Most software provides functions to calculate and the cumulative probabilities of the form . These cumulative probability calculations make solving many problems less tedious. Here’s an example.

330

Under the Poisson setting, this probability of 0.028 applies not only to the next hour, but also to any other hour in the future. The probability does not change because the units of measure are the same size and nonoverlapping.

If we add counts from successive nonoverlapping areas of equal size, we are just counting the successes in a larger area. That count still meets the conditions of the Poisson setting. However, because our unit of measure has doubled, the mean of this new count is twice as large. Put more formally, if X is a Poisson random variable with mean and Y is a Poisson random variable with mean and Y is independent of X, then is a Poisson random variable with mean This fact means that we can combine areas or look at a portion of an area and still use Poisson distributions to model the count.

331

When the mean of the Poisson distribution is large, it may be difficult to calculate Poisson probabilities using a calculator or software. Fortunately, when μ is large, Poisson probabilities can be approximated using the Normal distribution with mean μ and standard deviation . Here is an example.

332

While the Normal approximation is adequate for many practical purposes, we recommend using statistical software when possible so you can get exact Poisson probabilities.

There is one other approximation associated with the Poisson distribution that is worth mentioning. It is related to the binomial distribution. Previously, we recommended using the Normal distribution to approximate the binomial distribution when n and p satisfy and . In cases where n is large but p is so small that , the Poisson distribution with yields more accurate results. For example, suppose that you wanted to calculate when X has the distribution. Using R, the actual binomial probability and the Poisson approximation are

pbinom(2,1000,.001) ppois(2,1)

[1] 0.9197907 [1] 0.9196986

The Poisson approximation gives a very accurate probability calculation for the binomial distribution in this case.