344

The SAT is a widely used measure of readiness for college study. It consists of two sections, one for mathematical reasoning ability (SATM), and one for reading and writing ability (SATV). Possible scores on each section range from 200 to 800, for a total range of 400 to 1600. Since 1995, section scores have been recentered so that the mean is approximately 500 with a standard deviation of 100 in a large “standardized group.” This scale has been maintained so that scores have a constant interpretation.

The sample mean is the natural estimator of the unknown population mean μ. We know that is an unbiased estimator of μ. More important, the law of large numbers says that the sample mean must approach the population mean as the size of the sample grows. The value , therefore, appears to be a reasonable estimate of the mean score μ that all 485,264 students would achieve if they took the test.

But how reliable is this estimate? A second sample of 500 students would surely not give a sample mean of 495 again. Unbiasedness says only that there is no systematic tendency to underestimate or overestimate the truth. Could we plausibly get a sample mean of 485 and a sample mean of 520 in repeated samples? An estimate without an indication of its variability is of little value.

Statistical confidence

The unbiasedness of an estimator concerns the center of its sampling distribution, but questions about variation are answered by looking at its spread. The central limit theorem says that if the entire population of SATM scores has mean μ and standard deviation σ, then in repeated SRSs of size 500, the sample mean is approximately . Let us suppose that we know that the standard deviation σ of SATM scores in our California population is σ = 100. (We will see in the next chapter how to proceed when σ is not known. For now, we are more interested in statistical reasoning than in details of realistic methods.) This means that in repeated sampling the sample mean has an approximately Normal distribution centered at the unknown population mean μ and a standard deviation of

345

Now we are ready to proceed. Consider this line of thought, which is illustrated in Figure 6.2:

We have simply restated a fact about the sampling distribution of . The language of statistical inference uses this fact about what would happen in the long run to express our confidence in the results of any one sample. Our sample gave . We say that we are 95% confident that the unknown mean score for all California seniors lies between

and

Be sure you understand the grounds for our confidence. There are only two possibilities for our SRS:

We cannot know whether our sample is one of the 95% for which the interval contains μ or one of the unlucky 5% for which it does not contain μ. The statement that we are 95% confident is shorthand for saying, “We arrived at these numbers by a method that gives correct results 95% of the time.”

Confidence intervals

In the setting of Example 6.3, the interval of numbers between the values is called a 95% confidence interval for μ. Like most confidence intervals we will discuss, this one has the form

estimate ± margin of error

The estimate ( in this case) is our guess for the value of the unknown parameter. The margin of error (9 here) reflects how accurate we believe our guess is, based on the variability of the estimate, and how confident we are that the procedure will produce an interval that will contain the true population mean μ.

Figure 6.3 illustrates the behavior of 95% confidence intervals in repeated sampling from a Normal distribution with mean μ. The center of each interval (marked by a dot) is at and varies from sample to sample. The sampling distribution of (also Normal) appears at the top of the figure to show the long-term pattern of this variation.

The 95% confidence intervals, ± margin of error, from 25 SRSs appear below the sampling distribution. The arrows on either side of the dot () span the confidence interval. All except one of the 25 intervals contain the true value of μ. In those intervals that contain μ, sometimes μ is near the middle of the interval and sometimes it is closer to one of the ends. This again reflects the variation of . In practice, we don’t know the value of μ, but we have a method such that, in a very large number of samples, 95% of the confidence intervals will contain μ.

347

We can construct confidence intervals for many different parameters based on a variety of designs for data collection. We will learn the details of a number of these in later chapters. Two important things about a confidence interval are common to all settings:

Users can choose the confidence level, but 95% is the standard for most situations. Occasionally, 90% or 99% is used. We use C to stand for the confidence level in decimal form. For example, a 95% confidence level corresponds to C = 0.95.

With the Confidence Interval applet, you can construct diagrams similar to the one displayed in Figure 6.3. The only difference is that the applet displays the Normal population distribution at the top rather than the Normal sampling distribution of . You choose the confidence level C, the sample size n, and whether you want to generate 1 or 25 samples at a time. A running total (and percent) of the number of intervals that contain μ is displayed so you can consider a larger number of samples.

When generating single samples, the data for the latest SRS are shown below the confidence interval. The spread in these data reflects the spread of the population distribution. This spread is assumed known, and it does not change with sample size. What does change, as you vary n, is the margin of error, since it reflects the uncertainty in the estimate of μ. As you increase n, you’ll find that the span of the interval gets smaller.

Confidence interval for a population mean

We now construct a level C confidence interval for the mean μ of a population when the data are an SRS of size n. The construction is based on the sampling distribution of the sample mean . This distribution is exactly when the population has the N(μ, σ) distribution. The central limit theorem says that this same sampling distribution is approximately correct for large samples whenever the population mean and standard deviation are μ and σ. For now, we will assume we are in one of these two situations. We discuss what we mean by “large sample” after we briefly study these intervals.

Our construction of a 95% confidence interval for the mean SATM score began by noting that any Normal distribution has probability about 0.95 within ±2 standard deviations of its mean. To construct a level C confidence interval we first catch the central C area under a Normal curve. That is, we must find the number z* such that any Normal distribution has probability C within ±z* standard deviations of its mean.

Because all Normal distributions have the same standardized form, we can obtain everything we need from the standard Normal curve. Figure 6.4 shows how C and z* are related. Values of z* for many choices of C appear in the row labeled z* at the bottom of Table D. Here are the most important entries from that row:

Notice that for 95% confidence the value 2 obtained from the 68–95–99.7 rule is replaced with the more precise 1.96.

349

As Figure 6.4 reminds us, any Normal curve has probability C between the point z* standard deviations below the mean and the point z* standard deviations above the mean. The sample mean has the Normal distribution with mean μ and standard deviation so there is probability C that lies between

This is exactly the same as saying that the unknown population mean μ lies between

That is, there is probability C that the interval contains μ. This is our confidence interval. The estimate of the unknown μ is , and the margin of error is .

Starting in 2008, Sallie Mae, a major provider of education loans and savings programs, has conducted an annual study titled “How America Pays for College.” In the 2015 survey, 1600 randomly selected individuals (800 parents of undergraduate students and 800 undergraduate students) were surveyed by telephone.³

350

Many of the survey questions focus on the composition of funding sources used to pay for college, so the undergraduates in the survey are often responding for their parents. For example, each participant is asked to report how much of the parent’s current income is used to pay for college. Do you think it is wise to combine responses across the parents and undergraduates? Are you fully aware of how much money your parents are spending and borrowing for college? The authors report overall averages and percents in their report. We will also consider this a sample from one population but this is certainly debatable.

Suppose that the researchers who designed this study had used a different sample size. How would this affect the confidence interval? We can answer this question by changing the sample size in our calculations and assuming that the sample mean is the same.

351

Notice that the margin of error for this example is three times as large as the margin of error that we computed in Example 6.4. The only change that we made was to assume that the sample size is 177 rather than 1593. This sample size is one-ninth of the original 1593. Thus, we triple the margin of error when we reduce the sample size to one-ninth of the original value. Figure 6.5 illustrates the effect in terms of the intervals.

The argument leading to the form of confidence intervals for the population mean μ rested on the fact that the statistic used to estimate μ has a Normal distribution. Because many sample estimates have Normal distributions (at least approximately), it is useful to notice that the confidence interval has the form

352

estimate ± z*σ_estimate

The estimate based on the sample is the center of the confidence interval. The margin of error is z*σ_estimate. The desired confidence level determines z* from Table D. The standard deviation of the estimate is found from knowledge of the sampling distribution in a particular case. When the estimate is from an SRS, the standard deviation of the estimate is . We return to this general form numerous times in the following chapters.

How confidence intervals behave

The margin of error for the mean of a Normal population illustrates several important properties that are shared by all confidence intervals in common use. The user chooses the confidence level, and the margin of error follows from this choice.

Both high confidence and a small margin of error are desirable characteristics of a confidence interval. High confidence says that our method almost always gives correct answers. A small margin of error says that we have pinned down the parameter quite precisely.

Suppose that in planning a study you calculate the margin of error and decide that it is too large. Here are your choices to reduce it:

For most problems, you would choose a confidence level of 90%, 95%, or 99%, so z* will be 1.645, 1.960, or 2.576, respectively. Figure 6.4 (page 349) shows that z* will be smaller for lower confidence (smaller C). The bottom row of Table D also shows this. If n and σ are unchanged, a smaller z* leads to a smaller margin of error.

353

Similarly, choosing a larger sample size n reduces the margin of error for any fixed confidence level. The square root in the formula implies that we must multiply the number of observations by 4 in order to cut the margin of error in half. Likewise, if we want to reduce the standard deviation of by a factor of 4, we must take a sample 16 times as large.

The standard deviation σ measures the variation in the population. You can think of the variation among individuals in the population as noise that obscures the average value μ. It is harder to pin down the mean μ of a highly variable population; that is why the margin of error of a confidence interval increases with σ.

In practice, we can sometimes reduce σ by carefully controlling the measurement process. We also might change the mean of interest by restricting our attention to only part of a large population. Focusing on a subpopulation will often result in a smaller σ. This is why many medical studies only use healthy male subjects. The tradeoff, however, is less generalizable results.

Choosing the sample size

A wise user of statistics never plans data collection without, at the same time, planning the inference. You can arrange to have both high confidence and a small margin of error. The margin of error of the confidence interval for a population mean is

Notice once again that it is the size of the sample that determines the margin of error. The size of the population (as long as the population is much larger than the sample) does not influence the sample size we need.

To obtain a desired margin of error m, plug in the value of σ and the value of z* for your desired confidence level, and solve for the sample size n. Here is the result.

This formula does not account for collection costs. In practice, taking observations costs time and money. The required sample size may be impossibly expensive. In those situations, you might consider a larger margin of error and/or a lower confidence level to find a workable sample size.

354

It is always safe to round up to the next higher whole number when finding n because this will give us a smaller margin of error. The purpose of this calculation is to determine a sample size that is sufficient to provide useful results, but the determination of what is useful is a matter of judgment.

Would we need a much larger sample size to obtain a margin of error of $25? Here is the calculation:

A sample of n = 13,519 is much larger, and the costs of such a large sample may be prohibitive.

Unfortunately, the actual number of usable observations is often less than what we plan at the beginning of a study. This is particularly true of data collected in surveys but is an important consideration in most studies. Careful study designers often assume a nonresponse rate or dropout rate that specifies what proportion of the originally planned sample will fail to provide data. We use this information to calculate the sample size to be used at the start of the study.

For example, if in Example 6.7 we expect only 50% of those contacted to respond, we would need to start with a sample size of 2 × 9388 = 18,776 to obtain usable information from 9388 undergraduates and parents of undergraduates.

355

Some cautions

We have already seen that small margins of error and high confidence can require large numbers of observations. You should also be keenly aware that any formula for inference is correct only in specific circumstances. If the government required statistical procedures to carry warning labels like those on drugs, most inference methods would have long labels. Our formula for estimating a population mean comes with the following list of warnings for the user:

The most important caution concerning confidence intervals is a consequence of the first of these warnings. The margin of error in a confidence interval covers only random sampling errors. The margin of error is obtained from the sampling distribution and indicates how much error can be expected because of chance variation in randomized data production.

356

Practical difficulties such as undercoverage and nonresponse in a sample survey cause additional errors. These errors can be larger than the random sampling error. This often happens when the sample size is large (so that is small). Remember this unpleasant fact when reading the results of an opinion poll or other sample survey. The practical conduct of the survey influences the trustworthiness of its results in ways that are not included in the announced margin of error.

Every inference procedure that we will meet has its own list of warnings. Because many of the warnings are similar to those we have mentioned, we will not print the full warning label each time. It is easy to state (from the mathematics of probability) conditions under which a method of inference is exactly correct. These conditions are never fully met in practice.

For example, no population is exactly Normal. Deciding when a statistical procedure should be used in practice often requires judgment assisted by exploratory analysis of the data. Mathematical facts are, therefore, only a part of statistics. The difference between statistics and mathematics can be stated thusly: mathematical theorems are true; statistical methods are often effective when used with skill.

Finally, you should understand what statistical confidence does not say. Based on our SRS in Example 6.3, we are 95% confident that the mean SATM score for the California students lies between 486 and 504. This says that this interval was calculated by a method that gives correct results in 95% of all possible samples. It does not say that the probability is 0.95 that the true mean falls between 486 and 504. No randomness remains after we draw a particular sample and compute the interval. The true mean either is or is not between 486 and 504. The probability calculations of standard statistical inference describe how often the method, not a particular sample, gives correct answers.