The “margin of error” that sample surveys announce translates sampling variability of the kind pictured in Figures 3.1 and 3.2 into a statement of how much confidence we can have in the results of a survey. Let’s start with the kind of language we hear so often in the news.
What margin of error means
“Margin of error plus or minus 4 percentage points” is shorthand for this statement:
If we took many samples using the same method we used to get this one sample, 95% of the samples would give a result within plus or minus 4 percentage points of the truth about the population.
Take this step-by-step. A sample chosen at random will usually not estimate the truth about the population exactly. We need a margin of error to tell us how close our estimate comes to the truth. But we can’t be certain that the truth differs from the estimate by no more than the margin of error. Although 95% of all samples come this close to the truth, 5% miss by more than the margin of error. We don’t know the truth about the population, so we don’t know if our sample is one of the 95% that hit or one of the 5% that miss. We say we are 95% confident that the truth lies within the margin of error.
EXAMPLE 3 Understanding the news
Here’s what the TV news announcer says: “A new Gallup Poll finds that a slight majority of 54% of American adults feel it is extremely important that parents vaccinate their children. The margin of error for the poll was 4 percentage points.” Plus or minus 4% starting at 54% is 50% to 58%. Most people think Gallup claims that the truth about the entire population lies in that range.
46
This is what the results from Gallup actually mean: “For results based on a sample of this size, one can say with 95% confidence that the error attributable to sampling and other random effects could be plus or minus 4 percentage points for American adults.” That is, Gallup tells us that the margin of error includes the truth about the entire population for only 95% of all its samples—“95% confidence” is shorthand for that. The news report left out the “95% confidence.”
Finding the margin of error exactly is a job for statisticians. You can, however, use a simple formula to get a rough idea of the size of a sample survey’s margin of error. The reasoning behind this formula and an exact calculation of the margin of error are discussed in Chapter 21. For now, we introduce an approximate calculation for the margin of error.
A quick and approximate method for the margin of error
Use the sample proportion from a simple random sample of size n to estimate an unknown population proportion p. The margin of error for 95% confidence is approximately equal to .
EXAMPLE 4 What is the margin of error?
The Gallup Poll in Example 1 interviewed 1015 people. The margin of error for 95% confidence will be about
Gallup announced a margin of error of 4%, and our quick method gave us 3.1%. Our quick and approximate method can disagree a bit with Gallup’s for two reasons. First, polls usually round their announced margin of error to the nearest whole percent to keep their press releases simple. Second, our rough formula works for an SRS. We will see in the next chapter that most national samples are more complicated than an SRS in ways that tend to slightly increase the margin of error. In fact, Gallup’s survey methods section for this particular poll included the statement, “Each sample of national adults includes a minimum quota of 50% cellphone respondents and 50% landline respondents, with additional minimum quotas by time zone within region. Landline and cellular telephone numbers are selected using random-digit-dial methods.” While these methods go beyond what we will study in the next chapter, the complexity of collecting a national sample increases the value of the margin of error.
47
Our quick and approximate method also reveals an important fact about how margins of error behave. Because the sample size n appears in the denominator of the fraction, larger samples have smaller margins of error. We knew that. Because the formula uses the square root of the sample size, however, to cut the margin of error in half, we must use a sample four times as large.
EXAMPLE 5 Margin of error and sample size
In Example 2, we compared the results of taking many SRSs of size n = 100 and many SRSs of size n = 1015 from the same population. We found that the variability of the middle 95% of the sample results was about three times larger for the smaller samples.
Our quick formula estimates the margin of error for SRSs of size 1015 to be about 3.1%. The margin of error for SRSs of size 100 is about
Because 1015 is roughly 10 times 100 and the square root of 10 is 3.1, the margin of error is about three times larger for samples of 100 people than for samples of 1015 people.
NOW IT’S YOUR TURN
3.1 Voting and personal views. In May 2015, the Gallup Poll asked a random sample of 1024 American adults, “Thinking about how the abortion issue might affect your vote for major offices, would you only vote for a candidate who shares your views on abortion or consider a candidate’s position on abortion as just one of many important factors or not see abortion as a major issue?” It found that 21% of respondents said they will only vote for a candidate with the same views on abortion that they have. What is the approximate margin of error for 95% confidence?