Confidence statements
Here is Gallup’s conclusion about the views of American adults about vaccinating children in short form: “The poll found that a slight majority of Americans, 54%, feel it is extremely important to vaccinate children. We are 95% confident that the truth about all American adults is within plus or minus 4 percentage points of this sample result.” Here is an even shorter form: “We are 95% confident that between 50% and 58% of all American adults feel it is extremely important to vaccinate children.” These are confidence statements.
Confidence statements
A confidence statement has two parts: a margin of error and a level of confidence. The margin of error says how close the sample statistic lies to the population parameter. The level of confidence says what percentage of all possible samples satisfy the margin of error.
A confidence statement is a fact about what happens in all possible samples and is used to say how much we can trust the result of one sample. The phrase “95% confidence” means “We used a sampling method that gives a result this close to the truth 95% of the time.” Here are some hints for interpreting confidence statements:
• The conclusion of a confidence statement always applies to the population, not to the sample. We know exactly the opinions of the 1015 people in the sample because Gallup interviewed them. The confidence statement uses the sample result to say something about the population of all American adults.
• Our conclusion about the population is never completely certain. Gallup’s sample might be one of the 5% that miss by more than 4 percentage points.
The telemarketer’s pause People who do sample surveys hate telemarketing. We all get so many unwanted sales pitches by phone that many people hang up before learning that the caller is conducting a survey rather than selling vinyl siding. Here’s a tip. Both sample surveys and telemarketers dial telephone numbers at random. Telemarketers automatically dial many numbers, and their sellers come on the line only after you pick up the phone. Once you know this, the telltale “telemarketer’s pause” gives you a chance to hang up before the seller arrives. Sample surveys have a live interviewer on the line when you answer.• A sample survey can choose to use a confidence level other than 95%. The cost of higher confidence is a larger margin of error. For the same sample, a 99% confidence statement requires a larger margin of error than 95% confidence. If you are content with 90% confidence, you get in return a smaller margin of error. Remember that our quick and approximate method gives the margin of error only for 95% confidence.
• It is usual to report the margin of error for 95% confidence. If a news report gives a margin of error but leaves out the confidence level, it’s pretty safe to assume 95% confidence.
• Want a smaller margin of error with the same confidence? Take a larger sample. Remember that larger samples have less variability. You can get as small a margin of error as you want and still have high confidence by paying for a large enough sample.
EXAMPLE 6 2012 election polls
In 2012, shortly before the presidential election, SurveyUSA, a polling organization, asked voters in several states who they would vote for. In Minnesota they asked a random sample of 574 likely voters, and 50% said they would vote for Barack Obama and 43% said Mitt Romney. SurveyUSA reported the margin of error to be plus or minus 4.2%. In Georgia they sampled 595 likely voters, and 44% said they would vote for Obama and 52% said Romney. The margin of error was reported to be plus or minus 4.1%.
There you have it: the sample of likely voters in Georgia was slightly larger, so the margin of error for conclusions about voters in Georgia is slightly smaller (4.1% compared to 4.2%). We are 95% confident that between 39.9% (that’s 44% minus 4.1%) and 48.1% (that’s 44% plus 4.1%) of likely voters in Georgia would vote for Obama. Note that the actual 2012 election results for Georgia were 45.5% for Obama, which is within the margin of error.
NOW IT’S YOUR TURN
Question 3.2
3.2 Voting and personal views. In May 2015, the Gallup Poll asked a random sample of 1024 American adults, “Thinking about how the abortion issue might affect your vote for major offices, would you only vote for a candidate who shares your views on abortion or consider a candidate’s position on abortion as just one of many important factors or not see abortion as a major issue?” It found that 21% of respondents said they will only vote for a candidate with the same views on abortion that they have. Suppose that the sample size had been 4000 rather than 1024. Find the margin of error for 95% confidence in this case. How does it compare with the margin of error for a sample of size 1024?
3.2 Now, n = 4000, so the margin of error for 95% confidence is
1√4000=163.24=.016
which is smaller than for the smaller sample (n = 1024).