Can you trust a sample?

imageGolfing at random Random drawings give all the same chance to be chosen, so they offer a fair way to decide who gets a scarce good—like a round of golf. Lots of golfers want to play the famous Old Course at St. Andrews, Scotland. A few can reserve in advance. Most must hope that chance favors them in the daily random drawing for tee times. At the height of the summer season, only one in six wins the right to pay £170 (about $260) for a round.

The Town Talk, Ann Landers, and mall interviews produce samples. We can’t trust results from these samples because they are chosen in ways that invite bias. The first question to ask of any sample is whether it was chosen at random. Opinion polls and other sample surveys carried out by people who know what they are doing use random sampling.

EXAMPLE 5 A Gallup Poll

A 2015 Gallup Poll on vaccinations asked the question, “From what you have heard or read, do you personally think certain vaccines do—or do not—cause autism in children, or are you unsure?” Gallup reported that the poll found that 52% of respondents were unsure. Is it actually the case that a majority of Americans are unsure about whether certain vaccines cause autism in children? Ask first how Gallup selected its sample. Later in the article we read this: “Results for this Gallup poll are based on telephone interviews conducted Feb. 28–March 1, 2015, on the Gallup U.S. Daily survey, with a random sample of 1,015 adults, aged 18 and older, living in all 50 U.S. states and the District of Columbia.” Gallup goes on to clarify that the sample included equal proportions of landline and cellular phone numbers selected using random-digit dialing.

31

This is a good start toward gaining our confidence. Gallup tells us what population it has in mind (people at least 18 years old living in the continental United States). We know that the sample from this population was of size 1015 and, most important, that it was chosen at random. There is more to consider in assessing a poll, and we will soon discuss this, but we have at least heard the comforting words “random sample.”