Sampling errors

Random sampling error is one kind of sampling error. The margin of error tells us how serious random sampling error is, and we can control it by choosing the size of our random sample. Another source of sampling error is the use of bad sampling methods, such as voluntary response. We can avoid bad methods. Other sampling errors are not so easy to handle. Sampling begins with a list of individuals from which we will draw our sample. This list is called the sampling frame. Ideally, the sampling frame should list every individual in the population. Because a list of the entire population is rarely available, most samples suffer from some degree of undercoverage.

65

Undercoverage

Undercoverage occurs when some groups in the population are left out of the process of choosing the sample.

image

If the sampling frame leaves out certain classes of people, even random samples from that frame will be biased. Using telephone directories as the frame for a telephone survey, for example, would miss everyone with an unlisted residential telephone number, everyone who cannot afford a residential phone, and everyone who has only a cell phone. Even if we consider only those with residential telephones, more than half the households in many large cities have unlisted numbers, so massive undercoverage and bias against urban areas would result. In fact, telephone surveys use random digit dialing equipment, which dials telephone numbers in selected regions at random. In effect, the sampling frame contains all residential telephone numbers.

EXAMPLE 1 Did we miss anyone?

Most opinion polls can’t afford even to attempt full coverage of the population of all adult residents of the United States. The interviews are done by telephone, thus missing the 2% of households without phones. Only households are contacted, so students in dormitories, prison inmates, and most members of the armed forces are left out. So are the homeless and people staying in shelters. Many polls interview only in English, which leaves some immigrant households out of their samples.

The kinds of undercoverage found in most sample surveys are most likely to leave out people who are young or poor or who move often. Nonetheless, random digit dialing comes close to producing a random sample of households with phones. Sampling errors in careful sample surveys are usually quite small. The real problems start when someone picks up (or doesn’t pick up) the phone. Now nonsampling errors take over.