How sample surveys go wrong

64

Random sampling eliminates bias in choosing a sample and allows control of variability. So once we see the magic words “randomly selected” and “margin of error,” do we know we have trustworthy information before us? It certainly beats voluntary response, but not always by as much as we might hope. Sampling in the real world is more complex and less reliable than choosing an SRS from a list of names in a textbook exercise. Confidence statements do not reflect all the sources of error that are present in practical sampling.

Errors in sampling

Sampling errors are errors caused by the act of taking a sample. They cause sample results to be different from the results of a census.

Random sampling error is the deviation between the sample statistic and the population parameter caused by chance in selecting a random sample. The margin of error in a confidence statement includes only random sampling error.

Nonsampling errors are errors not related to the act of selecting a sample from the population. They can be present even in a census.

Most sample surveys are afflicted by errors other than random sampling errors. These errors can introduce bias that makes a confidence statement meaningless. Good sampling technique includes the art of reducing all sources of error. Part of this art is the science of statistics, with its random samples and confidence statements. In practice, however, good statistics isn’t all there is to good sampling. Let’s look at sources of errors in sample surveys and at how samplers combat them.