Nonsampling errors, especially nonresponse, are always with us. What should a careful sample survey do about this? First, substitute other households for the nonresponders. Because nonresponse is higher in cities, replacing nonresponders with other households in the same neighborhood may reduce bias. Once the data are in, all professional surveys use statistical methods to weight the responses in an attempt to correct sources of bias. If many urban households did not respond, the survey gives more weight to those that did respond. If too many women are in the sample, the survey gives more weight to the men. Here, for example, is part of a statement in the New York Times describing one of its sample surveys:

The results have been weighted to take account of household size and number of telephone lines into the residence and to adjust for variations in the sample relating to geographic region, sex, race, age and education.

The goal is to get results “as if” the sample matched the population in age, gender, place of residence, and other variables.

The practice of weighting creates job opportunities for statisticians. It also means that the results announced by a sample survey are rarely as simple as they seem to be. Gallup announces that it interviewed 1523 adults and found that 57% of them bought a lottery ticket in the last 12 months. It would seem that because 57% of 1523 is 868, Gallup found that 868 people in its sample had played the lottery. Not so. Gallup no doubt used some quite fancy statistics to weight the actual responses: 57% is Gallup’s best estimate of what it would have found in the absence of nonresponse. Weighting does help correct bias. It usually also increases variability. The announced margin of error must take this into account, creating more work for statisticians.