4.11STATISTICS IN SUMMARY
Chapter Specifics
• Sampling in the real world is complex. Even professional sample surveys don’t give exactly correct information about the population.
• There are many potential sources of error in sampling. The margin of error announced by a sample survey covers only random sampling error, the variation due to chance in choosing a random sample.
• Other types of error are in addition to the margin of error and can’t be directly measured. Sampling errors come from the act of choosing a sample. Random sampling error and undercoverage are common types of sampling error. Undercoverage occurs when some members of the population are left out of the sampling frame, the list from which the sample is actually chosen.
• The most serious errors in most careful surveys, however, are nonsampling errors. These have nothing to do with choosing a sample—they are present even in a census.
• The single biggest problem for sample surveys is nonresponse: subjects can’t be contacted or refuse to answer.
• Mistakes in handling the data (processing errors) and incorrect answers by respondents (response errors) are other examples of nonsampling errors.
• Finally, the exact wording of questions has a big influence on the answers.
• People who design sample surveys use statistical techniques that help correct nonsampling errors, and they also use probability samples more complex than simple random samples, such as stratified samples.
82
• You can assess the quality of a sample survey quite well by just looking at the basics: use of random samples, sample size and margin of error, the rate of nonresponse, and the wording of the questions.
In Chapter 3, we saw that random samples can provide a sound basis for drawing conclusions about a population parameter. In this chapter, we learned that even when we take a random sample, our conclusions can be weakened by undercoverage, processing errors, response errors, nonresponse, and wording of questions. We must pay careful attention to every aspect of how we collect data to ensure that the conclusions we make are valid. In some cases, more complex probability samples, such as stratified samples, can help correct nonsampling errors. This chapter provides a list of questions you can ask to help you assess the quality of the results of samples collected by someone else.
CASE STUDY EVALUATED Use what you have learned in this chapter to evaluate the Case Study that opened the chapter. In particular, do the following.
1. Answer the questions given in the section “Questions to Ask before You Believe a Poll” on page 80.
2. Are the results of the Pew poll useless? You may want to refer to the discussion on pages 76–79.
Online Resources
• The video technology manuals explain how to select an SRS using Excel, JMP, Minitab, and the TI 83/84.
• The Statistical Applet Simple Random Sample can be used to select a simple random sample when the number of labels is 144 or fewer.