STATISTICS IN SUMMARY
Chapter Specifics
• Pay attention to self-reported statistics. Ask exactly what a number measures and decide if it is a valid measure.
• Look for the context of the numbers and ask if there is important missing information.
• Look for inconsistencies, numbers that don’t agree as they should, and check for incorrect arithmetic.
• Compare numbers that are implausible—surprisingly large or small—with numbers you know are right.
• Be suspicious when numbers are too regular or agree too well with what their author would like to see.
• Look with special care if you suspect the numbers are put forward in support of some hidden agenda.
In Chapters 1 through 8, we have seen that we should always ask how the data were produced and exactly what was measured. Both affect the quality of any conclusions drawn. The goal is to gain insight by means of the numbers that make up our data. Numbers are most likely to yield insights to those who examine them properly. We need to develop “number sense,” the habit of asking if the numbers make sense. To assist you, in this chapter we have provided you with examples of bad data and of good data used wrongly. If you form the habit of looking at numbers properly, your friends will soon think that you are brilliant. They might even be right.
197
CASE STUDY EVALUATED How were the issues of “fudging the numbers” reported in the New York Times article mentioned in the Case Study that opened the chapter discovered? In some cases, unusual changes in rankings prompted a more careful look at the data on which the rankings were based. In other cases, suspicious numbers raised concerns. Examples of such suspicious numbers are discussed more thoroughly in the EESEE story “Quality of College Rankings.” This story describes some “suspect” data in the 1995 U.S. News & World Report annual rankings of accredited four-year colleges and universities. Read the story and then examine the data described in the story. Are any of the numbers suspect? Why? You might look at the questions in the EESEE story to help you identify possible sources of suspicious numbers.
Online Resources
• LearningCurve has good questions to check your understanding of the concepts.