In the previous section, we used the shape, center, and spread as ways to describe the overall pattern of any distribution for a quantitative variable. In this section, we will learn specific ways to use numbers to measure the center and spread of a distribution. The numbers, like the graphs of Section 1.1, are aids to understanding the data, not “the answer” in themselves.

Measuring center: The mean

24

A description of a distribution almost always includes a measure of its center. The most common measure of center is the ordinary arithmetic average, or mean.

The (capital Greek sigma) in the formula for the mean is short for “add them all up.” The subscripts on the observations are just a way of keeping the observations distinct. They do not necessarily indicate order or any other special facts about the data. The bar over the indicates the mean of all the -values. Pronounce the mean as “x-bar.” This notation is very common. When writers who are discussing data use or , they are talking about a mean.

In practice, you can key the data into your calculator and hit the Mean key. You don't have to actually add and divide. But you should know that this is what the calculator is doing.

Exercise 1.33 illustrates an important fact about the mean as a measure of center: it is sensitive to the influence of one or more extreme observations. These may be outliers, but a skewed distribution that has no outliers will also pull the mean toward its long tail. Because the mean cannot resist the influence of extreme observations, we say that it is not a resistant measure of center.

Measuring center: The median

In Section 1.1, we used the midpoint of a distribution as an informal measure of center. The median is the formal version of the midpoint, with a specific rule for calculation.

Note that the formula does not give the median, just the location of the median in the ordered list. Medians require little arithmetic, so they are easy to find by hand for small sets of data. Arranging even a moderate number of observations in order is very tedious, however, so that finding the median by hand for larger sets of data is unpleasant. Even simple calculators have an button, but you will need software or a graphing calculator to automate finding the median.

Note that you can use the stemplot directly to compute the median. In the stemplot the cases are already ordered, and you simply need to count from the top or the bottom to the desired location.

Comparing the mean and the median

Exercises 1.33 (page 24) and 1.38 (page 26) illustrate an important difference between the mean and the median. Suriname pulls the mean time to start a business up from 16 days to 24 days. The increase in the median is very small, from 11.5 days to 12 days.

The median is more resistant than the mean. If the largest starting time in the data set was 1200 days, the median for all 25 countries would still be 12 days. The largest observation just counts as one observation above the center, no matter how far above the center it lies. The mean uses the actual value of each observation and so will chase a single large observation upward.

The best way to compare the response of the mean and median to extreme observations is to use an interactive applet that allows you to place points on a line and then drag them with your computer’s mouse. Exercises 1.60 to 1.62 (page 37) use the Mean and Median applet on the website for this book to compare mean and median.

The mean and median of a symmetric distribution are close together. If the distribution is exactly symmetric, the mean and median are exactly the same. In a skewed distribution, the mean is farther out in the long tail than is the median.

Consider the prices of existing single-family homes in the United States.¹⁸ The mean price in 2013 was $245,700, while the median was $197,400. This distribution is strongly skewed to the right. There are many moderately priced houses and a few very expensive mansions. The few expensive houses pull the mean up but do not affect the median.

27

Reports about house prices, incomes, and other strongly skewed distributions usually give the median (page midpoint”) rather than the mean (page arithmetic average”). However, if you are a tax assessor interested in the total value of houses in your area, use the mean. The total is the mean times the number of houses, but it has no connection with the median. The mean and median measure center in different ways, and both are useful.

Measuring spread: The quartiles

A measure of center alone can be misleading. Two nations with the same median household income are very different if one has extremes of wealth and poverty and the other has little variation among households. A drug with the correct mean concentration of active ingredient is dangerous if some batches are much too high and others much too low. We are interested in the spread or variability of incomes and drug potencies as well as their centers. The simplest useful numerical description of a distribution consists of both a measure of center and a measure of spread.

One way to measure spread is to give the smallest and largest observations. For example, the times to start a business in our data set that included Suriname ranged from 2 to 208 days. Without Suriname, the range is 2 to 53 days. These largest and smallest observations show the full spread of the data and are highly influenced by outliers.

28

We can improve our description of spread by also giving several percentiles. The th percentile of a distribution is the value such that percent of the observations fall at or below it. The median is just the 50th percentile, so the use of percentiles to report spread is particularly appropriate when the median is the measure of center.

The most commonly used percentiles other than the median are the quartiles. The first quartile is the 25th percentile, and the third quartile is the 75th percentile. That is, the first and third quartiles show the spread of the middle half of the data. (The second quartile is the median itself.) To calculate a percentile, arrange the observations in increasing order, and count up the required percent from the bottom of the list. Our definition of percentiles is a bit inexact because there is not always a value with exactly percent of the data at or below it. We are content to take the nearest observation for most percentiles, but the quartiles are important enough to require an exact recipe. The rule for calculating the quartiles uses the rule for the median.

Here is an example that shows how the rules for the quartiles work for both odd and even numbers of observations.

Notice that the quartiles are resistant. For example, would have the same value if the highest start time was 530 days rather than 53 days.

There are slight differences in the methods used by software to compute percentiles. However, the results will generally be quite similar, except in cases where the sample sizes are very small.

Be careful when several observations take the same numerical value. Write down all the observations, and apply the rules just as if they all had distinct values.

The five-number summary and boxplots

29

The smallest and largest observations tell us little about the distribution as a whole, but they give information about the tails of the distribution that is missing if we know only , and . To get a quick summary of both center and spread, combine all five numbers. The result is the five-number summary and a graph based on it.

You can draw boxplots either horizontally or vertically. Be sure to include a numerical scale in the graph. When you look at a boxplot, first locate the median, which marks the center of the distribution. Then look at the spread. The quartiles show the spread of the middle half of the data, and the extremes (the smallest and largest observations) show the spread of the entire data set. We now have the tools for a preliminary examination of the customer service center call lengths.

30

Because of the skewness in this distribution, we selected a software option to plot extreme points individually in Figure 1.14. This is one of several different ways to improve the appearance of boxplots for particular data sets. These variations are called modified boxplots.

Boxplots can show the symmetry or skewness of a distribution. In a symmetric distribution, the first and third quartiles are equally distant from the median. This is not what we see in Figure 1.14. Here, the distribution is skewed to the right. The third quartile is farther above the median than the first quartile is below it. The extremes behave the same way. Boxplots do not always give a clear indication of the nature of a skewed set of data. For example, the quartiles may indicate right-skewness, while the whiskers indicate left-skewness.

Boxplots are particularly useful for comparing several distributions. Here is an example.

Measuring spread: the standard deviation

The five-number summary is not the most common numerical description of a distribution. That distinction belongs to the combination of the mean to measure center and the standard deviation to measure spread. The standard deviation measures spread by looking at how far the observations are from their mean.

Notice that the “average” in the variance divides the sum by 1 less than the number of observations, that is, rather than . The reason is that the deviations always sum to exactly 0, so that knowing of them determines the last one. Only of the squared deviations can vary freely, and we average by dividing the total by . The number is called the degrees of freedom of the variance or standard deviation. Many calculators offer a choice between dividing by and dividing by , so be sure to use .

In practice, use software or your calculator to obtain the standard deviation from keyed-in data. Doing an example step-by-step will help you understand how the variance and standard deviation work, however.

More important than the details of hand calculation are the properties that determine the usefulness of the standard deviation:

You may rightly feel that the importance of the standard deviation is not yet clear. We will see in the next section that the standard deviation is the natural measure of spread for an important class of symmetric distributions, the Normal distributions. The usefulness of many statistical procedures is tied to distributions with particular shapes. This is certainly true of the standard deviation.

Choosing measures of center and spread

How do we choose between the five-number summary and and to describe the center and spread of a distribution? Because the two sides of a strongly skewed distribution have different spreads, no single number such as describes the spread well. The five-number summary, with its two quartiles and two extremes, does a better job.

33

Beyond The Basics: Risk And Return

A central principle in the study of investments is that taking bigger risks is rewarded by higher returns, at least on the average over long periods of time. It is usual in finance to measure risk by the standard deviation of returns on the grounds that investments whose returns show a large spread from year to year are less predictable and, therefore, more risky than those whose returns have a small spread. Compare, for example, the approximate mean and standard deviation of the annual percent returns on American common stocks and U.S. Treasury bills over a 50-year period starting in 1950:

Stocks are risky. They went up 14% per year on the average during this period, but they dropped almost 28% in the worst year. The large standard deviation reflects the fact that stocks have produced both large gains and large losses. When you buy a Treasury bill, on the other hand, you are lending money to the government for one year. You know that the government will pay you back with interest. That is much less risky than buying stocks, so (on the average) you get a smaller return.

Are and good summaries for distributions of investment returns? Figures 1.16(a) and 1.16(b) display stemplots of the annual returns for both investments. You see that returns on Treasury bills have a right-skewed distribution. Convention in the financial world calls for and because some parts of investment theory use them. For describing this right-skewed distribution, however, the five-number summary would be more informative.

34

Remember that a graph gives the best overall picture of a distribution. Numerical measures of center and spread report specific facts about a distribution, but they do not describe its entire shape. Numerical summaries do not disclose the presence of multiple peaks or gaps, for example. Always plot your data.