712

Which of four advertising offers mailed to sample households produces the highest sales in dollars? Which of six brands of automobile tires wears longest? In each of these settings, we wish to compare several groups or treatments. Also, the data are subject to sampling variability. We would not, for example, expect the same sales data if we mailed the advertising offers to different sets of households. We also would not expect the same tread lifetime data if the tire experiment were repeated under similar conditions.

Because of this variability, we pose the question for inference in terms of the mean response. The statistical methodology for comparing several means is called analysis of variance, or simply ANOVA. In this and the following sections, we examine the basic ideas and assumptions that are needed for ANOVA. Although the details differ, many of the concepts are similar to those discussed in the two-sample case.

We consider two ANOVA techniques. When there is only one way to classify the populations of interest, we use one-way ANOVA to analyze the data. We call the categorical explanatory variable that classifies these populations a factor. For example, to compare the average tread lifetimes of six specific brands of tires, we use one-way ANOVA with tire brand as our factor. This chapter presents the details for one-way ANOVA.

In many other comparison studies, there is more than one way to classify the populations. For the advertising study, the mail-order firm might also consider mailing the offers using two different envelope styles. Will each offer draw more sales on the average when sent in an attention-grabbing envelope? Analyzing the effect of the advertising offer and envelope style together requires two-way ANOVA. This technique is discussed in Chapter 15.

The ANOVA setting

One-way analysis of variance is a statistical method for comparing several population means. We draw a simple random sample (SRS) from each population and use the data to test the null hypothesis that the population means are all equal. Consider the following two examples.

These two examples are similar in that

There is, however, an important difference. Example 14.1 describes an experiment in which stores are randomly assigned to layouts. Example 14.2 is an observational study in which customers are selected during a particular time of day and not all agree to provide data. We treat our samples of customers as random samples even though this is only approximately true.

In both examples, we use ANOVA to compare the mean responses. The response variable is the number of magazines sold in the first example and the age of the customer in the second. The same ANOVA methods apply to data from randomized experiments and to data from random samples. Do keep the method by which data are produced in mind when interpreting the results, however. A strong case for causation is best made by a randomized experiment.

Comparing means

The question we ask in ANOVA is, “Do all groups have the same population mean?” We often use the term groups for the populations to be compared in a one-way ANOVA. To answer this question, we compare sample means. Figure 14.1 displays the sample means for Example 14.1. Layout 2 has the highest average sales. But are the observed differences among the sample means just the result of chance variation? We do not expect sample means to be equal even if the population means are all identical.

The purpose of ANOVA is to assess whether the observed differences among sample means are statistically significant. In other words, could variation in sample means this large be plausibly due to chance, or is it good evidence for a difference among the population means? This question can’t be answered from the sample means alone. Because the standard deviation of a sample mean is the population standard deviation divided by , the answer depends upon both the variation within the groups of observations and the sizes of the samples.

Side-by-side boxplots help us see the within-group variation. Compare Figures 14.2(a) and 14.2(b). The sample medians are the same in both figures, but the large variation within the groups in Figure 14.2(a) suggests that the differences among sample medians could be due simply to chance. The data in Figure 14.2(b) are much more convincing evidence that the populations differ.

Even the boxplots omit essential information, however. To assess the observed differences, we must also know how large the samples are. Nonetheless, boxplots are a good preliminary display of ANOVA data. While ANOVA compares means and boxplots display medians, these two measures of center will be close together for distributions that are nearly symmetric. If the distributions are not symmetric, we may consider a transformation prior to displaying and analyzing the data.

714

The two-sample statistic

Two-sample statistics compare the means of two populations. If the two populations are assumed to have equal but unknown standard deviations and the sample sizes are both equal to , the statistic is

The square of this statistic is

If we use ANOVA to compare two populations, the ANOVA statistic is exactly equal to this . Thus, we can learn something about how ANOVA works by looking carefully at the statistic in this form.

The numerator in the statistic measures the variation between the groups in terms of the difference between their sample means and . This is multiplied by a factor for the common sample size . The numerator can be large because of a large difference between the sample means or because the common sample size is large.

The denominator measures the variation within groups by , the pooled estimator of the common variance. If the within-group variation is small, the same variation between the groups produces a larger statistic and, thus, a more significant result.

Although the general form of the statistic is more complicated, the idea is the same. To assess whether several populations all have the same mean, we compare the variation among the means of several groups with the variation within groups. Because we are comparing variation, the method is called analysis of variance.

715

An overview of ANOVA

ANOVA tests the null hypothesis that the population means are all equal. The alternative is that they are not all equal. This alternative could be true because all of the means are different or simply because one of them differs from the rest. Because this alternative is a more complex situation than comparing just two populations, we need to perform some further analysis if we reject the null hypothesis. This additional analysis allows us to draw conclusions about which population means differ from others and by how much.

The computations needed for ANOVA are more lengthy than those for the test. For this reason, we generally use computer software to perform the calculations. Automating the calculations frees us from the burden of arithmetic and allows us to concentrate on interpretation.

We should always start an ANOVA with a careful examination of the data using graphical and numerical summaries in order to get an idea of what to expect from the analysis and also to check for unusual patterns in the data. Just as in linear regression and the two-sample t methods, outliers and extreme departures from Normality can invalidate the computed results.

716

In the setting of Case 14.1, we have an experiment in which participants were randomly assigned to one of three groups. Each of these groups has a mean, and our inference asks questions about these means. The participants in this study were all recruited through the same university. They also participated in return for financial payment.

Formulating a clear definition of the populations being compared with ANOVA can be difficult. Often, some expert judgment is required, and different consumers of the results may have differing opinions. Whether the samples in this study should be considered as SRSs from the population of consumers at the university or from the population of all consumers in the United States is open for debate. Regardless, we are more confident in generalizing our conclusions to similar populations when the results are clearly significant than when the level of significance just barely passes the standard of .

We first ask whether or not there is sufficient evidence in the data to conclude that the corresponding population means are not all equal. Our null hypothesis here states that the population mean score is the same for all three groups. The alternative is that they are not all the same. Rejecting the null hypothesis that the means are all the same using ANOVA is not the same as concluding that all the means are different from one another. The ANOVA null hypothesis can be false in many different ways. Additional analysis is required to compare the three means.

The researchers hypothesize that these moral reasoning strategies should allow a consumer to continue to support the company. Therefore, a reasonable question to ask is whether or not the mean for the control group is smaller than the others. When there are particular versions of the alternative hypothesis that are of interest, we use contrasts to examine them. Note that, to use contrasts, it is necessary that the questions of interest be formulated before examining the data. It is cheating to make up these questions after analyzing the data.

If we have no specific relations among the means in mind before looking at the data, we instead use a multiple-comparisons procedure to determine which pairs of population means differ significantly. In the next section, we explore both contrasts and multiple comparisons in detail.

The ANOVA model

When analyzing data, the following equation reminds us that we look for an overall pattern and deviations from it:

In the regression model of Chapter 10, the FIT was the population regression line, and the RESIDUAL represented the deviations of the data from this line. We now apply this framework to describe the statistical models used in ANOVA. These models provide a convenient way to summarize the conditions that are the foundation for our analysis. They also give us the necessary notation to describe the calculations needed.

First, recall the statistical model for a random sample of observations from a single Normal population with mean and standard deviation . If the observations are

we can describe this model by saying that the are an SRS from the distribution. Another way to describe the same model is to think of the ’s varying about their population mean. To do this, write each observation as

The are then an SRS from the distribution. Because is unknown, the ’s cannot actually be observed. This form more closely corresponds to our

way of thinking. The FIT part of the model is represented by . It is the systematic part of the model. The RESIDUAL part is represented by . It represents the deviations of the data from the fit and is due to random, or chance, variation.

There are two unknown parameters in this statistical model: and . We estimate by , the sample mean, and by , the sample standard deviation. The differences are the residuals and correspond to the in this statistical model.

The model for one-way ANOVA is very similar. We take random samples from each of different populations. The sample size is for the th population. Let represent the th observation from the th population. The population means are the FIT part of the model and are represented by . The random variation, or RESIDUAL, part of the model is represented by the deviations of the observations from the means.

719

Note that the sample sizes may differ, but the standard deviation is assumed to be the same in all the populations. Figure 14.6 pictures this model for . The three population means are all different, but the spreads of the three Normal distributions are the same, reflecting the condition that all three populations have the same standard deviation.

The ANOVA model assumes that the deviations are independent and Normally distributed with mean 0 and standard deviation . For Case 14.1, we have clear evidence that the data are not Normal. The values are numbers that can only range from 0 to 100, and we saw some skewness in the group distributions (Figure 14.3, page 716). However, because our inference is based on the sample means, which will be approximately Normal given the sample sizes, we are not overly concerned about this violation of model assumptions.

720

Estimates of population parameters

The unknown parameters in the statistical model for ANOVA are the population means and the common population standard deviation . To estimate we use the sample mean for the th group:

The residuals reflect the variation about the sample means that we see in the data and are used in the calculations of the sample standard deviations

In addition to the deviations being Normally distributed, the ANOVA model also states that the population standard deviations are all equal. Before estimating , it is important to check this equality assumption using the sample standard deviations. Most computer software provides at least one test for the equality of standard deviations. Unfortunately, many of these tests lack robustness against non-Normality.

Because ANOVA procedures are not extremely sensitive to unequal standard deviations, we do not recommend a formal test of equality of standard deviations as a preliminary to the ANOVA. Instead, we use the following rule of thumb.

If there is evidence of unequal population standard deviations, we generally try to transform the data so that they are approximately equal. We might, for example, work with or log . Fortunately, we can often find a transformation that both makes the group standard deviations more nearly equal and also makes the distributions of observations in each group more nearly Normal (see Exercises 14.65 and 14.66 later in the chapter). If the standard deviations are markedly different and cannot be made similar by a transformation, inference requires different methods that are beyond the scope of this book.

721

When we assume that the population standard deviations are equal, each sample standard deviation is an estimate of . To combine these into a single estimate, we use a generalization of the pooling method introduced in Chapter 7 (page 387).

Pooling gives more weight to groups with larger sample sizes. If the sample sizes are equal, is just the average of the sample variances. Note that is not the average of the sample standard deviations.

722

Testing hypotheses in one-way ANOVA

Comparison of several means is accomplished by using an statistic to compare the variation among groups with the variation within groups. We now show how the statistic expresses this comparison. Calculations are organized in an ANOVA table, which contains numerical measures of the variation among groups and within groups.

First, we must specify our hypotheses for one-way ANOVA. As usual, represents the number of populations to be compared.

We now use the moral strategy study (Case 14.1, page 715) to illustrate how to do a one-way ANOVA. Because the calculations are generally performed using statistical software on a computer, we focus on interpretation of the output.

723

Because we feel our data satisfy these three conditions, we proceed with the analysis of variance.

This concludes the basics of ANOVA: we first state the hypotheses, then verify that the conditions for ANOVA are met (Example 14.7), and finally look at the statistic and its -value, and state a conclusion (Example 14.8).

The ANOVA table

Software ANOVA output contains more than simply the test statistic. The additional information shows, among other things, where the test statistic comes from.

The information in an analysis of variance is organized in an ANOVA table. In the software output in Figure 14.8, the columns of this table are labeled “Source,” “DF,” “Sum of Squares,” “Mean Square,” “F Ratio,” and “” The rows are labeled “Grp,” “Error,” and “C. Total.” These are the three sources of variation in the one-way ANOVA. “Grp” was the name used in entering the data to distinguish the three treatment groups. Other software may use different column and row names but the rest of the table will be similar.

The Grp row in the table corresponds to the FIT term in our way of thinking. It gives information related to the variation among group means. The ANOVA model allows the groups to have different means.

The Error row in the table corresponds to the RESIDUAL term in . It gives information related to the variation within groups. The term error is most appropriate for experiments in the physical sciences, where the observations within a group differ because of measurement error. In business and the biological and social sciences, on the other hand, the within-group variation is often due to the fact that not all firms or plants or people are the same. This sort of variation is not due to errors and is better described as residual.

Finally, the Total row in the table corresponds to the DATA term in our framework. So, for analysis of variance,

translates into

725

The ANOVA idea is to break the total variation in the responses into two parts: the variation due to differences among the group means and that due to differences within groups. Variation is expressed by sums of squares. We use SSG, SSE, and SST for the sums of squares for groups, error, and total, respectively. Each sum of squares is the sum of the squares of a set of deviations that expresses a source of variation. SST is the sum of squares of , which measure variation of the responses around their overall mean. Variation of the group means around the overall mean is measured by SSG. Finally, SSE is the sum of squares of the deviations of each observation from its group mean.

It is always true that . This is the algebraic version of the ANOVA idea: total variation is the sum of between-group variation and within-group variation.

Associated with each sum of squares is a quantity called the degrees of freedom Because SST measures the variation of all observations around the overall mean, its degrees of freedom are , the degrees of freedom for the sample variance of the responses. Similarly, because SSG measures the variation of the sample means around the overall mean, its degrees of freedom are . Finally, SSE is the sum of squares of the deviations . Here, we have observations being compared with sample means and .

Note that the degrees of freedom add in the same way that the sums of squares add. That is, .

For each source of variation, the mean square is the sum of squares divided by the degrees of freedom. You can verify this by doing the divisions for the values given on the output in Figure 14.8. Generally, the ANOVA table includes mean squares only for the first two sources of variation. The mean square corresponding to the total source is the sample variance that we would calculate assuming that we have one sample from a single population—that is, assuming that the means of the three groups are the same.

726

We can use the mean square for error to find , the pooled estimate of the parameter of our model. It is true in general that

In other words, the mean square for error is an estimate of the within-group variance, . The estimate of is therefore the square root of this quantity. So,

The test

If is true, there are no differences among the group means. In that case, MSG will reflect only chance variation and should be about the same as MSE. The ANOVA statistic simply compares these two mean squares, . Thus, this statistic is near 1 if is true and tends to be larger if is true. In our example, and , so the ANOVA statistic is

When is true, the statistic has an distribution that depends on two numbers: the degrees of freedom for the numerator and the degrees of freedom for the denominator. These degrees of freedom are those associated with the mean squares in the numerator and denominator of the statistic. For one-way ANOVA, the degrees of freedom for the numerator are and the degrees of freedom for the denominator are . We use the notation for this distribution.

727

The One-Way ANOVA applet is an excellent way to see how the value of the statistic and its -value depend on both the variability of the data within the groups and the differences between the group means. Exercises 14.12 and 14.13 (page 729) make use of this applet.

The ANOVA test shares the robustness of the two-sample test. It is relatively insensitive to moderate non-Normality and unequal variances, especially when the sample sizes are similar. The constant variance assumption is more important when we compare means using contrasts and multiple comparisons, additional analyses that are generally performed after the ANOVA test. We discuss these analyses in the next section.

Tables of critical values are available for use when software does not give the -value. Table E in the back of the book contains the critical values for probabilities , and 0.001. For one-way ANOVA, we use critical values from the table corresponding to degrees of freedom in the numerator and degrees of freedom in the denominator. When determining the P-value, remember that the test is always one-sided because any differences among the group means tend to make large.

The following display shows the general form of a one-way ANOVA table with the statistic. The formulas in the sum of squares column can be used for calculations in small problems. There are other formulas that are more efficient for hand or calculator use, but ANOVA calculations are usually done by computer software.

728

One other item given by some software for ANOVA is worth noting. For an analysis of variance, we define the coefficient of determination as

The coefficient of determination plays the same role as the squared multiple correlation in a multiple regression. We can easily calculate the value from the ANOVA table entries.

About 14% of the variation in the likelihood scores is explained by the different groups. The other 86% of the variation is due to participant-to-participant variation within each of the groups. We can see this in the histograms of Figure 14.3 (page 716) and the boxplots of Figure 14.4 (page 716). Each of the groups has a large amount of variation, and there is a substantial amount of overlap in the distributions. The fact that we have strong evidence against the null hypothesis that the three population means are all the same does not tell us that the distributions of values are far apart.

Using software

We have used JMP to illustrate the analysis of the moral strategy study data. Other statistical software gives similar output, and you should be able to extract all the ANOVA information we have discussed. Here is an example on which to practice this skill.

731

BEYOND THE BASICS: Testing the Equality of Spread

The two most basic descriptive features of a distribution are its center and spread. We have described procedures for inference about population means for Normal populations and found that these procedures are often useful for non-Normal populations as well. It is natural to turn next to inference about spread.

While the standard deviation is a natural measure of spread for Normal distributions, it is not for distributions in general. In fact, because skewed distributions have unequally spread tails, no single numerical measure is adequate to describe the spread of a skewed distribution. Because of this, we recommend caution when testing equal standard deviations and interpreting the results.

Most formal tests for equal standard deviations are extremely sensitive to non- Normal populations. Of the tests commonly available in software packages, we suggest using the modified Levene’s (or Brown-Forsythe) test due to its simplicity and robustness against non-Normal data.⁴ The test involves performing a one-way ANOVA on a transformation of the response variable, constructed to measure the spread in each group. If the populations have the same standard deviation, then the average deviation from the population center should also be the same.

732

This test uses a more robust measure of deviation replacing the mean with the median and replacing squaring with the absolute value. Also, the transformed response variable is straightforward to create, so this test can easily be performed regardless of whether or not your software specifically has it.

Remember that our rule of thumb (page 720) is used to assess whether different standard deviations will impact the ANOVA results. It’s not a formal test that the standard deviations are equal. In Exercises 14.65 and 14.66 (pages 755–756), we examine a transformation of the likelihood scores that better stabilizes the variances. Part of those exercises will be a comparison of the transformed likelihood score ANOVA results to the results of this section.