13.1 One-Way Within-Groups ANOVA

There is not much difference between chapters 12 and 13. Chapter 12 taught you how to conduct the multiple-group equivalent of an independent-samples t test, the one-way between-groups ANOVA. You also learned how to calculate effect size and conduct a post hoc test for a one-way between-groups ANOVA. Chapter 13 draws on the same concepts, but this time you will be learning how to conduct the multiple-group equivalent of a paired-samples t test, called a one-way within-groups ANOVA (also known as a repeated-measures ANOVA). Just as you did for the one-way between-groups ANOVA, you will learn how to calculate effect size and conduct a post hoc test for the one-way within groups ANOVA. If you understand how a between-groups design differs from a within-groups design, then you already understand the key concept in this chapter.

EXAMPLE 13.1

Have you ever participated in a taste test? If you have, then you were probably a participant in a within-groups experiment. About a decade ago, when pricier microbrew beers were becoming popular in North America, the journalist James Fallows, who loves beer, found himself spending increasingly more on a bottle of beer. He began to wonder whether he was getting his money’s worth for these more expensive beers. So, he recruited 12 colleagues, all self-professed beer snobs, to participate in a taste test to see whether they really could tell whether a beer was expensive or cheap (Fallows, 1999).

329

Taste Tests Are Within-Groups Experiments In a taste test, every person tries every flavor to determine a favorite, so a taste test is an experiment in which every participant is in every group, or condition. If the order of the flavors is varied for each person, the researcher is using counterbalancing.
Meng Chenguang/ZUMAPRESS/Newscom

Fallows wanted to know whether his recruits could distinguish among widely available American beers that were categorized into three groups based on price—“high-end” beers like Sam Adams, “mid-range” beers like Budweiser, and “cheap” beers like Busch. All of these beers are lagers, a type of beer chosen because it can be found at every price point. Here are data—mean scores on a scale of 0–100 for each category of beer—for five of the participants. (Note: For teaching purposes, the means are slightly different and have been rounded to the nearest whole number; the take-home data story, however, remains the same.)

Participant Cheap Beer Mid-Range Beer High-End Beer
1 40 30 53
2 42 45 65
3 30 38 64
4 37 32 43
5 23 28 38

MASTERING THE CONCEPT

13.1: We use a one-way within-groups ANOVA when there is one independent variable with at least three levels, a scale dependent variable, and participants who are in every group.

The Benefits of Within-Groups ANOVA

Fallows only reported his overall findings. If he had conducted hypothesis testing, then he would have used a one-way within-groups ANOVA, the appropriate statistic when there is one nominal or ordinal independent variable (type of beer), an independent variable with more than two levels (cheap, mid-range, and high-end), a scale-dependent variable (ratings of beers), and participants who experience every level of the independent variable (each participant tasted the beers in every category).

MASTERING THE CONCEPT

13.2: The calculations for a one-way within-groups ANOVA are similar to those for a one-way between-groups ANOVA, but we now calculate a subjects sum of squares in addition to the between-groups, within-groups, and total sum of squares. The subjects sum of squares reduces the within-groups sum of squares by removing the variability associated with participants’ differences across groups.

The beauty of the within-groups design is that it reduces errors that are due to differences between the groups, because each group includes exactly the same participants. The beer study could not be influenced by individual taste preferences, amount of alcohol typically consumed, tendency to be critical or lenient when rating, and so on. This enables us to reduce the within-groups variability associated with differences among the people in the study across groups. The lower within-groups variability means a smaller denominator for the F statistic and a larger F statistic that makes it easier to reject the null hypothesis.

330

The Six Steps of Hypothesis Testing

EXAMPLE 13.2

We’ll use the data from the beer taste test to walk through the same six steps of hypothesis testing that you have used for every other statistical test.

STEP 1: Identify the populations, distribution, and assumptions.

The one-way within-groups ANOVA requires an additional assumption compared to the one-way between-groups ANOVA: We must be careful to avoid order effects. In the beer study, order may have influenced participants’ judgments because all participants tasted the beers in the same order: a mid-range beer, followed by a high-end beer, followed by a cheap beer, followed by another cheap beer, and so on. Perhaps the first sip of beer tastes the best, no matter what kind of beer is being tasted. Ideally, Fallows would have used counterbalancing, so that participants tasted the beers in different orders.

Summary: Population 1: People who drink cheap beer. Population 2: People who drink mid-range beer. Population 3: People who drink high-end beer.

The comparison distribution and hypothesis test: The comparison distribution is an F distribution. The hypothesis test is a one-way within-groups ANOVA.

Assumptions: (1) The participants were not selected randomly, so we must generalize with caution. (2) We do not know if the underlying population distributions are normal, but the sample data do not indicate severe skew. (3) After we calculate the test statistic, we will test the homoscedasticity assumption by checking to see whether the largest variance is more than twice the smallest. (4) The experimenter did not counterbalance, so there may be order effects.

STEP 2: State the null and research hypotheses.

This step is identical to that for a one-way between-groups ANOVA.

Summary: Null hypothesis: People who drink cheap, mid-range, and high-end beer rate their beverages the same, on average—H0: μ1 = μ2 = μ3. Research hypothesis: People who drink cheap, mid-range, and high-end beer do not rate their beverages the same, on average—H1 is that at least one μ is different from another μ.

STEP 3: Determine the characteristics of the comparison distribution.

We state that the comparison distribution is an F distribution and determine the degrees of freedom. Instead of three, we now calculate four kinds of degrees of freedom—between-groups, subjects, within-groups, and total. The subjects degrees of freedom corresponds to a sum of squares for differences across participants: the subjects sum of squares, or SSsubjects. In a one-way within-groups ANOVA, we calculate between-groups degrees of freedom and subjects degrees of freedom first because we multiply these two together to calculate the within-groups degrees of freedom.

We calculate the between-groups degrees of freedom exactly as before:

dfbetween = Ngroups − 1 = 3 − 1 = 2

331

We next calculate the degrees of freedom that pairs with SSsubjects. Called dfsubjects, it is calculated by subtracting 1 from the actual number of subjects, not from the number of data points. We use a lowercase n to indicate that this is the number of participants in a single sample (even though they’re all in every sample). The formula is:

MASTERING THE FORMULA

13-1: The formula for the subjects degrees of freedom is: dfsubjects = n − 1. We subtract 1 from the number of participants in the study. We use the lowercase n to indicate that this is the number of data points in a single sample, even though we know that every participant is in all three groups.

dfsubjects = n − 1 = 5 − 1 = 4

Once we know the between-groups degrees of freedom and the subjects degrees of freedom, we calculate the within-groups degrees of freedom by multiplying the first two:

MASTERING THE FORMULA

13-2: The formula for the within-groups degrees of freedom for a one-way within-groups ANOVA is: dfwithin = (dfbetween)(dfsubjects). We multiply the between-groups degrees of freedom by the subjects degrees of freedom. This gives a lower number than we calculated for a one-way between-groups ANOVA because we want to exclude individual differences across groups from the within-groups sum of squares, and the degrees of freedom must reflect that.

dfwithin = (dfbetween)(dfsubjects) = (2)(4) = 8

Note that the within-groups degrees of freedom is smaller than we would have calculated for a one-way between-groups ANOVA. For a one-way between-groups ANOVA, we would have subtracted 1 from each sample (5 − 1 = 4) and summed them to get 12. The within-groups degrees of freedom is smaller because we exclude variability related to differences among the participants from the within-groups sum of squares, and the degrees of freedom must reflect that.

Finally, we calculate total degrees of freedom using either method we learned earlier. We can sum the other degrees of freedom:

dftotal = dfbetween + dfsubjects + dfwithin = 2 + 4 + 8 = 14

Alternatively, we can use the second formula we learned before, treating the total number of participants as every data point, rather than every person. We know, of course, that there are just five participants and that they participate in all three levels of the independent variable, but for this step, we count the 15 total data points:

MASTERING THE FORMULA

13-3: We can calculate the total degrees of freedom for a one-way within-groups ANOVA in two ways. We can sum all of the other degrees of freedom: dftotal = dfbetween + dfsubjects + dfwithin. Or we can subtract 1 from the total number of observations in the study: dftotal = Ntotal − 1.

dftotal = Ntotal − 1 = 15 − 1 = 14

We have calculated the 4 degrees of freedom that we will include in the source table. However, we only report the between-groups and within-groups degrees of freedom at this step.

Summary: We use the F distribution with 2 and 8 degrees of freedom.

STEP 4: Determine the critical values, or cutoffs.

The fourth step is identical to that for a one-way between-groups ANOVA. We use the between-groups degrees of freedom and within-groups degrees of freedom to look up a critical value on the F table in Appendix B.

Summary: The critical value for the F statistic for a p level of 0.05 and 2 and 8 degrees of freedom is 4.46.

STEP 5: Calculate the test statistic.

As before, we calculate the test statistic in the fifth step. To start, we calculate four sums of squares—one each for between-groups, subjects, within-groups, and total. For each sum of squares, we calculate deviations between two different types of means or scores, square the deviations, and then sum the squared differences. We calculate a squared deviation for every score; so for each sum of squares in this example, we sum 15 squared deviations.

332

As we did with the one-way between-groups ANOVA, let’s start with the total sum of squares, SStotal. We calculate this exactly as we calculated it previously:

SStotal = Σ(XGM)2 = 2117.732

Type of Beer Rating (X) XGM (XGM)2
Cheap 40      −0.533      0.284     
Cheap 42      1.467      2.152     
Cheap 30      −10.533      110.944     
Cheap 37      −3.533      12.482     
Cheap 23      −17.533      307.406     
Mid-range 30      −10.533      110.944     
Mid-range 45      4.467      19.954     
Mid-range 38      −2.533      6.416     
Mid-range 32      −8.533      72.812     
Mid-range 28      −12.533      157.076     
High-end 53      12.467      155.426     
High-end 65      24.467      598.634     
High-end 64      23.467      550.700     
High-end 43      2.467      6.086     
High-end 38      −2.533      6.416     
GM = 40.533             Σ(XGM)2 = 2117.732

Next, we calculate the between-groups sum of squares. It, too, is the same as for a one-way between-groups ANOVA:

SSbetween = Σ(MGM)2 = 1092.130

Type of Beer Rating (X) Group mean (M) MGM (MGM)2
Cheap 40 34.4 −6.133 37.614
Cheap 42 34.4 −6.133 37.614
Cheap 30 34.4 −6.133 37.614
Cheap 37 34.4 −6.133 37.614
Cheap 23 34.4 −6.133 37.614
Mid-range 30 34.6 −5.933 35.200
Mid-range 45 34.6 −5.933 35.200
Mid-range 38 34.6 −5.933 35.200
Mid-range 32 34.6 −5.933 35.200
Mid-range 28 34.6 −5.933 35.200
High-end 53 52.6 12.067 145.612
High-end 65 52.6 12.067 145.612
High-end 64 52.6 12.067 145.612
High-end 43 52.6 12.067 145.612
High-end 38 52.6 12.067 145.612
GM = 40.533                Σ(MGM)2 = 1092.130

333

MASTERING THE FORMULA

13-4: The subjects sum of square in a one-way within-groups ANOVA is calculated using the following formula: SSsubjects = Σ(MparticipantGM)2. For each score, we subtract the grand mean from that participant’s mean for all of his or her scores and square this deviation. Note that we do not use the scores in any of these calculations. We sum all the squared deviations.

So far, the calculations of the sums of squares for a one-way within-groups ANOVA have been the same as they were for a one-way between-groups ANOVA. We left the subjects sum of squares and within-groups sum of squares for last. Here is where we see some changes. We want to remove the variability caused by participant differences from the estimate of variability across conditions. So we calculate the subjects sum of squares separately from the within-groups sum of squares. To do that, we subtract the grand mean from each participant’s mean for all of his or her scores. We first have to calculate a mean for each participant across the three conditions. For example, the first participant had ratings of 40 for cheap beers, 30 for mid-range beers, and 53 for high-end beers. This participant’s mean is 41.

So the formula for the subjects sum of squares is:

SSsubjects = Σ(MparticipantGM)2 = 729.738

Participant Type of Beer Rating (X) Participant Mean (Mparticipant) MparticipantGM (MparticipantGM)2
1 Cheap 40 41 0.467 0.218
2 Cheap 42 50.667 10.134 102.698
3 Cheap 30 44 3.467 12.02
4 Cheap 37 37.333 −3.2 10.24
5 Cheap 23 29.667 −10.866 118.07
1 Mid-range 30 41 0.467 0.218
2 Mid-range 45 50.667 10.134 102.698
3 Mid-range 38 44 3.467 12.02
4 Mid-range 32 37.333 −3.2 10.24
5 Mid-range 28 29.667 −10.866 118.07
1 High-end 53 41 0.467 0.218
2 High-end 65 50.667 10.134 102.698
3 High-end 64 44 3.467 12.02
4 High-end 43 37.333 −3.2 10.24
5 High-end 38 29.667 −10.866 118.07
GM = 40.533                Σ(MparticipantGM)2 = 729.738

We only have one sum of squares left to go. To calculate the within-groups sum of squares from which we’ve removed the subjects sum of squares, we take the total sum of squares and subtract the two others that we’ve calculated so far—the between-groups sum of squares and the subjects sum of squares. The formula is:

MASTERING THE FORMULA

13-5: The within-groups sum of squares for a one-way within-groups ANOVA is calculated using the following formula: SSwithin = SStotalSSbetweenSSsubjects. We subtract the between-groups sum of squares and the subjects sum of squares from the total sum of squares.

SSwithin = SStotalSSbetweenSSsubjects
= 2117.732 − 1092.130 − 729.738 = 295.864

We now have enough information to fill in the first three columns of the source table—the source, SS, and df columns. We calculate the rest of the source table as we did for a one-way between-groups ANOVA. For each of the three sources—between-groups, subjects, and within-groups—we divide the sum of squares by the degrees of freedom to get its variance, MS.

MASTERING THE FORMULA

13-6: We calculate the subjects mean square by dividing its associated sum of squares by its associated degrees of freedom: .

334

We then calculate two F statistics—one for between-groups and one for subjects. For the between-groups F statistic, we divide its MS by the within-groups MS. For the subjects F statistic, we divide its MS by the within-groups MS.

MASTERING THE FORMULA

13-7: The formula for the subjects F statistic is: . We divide the subjects mean square by the within-groups mean square.

The completed source table is shown here:

Source SS df MS F
Between-groups 1092.130       2       546.065       14.766      
Subjects 729.738       4       182.435       4.933      
Within-groups 295.864       8       36.981             
Total 2117.732       14      

Here is a recap of the formulas used to calculate a one-way within-groups ANOVA:

Source SS df MS F
Between-groups Σ(MGM)2 Ngroups − 1
Subjects Σ(MparticipantGM)2 dfsubjects = n − 1
Within-groups SStotalSSbetweenSSsubjects (dfbetween) (dfsubjects)
Total Σ(XGM)2 Ntotal − 1

We calculated two F statistics, but we’re really only interested in the between-groups F statistic, 14.766, that tells us whether there is a statistically significant difference between groups.

Summary: The F statistic associated with the between-groups difference is 14.77.

STEP 6: Make a decision.

This step is identical to that for the one-way between-groups ANOVA.

Summary: The F statistic, 14.77, is beyond the critical value, 4.46. We reject the null hypothesis. It appears that mean ratings of beers differ based on the type of beer in terms of price category, although we cannot yet know exactly which means differ. We report the statistics in a journal article as F(2,8) = 14.77, p < 0.05. (Note: If we used software, we would report the exact p value.)

335

CHECK YOUR LEARNING

Reviewing the Concepts

  • We use one-way within-groups ANOVA when we have a nominal or ordinal independent variable with at least three levels, a scale dependent variable, and participants who experience all levels of the independent variable.
  • Because all participants experience all levels of the independent variable, we reduce the within-groups variability by reducing individual differences; each person serves as a control for him- or herself. A possible concern with this design is order effects.
  • One-way within-groups ANOVA uses the same six steps of hypothesis testing that are used for one-way between-groups ANOVA—with one major exception. We calculate statistics for four sources rather than three. The fourth source, which is in addition to between-groups, within-groups, and total, is typically called “subjects.”

Clarifying the Concepts

  • 13-1 Why is the within-groups variability, or sum of squares, smaller for the within-groups ANOVA compared to the between-groups ANOVA?

Calculating the Statistics

  • 13-2 Calculate the four degrees of freedom for the following groups, assuming a within-groups design:
    Participant 1 Participant 2 Participant 3
    Group 1 7 9 8
    Group 2 5 8 9
    Group 3 6 4 6
    1. dfbetween = Ngroups − 1
    2. dfsubjects = n − 1
    3. dfwithin = (dfbetween)(dfsubjects)
    4. dftotal = dfbetween + dfsubjects + dfwithin; or dftotal = Ntotal − 1
  • 13-3 Calculate the four sums of squares for the data in Check Your Learning 13-2:
    1. SStotal = Σ(XGM)2
    2. SSbetween = Σ(MGM)2
    3. SSsubjects = Σ(MparticipantGM)2
    4. SSwithin = SStotalSSbetweenSSsubjects
  • 13-4 Using all of your calculations in Check Your Learning 13-2 and 13-3, perform the simple division to complete an ANOVA source table for these data.

Applying the Concepts

  • 13-5 Let’s create a context for the data presented in Check Your Learning 13-2. Suppose a car dealer wants to sell a car by having people test drive it and two other cars in the same class (e.g., midsize sedans). The data from these three groups might represent the ratings, ranging from 1 (low quality) to 10 (high quality), that drivers gave the driving experience after test-driving the cars. Using the F values you calculated above, complete the following:
    1. Write hypotheses, in words, for this study.
    2. How might you conduct this research such that you would satisfy the fourth assumption of the within-groups ANOVA?
    3. Determine the critical value for F and make a decision about the outcome of this research.

Solutions to these Check Your Learning questions can be found in Appendix D.

336