The chi-square test for goodness of fit is a nonparametric hypothesis test used when there is one nominal variable.

The chi-square test for independence is a nonparametric hypothesis test used when there are two nominal variables.

MASTERING THE CONCEPT

17.2: When we only have nominal variables, we use the chi-square statistic. Specifically, we use a chi-square test for goodness of fit when we have one nominal variable, and we use a chi-square test for independence when we have two nominal variables.

EXAMPLE 17.1

For example, researchers reported that the best youth soccer players in the world were more likely to have been born early in the year than later (Dubner & Levitt, 2006a). As one example, they reported that 52 elite youth players in Germany were born in January, February, or March, whereas only 4 players were born in October, November, or December. (Those born in other months were not included in this study.)

The null hypothesis predicts that when a person was born will not make any difference; the research hypothesis predicts that the month a person was born will matter when it comes to being an elite soccer player. Assuming that births in the general population are evenly distributed across months of the year, the null hypothesis posits equal numbers of elite soccer players born in the first 3 months and the last 3 months of the year. With 56 participants in the study (52 born in the first 3 months and 4 in the last 3 months), equal frequencies lead us to expect 28 players born in the first 3 months and 28 in the last 3 months just by chance. The birth months don’t appear to be evenly distributed, but is this a real pattern, or just chance?

Like previous hypothesis tests, the chi-square goodness of fit test uses the six steps of hypothesis testing.

STEP 1: Identify the populations, distribution, and assumptions.

There are always two populations involved in a chi-square test: one population that matches the frequencies of participants like those we observed and another population that matches the frequencies of participants like those we would expect according to the null hypothesis. In this case, there is a population of elite German youth soccer players with birth dates like those we observed and a population of elite German youth soccer players with birth dates like those in the general population. The comparison distribution is a chi-square distribution. There’s just one nominal variable, birth months, so we’ll conduct a chi-square test for goodness of fit.

The first assumption is that the variable (birth month) is nominal. The second assumption is that each observation is independent; no single participant can be in more than one category. The third assumption is that participants were randomly selected. If not, it may be unwise to confidently generalize beyond the sample. A fourth assumption is that there is a minimum number of expected participants in every category (also called a cell)—at least 5 and preferably more. An alternative guideline (Delucchi, 1983) is for at least five times as many participants as cells. In any case, the chi-square tests seem robust to violations of this last assumption.

Summary: Population 1: Elite German youth soccer players with birth dates like those we observed. Population 2: Elite German youth soccer players with birth dates like those in the general population.

The comparison distribution is a chi-square distribution. The hypothesis test will be a chi-square test for goodness of fit because we have one nominal variable only, birth months. This study meets three of the four assumptions: (1) The one variable is nominal. (2) Every participant is in only one cell (you can’t be born in January and November). (3) This is not a randomly selected sample of all elite soccer players. The sample includes only German youth soccer players in the elite leagues. We must be cautious in generalizing beyond young German elite players. (4) There are more than five times as many participants as cells (the table has two cells, and 2 × 5 = 10). We have 56 participants, far more than the 10 necessary to meet this guideline.

STEP 2: State the null and research hypotheses.

For chi-square tests, it’s easiest to state the hypotheses in words only, rather than in both words and symbols.

Summary: Null hypothesis: Elite German youth soccer players have the same pattern of birth months as those in the general population. Research hypothesis: Elite German youth soccer players have a different pattern of birth months than those in the general population.

STEP 3: Determine the characteristics of the comparison distribution.

Our only task at this step is to determine the degrees of freedom. In most previous hypothesis tests, the degrees of freedom have been based on sample size. For the chi-square hypothesis tests, however, the degrees of freedom are based on the numbers of categories, or cells, in which participants can be counted. The degrees of freedom for a chi-square test for goodness of fit is the number of categories minus 1:

Here, k is the symbol for the number of categories. The current example has only two categories: Each soccer player in this study was born in either the first 3 months of the year or the last 3 months of the year:

Summary: The comparison distribution is a chi-square distribution, which has 1 degree of freedom: df_χ² = 2 − 1 = 1

STEP 4: Determine the critical value, or cutoff.

To determine the cutoff, or critical value, for the chi-square statistic, we use the chi-square table in Appendix B. χ² is based on squares and can never be negative, so there is just one critical value. An excerpt from Appendix B that applies to the soccer study is given in Table 17-2. We look under the p level, usually 0.05, and across from the appropriate degrees of freedom, in this case, 1. For this situation, the critical chi-square statistic is 3.841.

Summary: The critical χ², based on a p level of 0.05 and 1 degree of freedom, is 3.841, as seen in the curve in Figure 17-1.

STEP 5: Calculate the test statistic.

To calculate a chi-square statistic, we determine the observed frequencies and the expected frequencies, as seen in Table 17-3 and in the second and third columns of Table 17-4. The expected frequencies are determined from the information we have about the general population. In this case, we estimate that, in the general population, about half of all births (only, of course, among those born in the first or last 3 months of the year) occur in the first 3 months of the year, a proportion of 0.50.

Of the 56 elite German youth soccer players in the study, we would expect to find that 28 were born in the first 3 months of the year (versus the last 3 months of the year) if these youth soccer players are no different from the general population with respect to birth date. Similarly, we would expect a proportion of 1 − 0.50 = 0.50 of these soccer players to be born in the last 3 months of the year:

These numbers are identical only because the proportions are 0.50 and 0.50. If the proportion expected for the first 3 months of the year, based on the general population, were 0.60, then we would expect a proportion of 1 − 0.60 = 0.40 for the last 3 months of the year.

The next step in calculating the chi-square statistic is to calculate a sort of sum of squared differences. We start by determining the difference between each observed frequency and its matching expected frequency. This is usually done in columns, so we use this format even though we have only two categories. The first three columns of Table 17-4 show us the categories, observed frequencies, and expected frequencies, respectively. The fourth column, using O for observed and E for expected, displays the differences. As in the other situations, if we sum the differences, we get 0; they cancel out because some are positive and some are negative. We solve this problem as we have others—by squaring the differences, as shown in the fifth column. Next, however, we have a step that we haven’t seen before with squared differences. We divide each squared difference by the expected value for its cell, as seen in the sixth column. The numbers in the sixth column are the ones we sum.

As an example, here are the calculations for the category “first 3 months”:

Once we complete the table, the last step is easy. We just add up the numbers in the sixth column. In this case, the chi-square statistic is 20.571 + 20.571 = 41.14. We can finish the formula by adding a summation sign to the formula in the sixth column. Note that we don’t have to divide this sum by anything, as we’ve done with other statistics. We already did the dividing before we summed. This sum is the chi-square statistic. Here is the formula:

Summary:

STEP 6: Make a decision.

This last step is identical to that of previous hypothesis tests. We reject the null hypothesis if the test statistic is beyond the critical value, and we fail to reject the null hypothesis if the test statistic is not beyond the critical value. In this case, the test statistic, 41.14, is far beyond the cutoff, 3.841, as seen in Figure 17-2. We reject the null hypothesis. Because there are only two categories, it’s clear where the difference lies. It appears that elite German youth soccer players are more likely to have been born in the first 3 months of the year, and less likely to have been born in the last 3 months of the year, than members of the general population. (If we had failed to reject the null hypothesis, we could only have concluded that these data did not provide sufficient evidence to show that elite German youth soccer players have a different likelihood of being born in the first, versus last, 3 months of the year than those in the general population.)

Summary: Reject the null hypothesis; it appears that elite German youth soccer players are more likely to have been born in the first 3 months of the year, and less likely to have been born in the last 3 months of the year, than people in the general population.

We report these statistics in a journal article in almost the same format that we’ve seen previously. We report the degrees of freedom, the value of the test statistic, and whether the p value associated with the test statistic is less than or greater than the cutoff based on the p level of 0.05. (As usual, we would report the actual p level if we conducted this hypothesis test using software.) In addition, we report the sample size in parentheses with the degrees of freedom. In the current example, the statistics read:

The researchers who conducted this study imagined four possible explanations: “a) certain astrological signs confer superior soccer skills; b) winter-born babies tend to have higher oxygen capacity, which increases soccer stamina; c) soccer-mad parents are more likely to conceive children in springtime, at the annual peak of soccer mania; d) none of the above” (Dubner & Levitt, 2006a). What’s your guess?

Dubner and Levitt (2006a) picked (d) and suggested another alternative. Participation in youth soccer leagues has a strict cutoff date: December 31. Compared to those born in December, children born the previous January are likely to be more physically and emotionally mature, perceived as more talented, chosen for the best leagues, and given better coaching—a self-fulfilling prophecy. All this from a simple chi-square test for goodness of fit!

Image Source/Getty Images

EXAMPLE 17.2

In the clown study, as reported in the mass media (Ryan, 2006), 186 women were randomly assigned to receive IVF treatment only or to receive IVF treatment followed by 15 minutes of clown entertainment. Eighteen of the 93 who received only the IVF treatment became pregnant, whereas 33 of the 93 who received both IVF treatment and clown entertainment became pregnant. The cells for these observed frequencies can be seen in Table 17-5. The table of cells for a chi-square test for independence is called a contingency table because it helps us see if the outcome of one variable (e.g., becoming pregnant versus not becoming pregnant) is contingent on the other variable (clown versus no clown). Let’s implement the six steps of hypothesis testing for a chi-square test for independence.

STEP 1: Identify the populations, distribution, and assumptions.

Population 1: Women receiving IVF treatment like the women we observed. Population 2: Women receiving IVF treatment for whom the presence of a clown is not associated with eventual pregnancy.

The comparison distribution is a chi-square distribution. The hypothesis test will be a chi-square test for independence because we have two nominal variables. This study meets three of the four assumptions: (1) The two variables are nominal. (2) Every participant is in only one cell. (3) The participants were not, however, randomly selected from the population of all women undergoing IVF treatment. We must be cautious in generalizing beyond the sample of Israeli women at this particular hospital. (4) There are more than five times as many participants as cells (186 participants and 4 cells; 4 × 5 = 20). We have far more participants, 186, than the 20 necessary to meet this guideline.

STEP 2: State the null and research hypotheses.

Null hypothesis: Pregnancy rates are independent of whether one is entertained by a clown after IVF treatment. Research hypothesis: Pregnancy rates depend on whether one is entertained by a clown after IVF treatment.

STEP 3: Determine the characteristics of the comparison distribution.

For a chi-square test for independence, we calculate degrees of freedom for each variable and then multiply the two to get the overall degrees of freedom. The degrees of freedom for the variable in the rows of the contingency table is:

The degrees of freedom for the variable in the columns of the contingency table is:

The overall degrees of freedom is:

To expand this last formula, we write:

The comparison distribution is a chi-square distribution, which has 1 degree of freedom:

STEP 4: Determine the critical values, or cutoffs.

The critical value, or cutoff, for the chi-square statistic, based on a p level of 0.05 and 1 degree of freedom, is 3.841 (Figure 17-3).

STEP 5: Calculate the test statistic.

The next step, determining the appropriate expected frequencies, is the most important in the calculation of the chi-square test for independence. Errors are often made in this step, and if the wrong expected frequencies are used, the chi-square statistic derived from them will also be wrong. Many students want to divide the total number of participants (here, 186) by the number of cells (here, 4) and place equivalent frequencies in all cells for the expected data. Here, that would mean that the expected frequencies would be 46.5.

But this would not make sense. Of the 186 women, only 51 became pregnant; 51/186 = 0.274, or 27.4%, of these women became pregnant. If pregnancy rates do not depend on clown entertainment, then we would expect the same percentage of successful pregnancies, 27.4%, regardless of exposure to clowns. If we have expected frequencies of 46.5 in all four cells, then we have a 50%, not a 27.4%, pregnancy rate. We must always consider the specifics of the situation.

In the current study, we already calculated that 27.4% of all women in the study became pregnant. If pregnancy rates are independent of whether a woman is entertained by a clown, then we would expect 27.4% of the women who were entertained by a clown to become pregnant and 27.4% of women who were not entertained by a clown to become pregnant. Based on this percentage, 100 − 27.4 = 72.6% of women in the study did not become pregnant. We would therefore expect 72.6% of women who were entertained by a clown to fail to become pregnant and 72.6% of women who were not entertained by a clown to fail to become pregnant. Again, we expect the same pregnancy and nonpregnancy rates in both groups—those who were and were not entertained by clowns.

Table 17-6 shows the observed data, and it also shows totals for each row, each column, and the whole table.

From Table 17-6, we see that 93 women were entertained by a clown after IVF treatment. As we calculated above, we would expect 27.4% of them to become pregnant:

Of the 93 women who were not entertained by a clown, we would expect 27.4% of them to become pregnant if clown entertainment is independent of pregnancy rates:

We now repeat the same procedure for not becoming pregnant. We would expect 72.6% of women in both groups to fail to become pregnant. For the women who were entertained by a clown, we would expect 72.6% of them to fail to become pregnant:

For the women who were not entertained by a clown, we would expect 72.6% of them to fail to become pregnant:

(Note that the two expected frequencies for the first row are the same as the two expected frequencies for the second row, but only because the same number of people were in each clown condition, 93. If these two numbers were different, we would not see the same expected frequencies in the two rows.)

The method of calculating the expected frequencies that we described above is ideal because it is directly based on our own thinking about the frequencies in the rows and in the columns. Sometimes, however, our thinking can get muddled, particularly when the two (or more) row totals do not match and the two (or more) column totals do not match. For these situations, a simple set of rules leads to accurate expected frequencies. For each cell, we divide its column total (Total_column) by the grand total (N) and multiply that by the row total (Total_row):

As an example, the observed frequency of those who became pregnant and were entertained by a clown is 33. The row total for this cell is 93. The column total is 51. The grand total, N, is 186. The expected frequency, therefore, is:

Notice that this result is identical to what we calculated without a formula. The middle step above shows that, even with the formula, we actually did calculate the pregnancy rate overall, by dividing the column total (51) by the grand total (186). We then calculated how many in that row of 93 participants we would expect to become pregnant using this overall rate:

The formula follows our logic, but it also keeps us on track when there are multiple calculations.

As a final check on the calculations, shown in Table 17-7, we can add up the frequencies to be sure that they still match the row, column, and grand totals. For example, if we add the two numbers in the first column, 25.482 and 25.482, we get 50.964 (different from 51 only because of rounding decisions). If we had made the mistake of dividing the 186 participants into cells by dividing by 4, we would have had 46.5 in each cell; then the total for the first column would have been 46.5 + 46.5 = 93, which is not a match with 51. This final check ensures that we have the appropriate expected frequencies in the cells.

The remainder of the fifth step is identical to that for a chi-square test for goodness of fit, as seen in Table 17-8. As before, we calculate the difference between each observed frequency and its matching expected frequency, square these differences, and divide each squared difference by the appropriate expected frequency. We add up the numbers in the final column of the table to calculate the chi-square statistic:

STEP 6: Make a decision.

Reject the null hypothesis; it appears that pregnancy rates depend on whether a woman receives clown entertainment following IVF treatment (Figure 17-4). The statistics, as reported in a journal article, would follow the format we learned for a chi-square test for goodness of fit as well as for other hypothesis tests in earlier chapters. We report the degrees of freedom and sample size, the value of the test statistic, and whether the p value associated with the test statistic is less than or greater than the critical value based on the p level of 0.05. (We would report the actual p level if we conducted this hypothesis test using software.) In the current example, the statistics would read:

CHECK YOUR LEARNING

Clarifying the Concepts

• 17-5 When do we use chi-square tests?
• 17-6 What are observed frequencies and expected frequencies?

Calculating the Statistics

• 17-7 Imagine a town that boasts clear blue skies 80% of the time. You get to work in that town one summer for 78 days and record the following data. (Note: For each day, you picked just one label.)
  Clear blue skies: 59 days
  Cloudy/hazy/gray skies: 19 days
  1. Calculate degrees of freedom for this chi-square test for goodness of fit.
  2. Determine the observed and expected frequencies.
  3. Calculate the differences and squared differences between frequencies, and calculate the chi-square statistic. Use the six-column format provided here.

  Category	Observed (O)	Expected (E)	O−E	(O−E)2
  Clear blue skies
  Unclear skies

Applying the Concepts

• 17-8 The Chicago Police Department conducted a study comparing two types of lineups for suspect identification: simultaneous lineups and sequential lineups (Mecklenburg, Malpass, & Ebbesen, 2006). In simultaneous lineups, witnesses saw the suspects all at once, either live or in photographs, and then made their selection. In sequential lineups, witnesses saw the people in the lineup one at a time, either live or in photographs, and said yes or no to suspects one at a time. After numerous high-profile cases in which DNA evidence exonerated people who had been convicted, including many on death row, many police departments shifted to sequential lineups in the hope of reducing incorrect identifications. Several previous studies had indicated the superiority of sequential lineups with respect to accuracy. Over one year, three jurisdictions in Illinois compared the two types of lineups. Of 319 simultaneous lineups, 191 led to identification of the suspect, 8 led to identification of another person in the lineup, and 120 led to no identification. Of 229 sequential lineups, 102 led to identification of the suspect, 20 led to identification of another person in the lineup, and 107 led to no identification.
  1. Who or what are the participants in this study? Identify the independent variable and its levels as well as the dependent variable and its levels.
  2. Conduct all six steps of hypothesis testing.
  3. Report the statistics as you would in a journal article.
  4. Why is this study an example of the importance of using two-tailed rather than one-tailed hypothesis tests?