How do retail companies that fail differ from those that succeed? An accounting professor compares two samples of retail companies: one sample of failed retail companies and one of retail companies that are still active. Which of two incentive packages will lead to higher use of a bank’s credit cards? The bank designs an experiment where credit card customers are assigned at random to receive one or the other incentive package. Two-sample problems such as these are among the most common situations encountered in statistical practice.

379

You must carefully distinguish two-sample problems from the matched pairs designs studied earlier. In two-sample problems, there is no matching of the units in the two samples, and the two samples may be of different sizes. As a result, inference procedures for two-sample data differ from those for matched pairs.

We can present two-sample data graphically with a back-to-back stemplot for small samples (page 17) or with side-by-side boxplots for larger samples (page 29). Now we will apply the ideas of formal inference in this setting. When both population distributions are symmetric, and especially when they are at least approximately Normal, a comparison of the mean responses in the two populations is most often the goal of inference.

We have two independent samples, from two distinct populations (such as failed companies and active companies). We measure the same quantitative response variable (such as the cash flow margin) in both samples. We will call the variable in the first population and in the second because the variable may have different distributions in the two populations. Here is the notation that we will use to describe the two populations:

We want to compare the two population means, either by giving a confidence interval for or by testing the hypothesis of no difference, . We base inference on two independent SRSs, one from each population. Here is the notation that describes the samples:

Throughout this section, the subscripts 1 and 2 show the population to which a parameter or a sample statistic refers.

The two-sample statistic

The natural estimator of the difference is the difference between the sample means, . If we are to base inference on this statistic, we must know its sampling distribution. Here are some facts:

380

Because any Normal random variable has the distribution when standardized, we have arrived at a new statistic. The two-sample statistic

has the standard Normal sampling distribution and would be used in inference when the two population standard deviations and are known.

In practice, however, and are not known. We estimate them by the sample standard deviations and from our two samples. Following the pattern of the one-sample case, we substitute the standard errors for the standard deviations in the two-sample statistic. The result is the two-sample statistic:

Unfortunately, this statistic does not have a distribution. A distribution replaces an distribution only when a single standard deviation () is replaced by an estimate (). In this case, we replaced two standard deviations ( and ) by their estimates ( and ).

Nonetheless, we can approximate the distribution of the two-sample statistic by using the distribution with an approximation for the degrees of freedom . We use these approximations to find approximate values of for confidence intervalsand to find approximate -values for significance tests. There are two procedures used in practice:

The choice of approximation rarely makes a difference in our conclusion. Most statistical software uses the first option to approximate the distribution unless the user requests another method. Use of this approximation without software is a bit complicated.¹⁸

If you are not using software, we recommend the second approximation. This approximation is appealing because it is conservative.¹⁹ That is, margins of error for confidence intervals are a bit wider than they need to be, so the true confidence level is larger than . For significance testing, the true -values are a bit smaller than those we obtain from the approximation; thus, for tests at a fixed significance level, we are a little less likely to reject when it is true.

381

The two-sample confidence interval

We now apply the basic ideas about procedures to the problem of comparing two means when the standard deviations are unknown. We start with confidence intervals.

383

The two-sample significance test

The same ideas that we used for the two-sample confidence intervals also apply to two-sample significance tests. We can use either software or the conservative approach with Table D to approximate the -value.

Robustness of the two-sample procedures

The two-sample procedures are more robust than the one-sample methods. When the sizes of the two samples are equal and the distributions of the two populations being compared have similar shapes, probability values from the table are quite accurate for a broad range of distributions when the sample sizes are as small as .²¹ When the two population distributions have different shapes, larger samples are needed. The guidelines given on page 372 for the use of one-sample procedures can be adapted to two-sample procedures by replacing “sample size” with the “sum of the sample sizes” . Specifically,

These guidelines are rather conservative, especially when the two samples are of equal size. In planning a two-sample study, you should usually choose equal sample sizes. The two-sample procedures are most robust against non-Normality in this case, and the conservative probability values are most accurate.

Here is an example with large sample sizes that are almost equal. Even if the distributions are not Normal, we are confident that the sample means will be approximately Normal. The two-sample procedures are very robust in this case.

In this example, the exact -value is very small because says that the observed mean is more than five standard deviations above the hypothesized mean. The difference in mean prices is not only highly significant but large enough (27 cents per bushel) to be important to producers.

In this and other examples, we can choose which population to label 1 and which to label 2. After inspecting the data, we chose July as Population 1 because this choice makes the statistic a positive number. This avoids any possible confusion from reporting a negative value for . Choosing the population labels is not the same as choosing a one-sided alternative after looking at the data. Choosing hypotheses after seeing a result in the data is a violation of sound statistical practice.

Inference for small samples

Small samples require special care. We do not have enough observations to examine the distribution shapes, and only extreme outliers stand out. The power of significance tests tends to be low, and the margins of error of confidence intervals tend to be large. Despite these difficulties, we can often draw important conclusions from studies with small sample sizes. If the size of an effect is as large as it was in the preceding wheat price example, it should still be evident even if the n’s are small.

Figure 7.10 gives outputs for this analysis from several software systems. Although the formats and labels differ, the basic information is the same. All report the sample sizes, the sample means and standard deviations (or variances), the statistic, and its -value. All agree that the -value is very small, though some give more detail than others. Excel and JMP outputs, for example, provide both one-sided and two-sided -values. Some software (SAS, SPSS, and Minitab) labels the groups in alphabetical order. In this example, January is then the first population and , the negative of our result. Always check the means first and report the statistic (you may need to change the sign) in an appropriate way. Be sure to also mention the size of the effect you observed, such as “The sample mean price for July was 31.7 cents higher than in January.”

SAS and SPSS report the results of two procedures: a special procedure that assumes that the two population variances are equal and the general two-sample procedure that we have just studied. This “equal-variances” procedure is most helpful when the sample sizes and are small and it is reasonable to assume equal variances.

The pooled two-sample procedures

There is one situation in which a statistic for comparing two means is not approximately distributed but has exactly a distribution. Suppose that the two Normal population distributions have the same standard deviation. In this case, we need substitute only a single standard error in a statistic, and the resulting statistic has a distribution. We will develop the statistic first, as usual, and from it the statistic.

Call the common—but still unknown—standard deviation of both populations . Both sample variances and estimate . The best way to combine these two estimates is to average them with weights equal to their degrees of freedom. This gives more weight to the information from the larger sample. The resulting estimator of is

387

This is called the pooled estimator of because it combines the information in both samples.

When both populations have variance , the addition rule for variances says that has variance equal to the sum of the individual variances, which is

The standardized difference of means in this equal-variance case is, therefore,

388

This is a special two-sample statistic for the case in which the populations have the same . Replacing the unknown by the estimate gives a statistic. The degrees of freedom are , the sum of the degrees of freedom of the two sample variances. This statistic is the basis of the pooled two-sample inference procedures.

389

390

As usual, we first examine the data. Histograms for the two groups of firms are given in Figure 7.11. Normal curves with mean and standard deviation equal to the sample values are superimposed on the histograms. The distribution for the active firms looks more Normal than the distribution for the failed firms. However, there are no outliers or strong departures from Normality that will prevent us from using the procedures for these data. Let’s compare the mean cash flow margin for the two groups of firms using a significance test.

Of course, a -value is rarely a complete summary of a statistical analysis. To make a judgment regarding the size of the difference between the two groups of firms, we need a confidence interval.

The pooled two-sample procedures are anchored in statistical theory and have long been the standard version of the two-sample in textbooks. But they require the condition that the two unknown population standard deviations are equal. This condition is hard to verify. We discuss methods to assess this condition in Chapter 14.

The pooled procedures are, therefore, a bit risky. They are reasonably robust against both non-Normality and unequal standard deviations when the sample sizes are nearly the same. When the samples are quite different in size, the pooled procedures become sensitive to unequal standard deviations and should be used with caution unless the samples are large. Unequal standard deviations are quite common. In particular, it is common for the spread of data to increase when the center moves up. We recommend regular use of the unpooled procedures, particularly when software automates the Satterthwaite approximation.

392