RESOLVING THE CONTROVERSY

Chapter 23: Should Hypothesis Tests Be Banned?

It will probably not surprise you that the American Statistical Association (ASA) did not take kindly to the BASP ban on hypothesis testing and confidence intervals. As of the writing of this text, a formal response is being crafted by the ASA.

It is interesting to note that in 1999, the American Psychological Association (APA) appointed a Task Force on Statistical Inference. At that time, the task force did not want to ban hypothesis tests. The report that was produced by the task force was, in fact, a summary of good statistical practice:

625

• Define your population clearly.
• Describe your data production and prefer randomized methods whenever possible.
• Describe your variables and how they were measured.
• Give your sample size and explain how you decided on the sample size.
• If there were dropouts or other practical problems, mention them.
• “As soon as you have collected your data, before you compute any statistics, look at your data.’’
• Ask whether the results of computations make sense to you.
• Recognize that “inferring causality from nonrandomized designs is a risky enterprise.’’

The APA task force did say, “It is hard to imagine a situation in which a dichotomous accept-reject decision is better than reporting an actual p value or, better still, a confidence interval. . . . Always provide some effect-size estimate when reporting a p value.’’ But would the task force ban hypothesis tests altogether? “Although this might eliminate some abuses, the committee thought there were enough counterexamples to justify forbearance.’’

Sixteen years later, BASP banned hypothesis tests and confidence intervals. The controversy is not over. We encourage you to search the Web for the most up-to-date information about this issue.