Chapter 1. Statistical Power

Statistical Applets

To set up the test, enter your information: What null hypothesis H0 about the mean μ do you want to test? Is your alternative hypothesis one-sided high, one-sided low, or two-sided? What level of significance α do you require? How many observations do you have (50 or fewer)? What is the known value of the standard deviation σ? Finally, against what specific alternative value of the mean μ do you want to find the power? When you have entered your information, click UPDATE.

This applet illustrates the power of statistical tests. Finding the power of a test assumes that you have set a fixed significance level α for the test. Review the "Statistical Significance" applet to recall how tests with significance level α work.

The top curve shows the sampling distribution of the sample mean when your null hypothesis is true. The yellow area under this curve is α, the probability of rejecting H0 when it is really true. The bottom curve shows the sampling distribution of when your chosen alternative is true. The red area under this curve is the power, the probability of rejecting H0 when the alternative is really true.

1.

Suppose you're planning to collect a set of data in an experiment where you expect the original population mean to be 10, with a standard deviation of 3. Your initial plan is to collect 20 observations, and you expect your experimental manipulation to increase the mean of your observations by 1. You plan to test with an alpha = .05 level of significance, using a two-tailed test (that is, testing whether μ ≠ 10). Enter the following values that you should enter in the applet to calculate the power for this situation:

  • H0: μ =
  • alt. μ =
3
Try again.
Incorrect. See above for the correct answers.
Great job.

2.

What does the applet indicate is the power of this test—the probability that you will be able to reject the null hypothesis if your assumptions are correct?
2
Correct.
Try again.
Incorrect.

3.

Suppose you feel this is not enough power. Specifically, you'd like to be about 90% certain of rejecting the null hypothesis, assuming your assumptions are correct. What would likely be the most practical way for you to increase your power?

A.
B.
C.
D.
E.
F.
2
Correct.
Try again.
Incorrect.

4.

Use the applet to determine how large a sample size you would need to generate power of approximately 90% (.90)
3
Correct.
Keep adjusting the value for n over and over again (clicking UPDATE if necessary after each change) until you find that the Power estimate is just over .90. The value you have for n at this point is the answer to the question.
Incorrect.

5.

Instead of increasing your sample size, might you alternatively increase the alpha level for your test to increase power? Discuss why this might or might not be an acceptable strategy.
Increasing alpha is, technically, another way to increase power. Indeed, this is the most direct way to increase the probability that an observed sample mean will fall in the "rejection region" for your hypothesis test, since increasing alpha directly expands this rejection region. However, many areas of study have accepted standards for the alpha level to use when evaluating null hypotheses (e.g. the .05 level is ubiquitous in behavioral science research), so practically speaking this is usually not an option.