Chapter 1. The Reasoning of a Statistical Test

Statistical Applets

Ask a State University player to shoot free throws by clicking SHOOT. You can get more data by clicking Shoot repeatedly. Do the data appear to agree with the 80% claim or to give evidence against it? When you are satisfied, click "Show true probability" to see the truth for this player. Click NEW SHOOTER to test a different player, who may have a different free throw percent.

A statistical test assesses the evidence provided by data against some claim (the null hypothesis H0). This applet allows you to gather data until you are ready to reach a conclusion about the truth of a null hypothesis. It illustrates the reasoning of tests: are the data compatible with a claim, or do they give evidence against it?

The basketball team at State University claims that each of their players makes 80% of his free throws. That's a null hypothesis, and is represented by the blue horizontal line on the graph. Have players try shooting free throws and see whether or not you think this null hypothesis can be rejected.

1.

Click NEW SHOOTER, then have the new shooter take 5 shots. What percentage of the shots so far has the shooter hit?
3
Correct.
To answer this question correctly, you must have a shooter take exactly 5 shots, then enter the percentage of shots the shooter has made ("hits"). You can click NEW SHOOTER and start over if necessary.
Incorrect.

2.

Now have the same shooter take 10 more shots, for a total of 15. Now what percentage of the shots has the shooter hit?
3
Correct.
To answer this question correctly, you must have a shooter take exactly 15 shots, then enter the percentage of shots the shooter has made ("hits"). You can click NEW SHOOTER and start over if necessary.
Incorrect.

3.

Now have the same shooter take at least 35 more shots, for a total of 50 or more. Now what percentage of the shots has the shooter hit?
3
Correct.
To answer this question correctly, you must have a shooter take at least 50 shots, then enter the percentage of shots the shooter has made ("hits"). You can click NEW SHOOTER and start over if necessary.
Incorrect.

4.

Do you feel that the evidence you've seen here allows you to reject the null hypothesis that the shooter makes 80% of all his free throws? Why or why not? Do you think it is better to make this decision after seeing 5, 15, or 50 shots? (Click the "Show true probability" checkbox to reveal the true probability of hitting a free throw for this shooter and therefore see if the null hypothesis is, in fact, true.)
The more shots we see the shooter take, the larger the sample size we have to estimate the shooter's true underlying probability of making a shot. So it is certainly better to make a decision after seeing 50 shots than after seeing just 5 shots. However, one thing this applet demonstrates is that even after many, many observations, it's still impossible to know for sure whether or not a null hypothesis should be rejected. For instance, say that after 50 shots the shooter's hit percentage is exactly .80. That would seem to count as perfect evidence for the null hypothesis that his true underlying ability is that he hits exactly 4 out of every 5 shots. But note that if the shooter takes one more shot, he will either hit it or miss it, pushing his hit rate to either .804 or .784, neither of which is exactly .80. Practically speaking, .804 and .784 are both close enough to .80 that there is really no appreciable difference between them; nevertheless, it is good to remember that null hypotheses can never be completely ruled out or accepted.