22.27 Finding a P-value by simulation. Is a new method of teaching reading to first-graders (Method B) more effective than the method now in use (Method A)? You design a matched pairs experiment to answer this question. You form 20 pairs of first-graders, with the two children in each pair carefully matched by IQ, socioeconomic status, and reading-readiness score. You assign at random one student from each pair to Method A. The other student in the pair is taught by Method B. At the end of first grade, all the children take a test to determine their reading skill. Assume that the higher the score on this test, the more proficient the student is at reading. Let p stand for the proportion of all possible matched pairs of children for which the child taught by Method B has the higher score. Your hypotheses are

H0: p = 0.5  (no difference in effectiveness)
Ha: p > 0.5  (Method B is more effective)

The result of your experiment is that Method B gave the higher score in 12 of the 20 pairs, or .

  1. (a) If H0 is true, the 20 pairs of students are 20 independent trials with probability 0.5 that Method B “wins’’ each trial (is the more effective method). Explain how to use Table A to simulate these 20 trials if we assume for the sake of argument that H0 is true.

  2. (b) Use Table A, starting at line 105, to simulate 10 repetitions of the experiment. Estimate from your simulation the probability that Method B will do better (be the more effective method) in 12 or more of the 20 pairs when H0 is true. (Of course, 10 repetitions are not enough to estimate the probability reliably. Once you understand the idea, more repetitions are easy.)

  3. (c) Explain why the probability you simulated in part (b) is the P-value for your experiment. With enough patience, you could find all the P-values in this chapter by doing simulations similar to this one.