• A test of significance is intended to assess the evidence provided by data against a null hypothesis H0 in favor of an alternative hypothesis Ha.
• The hypotheses are stated in terms of population parameters. Usually, H0 is a statement that no effect or no difference is present, and Ha says that there is an effect or difference in a specific direction (one-sided alternative) or in either direction (two-sided alternative).
• The test is based on a test statistic. The P-value is the probability, computed assuming that H0 is true, that the test statistic will take a value at least as extreme as that actually observed. Small P-values indicate strong evidence against H0. Calculating P-values requires knowledge of the sampling distribution of the test statistic when H0 is true.
• If the P-value is as small or smaller than a specified value α, the data are statistically significant at significance level α.
• Significance tests for the hypothesis H0: μ = μ0 concerning the unknown mean μ of a population are based on the z statistic:
The z test assumes an SRS of size n, known population standard deviation σ, and either a Normal population or a large sample. P-values are computed from the Normal distribution (Table A). Fixed α tests use the table of standard Normal critical values (Table D).