1. Perform multiple comparisons tests using the Bonferroni method.
2. Use Tukey's test to perform multiple comparisons.
3. Use confidence intervals to perform multiple comparisons for Tukey's test.

Recall that a Type I error is rejecting the null hypothesis when it is true.

• When performing multiple comparisons, the experimentwise error rate ${\alpha }_{\text{EW}}$ is the probability of making at least one Type I error in the set of hypothesis tests.
• ${\alpha }_{\text{EW}}$ is always greater than the comparison level of significance $\alpha$ by a factor approximately equal to the number of comparisons being made.
• Thus, the Bonferroni adjustment corrects for the experimentwise error rate by multiplying the $p$-value of each pairwise hypothesis test by the number of comparisons being made. If the Bonferroni-adjusted $p$-value is greater than 1, then set the adjusted value equal to 1.

• the requirements for ANOVA have been met, and
• the null hypothesis that the population means are all equal has been rejected.

• Step 1 For each of the c hypothesis tests, state the hypotheses and the rejection rule. There are $k=3$ means, so there will be $c=3$ hypothesis tests. Our hypotheses are

  • Test 1: $\begin{array}{lll}{H}_0:{\mu }_{\text{Group}0}={\mu }_{\text{Group}1}\hfill & \text{versus}\hfill & H_a:{\mu }_{\text{Group}0}\ne \hfill \end{array}{\mu }_{\text{Group}1}$
  • Test 2: $\begin{array}{lll}{H}_0:{\mu }_{\text{Group}0}={\mu }_{\text{Group}2}\hfill & \text{versus}\hfill & H_a:{\mu }_{\text{Group}0}\ne \hfill \end{array}{\mu }_{\text{Group}2}$
  • Test 3: $\begin{array}{lll}{H}_0:{\mu }_{\text{Group}1}={\mu }_{\text{Group}2}\hfill & \text{versus}\hfill & H_a:{\mu }_{\text{Group}1}\ne \hfill \end{array}{\mu }_{\text{Group}2}$

  where ${\mu }_i$ represents the population mean time spent in the open-ended sections of the maze, for the $i$th group. For each hypothesis test, reject $H_0$ if the Bonferroni-adjusted $\mathrm{p}-value\le \alpha =0.01$.

• Step 2 Calculate $t_{\text{data}}$ for each hypothesis test. From Figure 11 on page 676, we have the mean square error from the original ANOVA as MSE = 52.9485079 and from Figure 21 we get the sample means and the sample sizes. Thus,

  • Test 1:
    

    $t_{\text{data}}=\frac{{\overline{x}}_{\text{Group}0}-{\overline{x}}_{\text{Group}1}}{\sqrt{\text{MSE}\cdot \left(\frac{1}{n_{\text{Group}0}}+\frac{1}{n_{\text{Group}1}}\right)}}=\frac{19.387-8.660}{\sqrt{\left(52.9485079\right)\left(\frac{1}{15}+\frac{1}{15}\right)}}\approx 4.037$

  • Test 2:
    

    $t_{\text{data}}=\frac{{\overline{x}}_{\text{Group}0}-{\overline{x}}_{\text{Group}2}}{\sqrt{\text{MSE}\cdot \left(\frac{1}{n_{\text{Group}0}}+\frac{1}{n_{\text{Group}2}}\right)}}=\frac{19.387-8.620}{\sqrt{\left(52.9485079\right)\left(\frac{1}{15}+\frac{1}{15}\right)}}\approx 4.052$

  • Test 3:
    

    $t_{\text{data}}=\frac{{\overline{x}}_{\text{Group}1}-{\overline{x}}_{\text{Group}2}}{\sqrt{\text{MSE}\cdot \left(\frac{1}{n_{\text{Group}1}}+\frac{1}{n_{\text{Group}2}}\right)}}=\frac{8.660-8.620}{\sqrt{\left(52.9485079\right)\left(\frac{1}{15}+\frac{1}{15}\right)}}\approx 0.015$

  When the requirements are met, $t_{\text{data}}$ follows a $t$ distribution with $n_t-k=45-3=42$ degrees of freedom, where $n_t$ represents the total sample size.

  

  FIGURE 22 Unadjusted $p$-values from Excel.
• Step 3 Find the Bonferroni-adjusted $p$-value for each hypothesis test. Figure 22 shows the unadjusted $p$-values for the values of $t_{\text{data}}$ from Step 2, using the function tdist $(t_{data},\mathrm{ df},2)$, where $\mathrm{df}=42$ and the 2 represents a two-tailed test. Then the Bonferroni-adjusted $\mathrm{p}\text{-}value=c\cdot \left(p\text{-value}\right)=3\cdot \left(p\text{-value}\right)$, for each hypothesis test.
  Page 688
  • Test 1: Bonferroni-adjusted $p\text{-value}=3\cdot 0.000225=0.000675$.
  • Test 2: Bonferroni-adjusted $p\text{-value}=3\cdot 0.000215=0.000645$.
  • Test 3: Bonferroni-adjusted $p\text{-value}=3\cdot 0.988103=2.964309$, but this value exceeds 1, so we set this $p$-value equal to 1.
• Step 4 For each hypothesis test, state the conclusion and the interpretation.
  • Test 1: The adjusted $p\text{-value} = 0.000675$, which is ≤0.01; therefore, reject $H_0$. There is evidence at the 0.01 level of significance that the population mean time spent in the open-ended part of the maze differs between Group 0 and Group 1.
  • Test 2: The adjusted $p\text{-value} = 0.000645$, which is ≤0.01; therefore, reject $H_0$. There is evidence at the 0.01 level of significance that the population mean time differs between Group 0 and Group 2.
  • Test 3: The adjusted $p\text{-value} = 1$, which is not ≤0.01; therefore, do not reject $H_0$. There is insufficient evidence at the 0.01 level of significance that the population mean time differs between Group 1 and Group 2.

• the requirements for ANOVA have been met, and
• the null hypothesis that the population means are all equal has been rejected.

• Step 1 For each of the $c$ hypothesis tests, state the hypotheses. There are $k=3$ means, so there will be $c=3$ hypothesis tests. Our hypotheses are:

  • Test 1: $\begin{array}{lll}{H}_0:{\mu }_{\text{High}}={\mu }_{\text{Medium}}\hfill & \text{versus}\hfill & H_a:{\mu }_{\text{High}}\ne {\mu }_{\text{Medium}}\hfill \end{array}$
  • Test 2: $\begin{array}{lll}{H}_0:{\mu }_{\text{High}}={\mu }_{\text{Low}}\hfill & \text{versus}\hfill & H_a:{\mu }_{\text{High}}\ne {\mu }_{\text{Low}}\hfill \end{array}$
  • Test 3: $\begin{array}{lll}{H}_0:{\mu }_{\text{Medium}}={\mu }_{\text{Low}}\hfill & \text{versus}\hfill & H_a:{\mu }_{\text{Medium}}\ne {\mu }_{\text{Low}}\hfill \end{array}$

  where ${\mu }_i$ represents the population mean score, for the $i$th category.

• Step 2 Find the Tukey critical value $q_{\text{crit}}$ and state the rejection rule. The total sample size is $n_t=43+44+43=130$. Use experimentwise error rate ${\alpha }_{\text{EW}}=0.05$, degrees of freedom $\text{df}=n_t-k=130-3=127$, and $k=\text{number of population means}=3$. Using the table of Tukey critical values (Table G in the Appendix), we seek $\text{df}=127$ on the left, but, when we don't find it, we conservatively choose df = 120. Then, in the column for $k=3$, we find the Tukey critical value $q_{\mathrm{crit}}=3.356$ (Figure 23). The rejection rule for the Tukey method is “Reject $H_0\text{if}{q}_{\text{data}}\ge q_{\text{crit}}$,” that is, Reject $H_0$ if $q_{\text{data}}\ge 3.356$.

  

  FIGURE 23 Finding the Tukey critical value $q_{\text{crit}}$.
  Page 689
• Step 3 Calculate the Tukey test statistic $q_{\text{data}}$ for each hypothesis test. From Figure 18 on page 678, we get the sample means, the sample sizes, and the mean square error MSE = 168. Thus,
  • Test 1:
    

    $q_{\text{data}}=\frac{{\overline{x}}_{\text{High}}-{\overline{x}}_{\text{Medium}}}{\sqrt{\frac{\text{MSE}}{2}\cdot \left(\frac{1}{n_{\text{High}}}+\frac{1}{n_{\text{Medium}}}\right)}}=\frac{81.09-79.36}{\sqrt{\left(\frac{168}{2}\right)\left(\frac{1}{43}+\frac{1}{44}\right)}}\approx 0.880$

  • Test 2:
    

    $q_{\text{data}}=\frac{{\overline{x}}_{\text{High}}-{\overline{x}}_{\text{Low}}}{\sqrt{\frac{\text{MSE}}{2}\cdot \left(\frac{1}{n_{\text{High}}}+\frac{1}{n_{\text{Low}}}\right)}}=\frac{81.09-79.63}{\sqrt{\left(\frac{168}{2}\right)\left(\frac{1}{43}+\frac{1}{43}\right)}}\approx 5.292$

  • Test 3:
    

    $q_{\text{data}}=\frac{{\overline{x}}_{\text{Medium}}-{\overline{x}}_{\text{Low}}}{\sqrt{\frac{\text{MSE}}{2}\cdot \left(\frac{1}{n_{\text{Medium}}}+\frac{1}{n_{\text{Low}}}\right)}}=\frac{79.36-70.63}{\sqrt{\left(\frac{168}{2}\right)\left(\frac{1}{44}+\frac{1}{43}\right)}}\approx 4.442$

• Step 4 For each hypothesis test, state the conclusion and the interpretation.
  • Test 1: $q_{\text{data}}=0.880$, which is not $\ge q_{\text{crit}}=3.356$; therefore, do not reject $H_0$. There is insufficient evidence at the 0.05 level of significance that the population mean student motivation scores differ between professors having high and medium self-disclosure on Facebook.
  • Test 2: $q_{\text{data}}=5.292$, which is $\ge q_{\text{crit}}=3.356$; therefore, reject $H_0$. There is evidence at the 0.05 level of significance that the population mean scores differ between high and low professor self-disclosure on Facebook.
  • Test 3: $q_{\text{data}}=4.442$, which is $\ge q_{\text{crit}}=3.356$; therefore, reject $H_0$. There is evidence at the 0.05 level of significance that the population mean scores differ between medium and low professor self-disclosure on Facebook.

FIGURE 24 Using Minitab confidence intervals to perform Tukey's test.

FIGURE 25 Using JMP confidence intervals to perform Tukey's test.

• Test 1: $\begin{array}{lll}{H}_0:{\mu }_{\text{Medium}}={\mu }_{\text{Low}}\hfill & \text{versus}\hfill & H_a:{\mu }_{\text{Medium}}\ne {\mu }_{\text{Low}}\hfill \end{array}$

  95% confidence interval for ${\mu }_{\text{Medium}}-{\mu }_{\text{Low}}$ is (2.14, 15.33), which does not contain zero, so we reject $H_0:{\mu }_{\text{Medium}}={\mu }_{\text{Low}}$ for level of significance $\alpha =0.05$.

• Test 2: $\begin{array}{lll}{H}_0:{\mu }_{\text{High}}={\mu }_{\text{Low}}\hfill & \text{versus}\hfill & H_a:{\mu }_{\text{High}}\ne {\mu }_{\text{Low}}\hfill \end{array}$

  95% confidence interval for ${\mu }_{\text{High}}-{\mu }_{\text{Low}}$ is (3.84, 17.09), which does not contain zero, so we reject $H_0:{\mu }_{\text{High}}={\mu }_{\text{Low}}$ for level of significance $\alpha =0.05$.

  Page 691

• Test 3: $\begin{array}{lll}{H}_0:{\mu }_{\text{High}}={\mu }_{\text{Medium}}\hfill & \text{versus}\hfill & H_a:{\mu }_{\text{High}}\ne {\mu }_{\text{Medium}}\hfill \end{array}$

  95% confidence interval for ${\mu }_{\text{High}}-{\mu }_{\text{Medium}}$ is (–4.86, 8.32), which does contain zero, so we do not reject $H_0:{\mu }_{\text{High}}={\mu }_{\text{Medium}}$ for level of significance $\alpha =0.05$.