Chapter 1. Chi-Square Test: And the Winner Is?

1.1 Chi-Square Test: And the Winner Is?

There are all kinds of awards in life. Some come from beating out the competition, and others from crossing over a finish line. But what makes an award an award? Is it the actual prize that comes with a win or merely the symbol that the award represents? One study looked at whether symbolic awards would encourage altruistic behavior even without any tangible benefit to the winner (Gallus, 2017).

John D. Buffington/DigitalVision/Getty Images

Question 1.1

The Wikipedia platform set up a natural field experiment to see if giving out awards would make its new editors (all volunteers) more likely to keep its pages up to date with correct information—even without any personal benefit for doing so (Gallus, 2017). The Swiss national Wikipedia portal made up an honor, the “Edelweiss with Star,” for a random selection of new editors. They recognized the new editors for joining the site and for their contributions, without explicitly defining what those contributions were. Of course, the new editors did not know that their selection for the honor was completely random! The researchers examined whether new editors who received the “Edelweiss with Star” would be more likely to continue editing than those who did not. An additional twist is that on Wikipedia the volunteer editors are anonymous, so the “Edelweiss with Star” went to a username rather than the actual person. That’s not something easy to advertise offline or put on a résumé.

QVxb3lHBeUa0lFDgXiSMVfvKUEkNOubWDVc5CXyd/BEoyA5K526lA1aMsaY7fVQUsXKIF4a13j3HIpPEyN2t+pMfn3+Ar3lE58pjGdbqaSYxI0Wj7W2x26ay83Aqzn3R2zSk4Wfa6Y/D/3v6xH/Qm5bwURAfssRbhcRJ+qjhc8gib8hP7X1PmC8ZDKYX+BVZihNNO8swb9rWlvqoFe9nXDn91K7sPhjii7nhzDC7fT7ABkOgRNbm8E7m027iRFVr5vDRfSLWqW+qrgmt2RGZtYIhYl5ZdzsziD7cZ6yLkMPr/jtpfT2nI/8u+SdeosO8yJtDY95kUETLCF/CSl6B2wp5BnEU/UQ9FIrXQdeYDdYmG02aSkWHJI81YcPrPUJ6V+Dt52Qiu0sDI3HfQW5ikjjWUuRRoXnlkG4A9Y36XWi3eKmkyhrgWadrNhRBiFsCIm81o4BzbETocqLUfe3HE3pdvIqFZULEmmNEy6DoxGdfdAF3gbSbFoGgwvvpp3X0wygkdE7xAXwB2z0iNz4XB1hJ/qE45O0oLbtFfpsMn91yxulhATKh81cLf//laO36apb5Jm8b949vugXtfBgWOzc3OrSeydU145CmOsXMh3uoI7ls6rmwm4U53jKzSdqfUwfpJJdFbcSzvUv7B4fOoOm1Ouc/0rKWLx/C0dpFQgmsRs3CbN+8ykU8CplguB1kp75NuhP4utUi2KK/e6U2GLhvAiJvWR4ErRQ9OnpJEWeDJeEsuqQhLm0dGnWR599KXO3nC4JpDisT2dVjITEZPsXvxns=

1.2

The award’s intention was to find out how people behaved when they believed they had been given an award without knowing the award was randomly distributed to the “winners.” The researchers examined what we’ll call “award status”—whether editors did or did not receive an award. They compared whether award recipients—what they called the “treatment group”—would be more likely to continue using the platform than those who did not receive an award—the control group. The researchers wrote: “Treatment and control groups therefore comprise 1,617 and 2,390 editors, respectively. … Thanks to the random assignment of the treatment, potential confounding variables are on average distributed equally between the treatment and control groups.”

1.3

This graph shows the share of Wikipedia editors who were active on the site in the month after awards were issued. The researchers reported that “the difference observed in the bar chart is indeed statistically significant … (χ ² (1) = 18.22, p = 0.000)” (p. 4006).

image description

The bar graphs shows whether or not a Wikipedia editor was active on the site in the month after awards were issued. The horizontal x-axis is labeled “Condition” and shows two vertical bars, one for the condition in which an editor received an award and one for the condition in which the editor received no award. The vertical y-axis is labeled “Share of group active over the next month” and runs from 0.0 on the bottom to 0.5 at the top, in intervals of 0.1. The share of the group that was active after receiving an award was 0.42, or 42%, and the share of the group active after not receiving an award was 0.35, or 35%.

Question 1.3

C2z2tkYGMiMNGnNzttKWFRpP6cq2Px1KfzsVJ0rnxHlsb1gB18p2dQjE1vk2KeF0n6HC8qoKcYx46ikplGInDHRxnPfnAACoSMFBvVcB0KakiRuIB3eEdOc6yGU6B5mFHQBGro59D4nb8bBYKWSgoi+DIhtX3PMkFrnxVcZre7+Bd8K2TOl0o6jWBXLPa/spcje52DShM/HFTJo8PrTs2u9shiznc2mB0els8ncrlcFWGD8x50KtkPt1E0Zasz+UtoN6K9vfjU6LMhm9mCftdGuY9Azlt/97XGtgrPEvcuHSUef/CXmVZO2C0xwySGbyqghSK/y3KM+aVzh5j1htNaUNqBtoZbH8zaH7pmuZW6aWosbv4E+nAAiYcGNKh602CqL8LDnA4KiKO6yxTVdH4d/1Cuomik7qXYfIzEcsY5OrIEsPRwHYVr4kr2nPaF79PuxgLNoQLtsut/F2Dmcualm/b9xOw2HfL78npOXtEjd87lmPrEGf+73zn5djW98fOzWH9PbxNKCz/hA7cNMH58Q5bxTqf8wct+JE9zBXoXTy0fifP2waQMrGRRs=

Correct! We also know that this finding is statistically significant because the p value is less than the typical alpha level of 0.05.

Actually, we also know that this finding is statistically significant because the p value is less than the typical alpha level of 0.05.

1.4

Here is the same graph, depicting the share of Wikipedia editors who were active on the site in the month after awards were issued, and the same results that the researchers reported: “The difference observed in the bar chart is indeed statistically significant … (χ ² (1) = 18.22, p = 0.000)” (p. 4006).

image description

The bar graphs shows whether or not a Wikipedia editor was active on the site in the month after awards were issued. The horizontal x-axis is labeled “Condition” and shows two vertical bars, one for the condition in which an editor received an award and one for the condition in which the editor received no award. The vertical y-axis is labeled “Share of group active over the next month” and runs from 0.0 on the bottom to 0.5 at the top, in intervals of 0.1. The share of the group that was active after receiving an award was 0.42, or 42%, and the share of the group active after not receiving an award was 0.35, or 35%.

Question 1.4

J0N22fvTSAd6i7CvuywgM7XR7OpM9qi46U82segU4cW1WwFKXjJbKQcm5IFlLk9CpXnpsAR6VZx2PtyCVxAKSI/AZNj352KEMlzvqYTQ9RASjkCSGrzOZnauUYRnLLVWAiqoNmYp1SstvIm5O+BtROUTz0xozwPxTf89y/BGGsU+uf7U3XQhtATtzdSD8GZSdQ9q5VhiOhDYfKQBvwmxzNXIOZR81bbjfDtvW9COnpiO4DrxNFrrus+Oo7QqeJfBwimJ7I5kdZnWBQUMeQlnf4bZuuZgYuqbImqK01kt3V4Ndfgiew5v8ikRpETRebqdP3rbyLlriZPXYqzL/K3bglZ4v14KDibz2VYz86zMIzcRLlaSPLPkTRpDZ5w6+zbxqG1ERlc2rJaxpPYCrVjpqCtKl4nSYoYQnnOzibqGXyRLS9/wMXasASnZgEaVyogMy+2wO6UbcaRxuEmXjsp3C/YfshT/MiF8jWO5Wvvy7o0rGNfG+XE4sGNXZuxq/smIq847nsaZ/AXLzXD0RklOtrLzVW6Ioh83lMnB03BxPbgyB4Z4KiK/RYryVf4r3bOV7MzkV/azT9NoJAgLpLAr+henN601zjRG9hcWUoDLPmEcJrwHcNwFITChe4g4DmUeWZzP+q1q9zsMcuHN634kElEbdT4BH+evhZ8eUd37oz9xFuWRe0FF2B005Li+w1c3OsSNb304kyi6GTNHbd1swOZhB9kRWnmBYdLPE75KRBPPN0MlM0pk/T9T7d5MsvygFYWyjjRd087BNDlszqezHDwBW+WxgiC0lU4OxMN8QIA9pZgtkHkGwGiDlzfnh6Eh2BQzmoZ776iV/57+MOuWYloL0KJnIF/6UpunZfQB3OTDlsCuDZJQPlJ6GRVmfB9CM3Q10aXmqkmFqymws3ddGiSm5MVFmyTr

Correct! We can reject the null hypothesis and conclude that editors who received an award were more likely to be active in the month following the award than those who did not receive the award. (It turns out that this finding extends for up to a year, according to the additional analyses that the researchers performed!) Note that we do not need to calculate adjusted standardized residuals in this case because there are just two groups, so we already know where the difference lies.

Actually, we can reject the null hypothesis and conclude that editors who received an award were more likely to be active in the month following the award than those who did not receive the award. (It turns out that this finding extends for up to a year, according to the additional analyses that the researchers performed!) Note that we do not need to calculate adjusted standardized residuals in this case because there are just two groups, so we already know where the difference lies.

1.5

1.6

1.7

1.8

The researchers found that award winners were more likely to be active than non-award winners, not just during the month after the awards were given, but for up to a year! (And these were made-up awards with no actual prizes!) But the researchers didn’t stop with just those analyses. They reported: “Excluding the award project’s page … does not change the results. … This suggests that award recipients who post [additional information] on the project’s page also make other contributions. Additionally, excluding editors’ own pages somewhat reduces the effect, but still shows a difference of five percentage points, which is highly statistically significant” (p. 4009). So, the additional Wikipedia activity wasn’t just editors adding to their original entry or writing more about themselves—the editors evidently took on something new.

1.9

The bottom line: Even symbolic awards can make a difference in behavior. So, when considering how to motivate people, remember that a job well done might be its own reward, but a little recognition doesn’t hurt.

Caiaimage/Sam Edwards/Getty Images