Why do we study samples rather than populations?

What is the difference between a random sample and a convenience sample?

What is generalizability?

What is a volunteer sample, and what is the main risk associated with it?

What is the difference between random sampling and random assignment?

What does it mean to replicate research, and how does replication impact our confidence?

Ideally, an experiment would use random sampling so that the data would accurately reflect the larger population. For practical reasons, this is difficult to do. How does random assignment help make up for a lack of random selection?

What is the confirmation bias?

What is an illusory correlation?

How does the confirmation bias lead to the perpetuation of an illusory correlation?

In your own words, what is personal probability?

In your own words, what is expected relative-frequency probability?

Statisticians use terms like trial, outcome, and success in a particular way in reference to probability. What do each of these three terms mean in the context of flipping a coin?

We distinguish between probabilities and proportions. How does each capture the likelihood of an outcome?

How is the term independent used by statisticians?

One step in hypothesis testing is to randomly assign some members of the sample to the control group and some to the experimental group. What is the difference between these two groups?

What is the difference between a null hypothesis and a research hypothesis?

What are the two decisions or conclusions we can make about our hypotheses, based on the data?

What is the difference between a Type I error and a Type II error?

Forty-three tractor-trailers are parked for the night in a rest stop along a major highway. You assign each truck a number from 1 to 43. Moving from left to right and using the second line in the random numbers table below, select four trucks to weigh as they leave the rest stop in the morning.

Airport security makes random checks of passenger bags every day. If 1 in every 10 passengers is checked, use the random numbers table in Exercise 5.20 to determine the first 6 people to be checked. Work from top to bottom, starting in the fourth column (the fourth digit from the left in the top line), and allow the number 0 to represent the 10th person.

Randomly assign eight people to three conditions of a study, numbered 1, 2, and 3. Use the random numbers table in Exercise 5.20, and read from right to left, starting in the top row. (Note: Assign people to conditions without concern for having an equal number of people in each condition.)

You are running a study with five conditions, numbered 1 through 5. Assign the first seven participants who arrive at your lab to conditions, not worrying about equal assignment across conditions. Use the random numbers table in Exercise 5.20, and read from left to right, starting in the third row from the top.

Explain why, given the general tendency people have of exhibiting the confirmation bias, it is important to collect objective data.

Explain why, given the general tendency people have of perceiving illusory correlations, it is important to collect objective data.

What is the probability of hitting a target if, in the long run, 71 out of every 489 attempts actually hit the target?

On a game show, 8 people have won the grand prize and a total of 266 people have competed. Estimate the probability of winning the grand prize.

Convert the following proportions to percentages:

Convert the following percentages to proportions:

Using the random numbers table in Exercise 5.20, estimate the probability of the number 6 appearing in a random sequence of numbers. Base your answer on the numbers that appear in the first two rows.

Indicate whether each of the following statements refers to personal probability or to expected relative-frequency probability. Explain your answers.

Coincidence and the lottery: “Woman wins millions from Texas lottery for 4th time” read the headline about Joan Ginther’s amazing luck (Baird, 2010). Two of the tickets were from the same store, whose owner, Bob Solis, said, “This is a very lucky store.” Citing concepts from the chapter, what would you tell Ginther and Solis about the roles that probability and coincidence played in their fortunate circumstances?

Random numbers and PINs: How random is your personal identification number or PIN? Your PIN is one of the most important safeguards for the accounts that hold your money and valuable information about you. The BBC reported that, when choosing a four-digit PIN, “people drift towards a small subset of the 10,000 available. In some cases, up to 80% of choices come from just 100 different numbers” (Ward, 2013). Based on what you know about our ability to think randomly, explain this finding.

Random selection and a school psychologist career survey: The Canadian government reported that there are 7550 psychologists working in Canada (2013). A researcher wants to randomly select 100 of the Canadian psychologists for a survey study regarding aspects of their jobs. Use this excerpt from a random numbers table to answer the following questions:

Hypotheses and the school psychologist career survey: Continuing with the study described in Exercise 5.34, once the researcher had randomly selected the sample of 100 Canadian psychologists, she decided to randomly assign 50 of them to receive, as part of their survey materials, a (fictional) newspaper article about the improving job market. She assigned the other 50 to receive a (fictional) newspaper article about the declining job market. The participants then responded to questions about their attitudes toward their careers.

Random assignment and the school psychologist career survey: Refer to Exercises 5.34 and 5.35 when responding to the following questions:

Random selection and a survey of psychology majors: Imagine that you have been hired by the psychology department at your school to administer a survey to psychology majors about their experiences in the department. You have been asked to randomly select 60 of these majors from the overall pool of 300. You are working on this project in your dorm room using a random numbers table because the server is down and you cannot use an online random numbers generator. Your roommate offers to write down a list of 60 random numbers between 001 and 300 for you so you can be done quickly. In three to four sentences, explain to your roommate why she is not likely to create a list of random numbers.

Random selection and random assignment: For each of the following studies, state (1) whether random selection was likely to have been used, and explain whether it would have been possible to use it. Also, describe the population to which the researcher wanted to and could generalize, and state (2) whether random assignment was likely to have been used, and whether it would have been possible to use it.

Volunteer samples and a college football poll: A volunteer sample is a kind of convenience sample in which participants select themselves to participate. One recent year, USA Today published an online poll on its Web site asking this question about U.S. college football: “Who is your pick to win the ACC conference this year?” Eight options—seven universities, including top vote-getters Virginia Tech and Miami, as well as “Other”—were provided.

Samples and Cosmo quizzes: Cosmopolitan magazine (Cosmo, as it’s known popularly) publishes many of its well-known quizzes on its Web site. One quiz, aimed at heterosexual women, is titled “Are You Way Too Obsessed with Your Ex?” A question about “your rebound guy” offers these three choices: “Any random guy who will take your mind off the split,” “A doppelgänger of your ex,” and “The polar opposite of the last guy you dated.” Consider whether you want to use the quiz data to determine how obsessed women are with their exes.

Samples and political leanings: On its Web site, Advocates for Self-Government offers the “World’s Smallest Internet Political Quiz,” focusing on the U.S. political spectrum. Using just 10 questions, the quiz identifies a person’s political leanings. As of 2012, almost 20 million people had taken the quiz. In 2007, the Web site reported the following breakdown into the five possible categories: centrist, 33.49%; conservative, 8.88%; libertarian, 32.64%; liberal, 17.09%; and statist (big government), 7.89%.

Random selection or random assignment: For each of the following hypothetical scenarios, state whether selection or assignment is being described. Is the method of selection or assignment random? Explain your answer.

Bias about driver gender: Assume that one of your male friends is complaining about female drivers, and says that men are much better drivers than women. If objective studies of the driving performance of men and women revealed no mean difference between the two groups, what kind of bias has your friend shown?

Confirmation bias, illusory correlation, and driver gender: Referring to your friend from Exercise 5.43, assume he backs up his claim by recounting two events over the past week in which female drivers have erred (e.g., cut him off in traffic, not used a turn signal). Explain how the confirmation bias is at work in your friend’s statements and how this confirmation bias may be perpetuating an illusory correlation.

Confirmation bias and negative thought ­patterns: Explain how the general tendency of a confirmation bias might make it difficult to change negative thought patterns that accompany Major Depressive Disorder.

Probability and coin flips: Short-run proportions are often quite different from long-run probabilities.

Probability, proportion, percentage, and Where’s Waldo?: Salon.com reporter Ben Blatt analyzed the location of Waldo in the game in which you must find Waldo, a cartoon man who always wears a red-and-white-striped sweater and hat, in a highly detailed illustration (2013). Blatt reported that “53 percent of the time Waldo is hiding within one of two 1.5-inch tall bands, one starting three inches from the bottom of the page and another one starting seven inches from the bottom, stretching across the spread.” One of Blatt’s colleagues used this trick to find Waldo more quickly than another colleague who did not have this information across 11 illustrations.

Independent trials and Eurovision Song Contest bias: As reported in the Telegraph (Highfield, 2005), Oxford University researchers investigated allegations of voting bias in the annual Eurovision Song Contest, which pits pop music acts from across Europe, one per country, against each other. The research team found that neighboring countries tended to vote as a block—Norway with Sweden, Belarus with Russia, and Greece with Cyprus, for example. Explain why one could not consider the votes to be independent of each other in this case.

Independent trials and the U.S. presidential ­election: Nate Silver is a statistician and journalist well known for his accurate prediction tools. In an article leading up to the 2012 U.S. presidential election in which Barack Obama bested Mitt Romney, Silver (2012) explained his prediction methods as “principally, an Electoral College simulation, [which] therefore relies more heavily on state-by-state polls.” Consider Silver’s consolidation of data from polls across the 50 states. In what way are these polls likely to be independent trials? Why could someone argue that they are not truly independent trials?

Independent or dependent trials and ­probability: Gamblers often falsely predict the outcome of a future trial based on the outcome of previous trials. When trials are independent, the outcome of a future trial cannot be predicted based on the outcomes of previous trials. For each of the following examples, (1) state whether the trials are independent or dependent and (2) explain why. In addition, (3) state whether it is possible that the quote is accurate or whether it is definitely fallacious, explaining how the independence or dependence of trials influences accuracy.

Null hypotheses and research hypotheses: For each of the following studies, cite the likely null hypothesis and the likely research hypothesis.

Decision about null hypotheses: For each of the following fictional conclusions, state whether the researcher seems to have rejected or failed to reject the null hypothesis (contingent, of course, on inferential statistics having backed up the statement). Explain the rationale for your decision.

Type I versus Type II errors: Examine the statements from Exercise 5.52, repeated here. For each, if this conclusion were incorrect, what type of error would the researcher have made? Explain your answer.

Rejecting versus failing to reject an invitation: Imagine you have found a new study partner in your statistics class. One day, your study partner asks you to go on a date. This invitation takes you completely by surprise, and you have no idea what to say. You are not attracted to the person in a romantic way, but at the same time you do not want to hurt his or her feelings.

Confirmation bias, errors, replication, and horoscopes: A horoscope on Astrology.com stated: “A big improvement is in the works, one that you may know nothing about, and today is the day for the big unveiling.” A job-seeking recent college graduate might spot some new listings for interesting positions and decide the horoscope was right. If you look for an association, you’re likely to find it. Yet, over and over again, careful researchers have failed to find evidence to support the accuracy of astrology (e.g., Dean & Kelly, 2003).

Probability and sumo wrestling: In their book Freakonomics, Levitt and Dubner (2005) describe a study conducted by Duggan and Levitt (2002) that broached the question: Do sumo wrestlers cheat? Sumo wrestlers garner enormous respect in Japan, where sumo wrestling is considered the national sport. The researchers examined the results of 32,000 wrestling matches over an 11-year time span. If a wrestler finishes a tournament with a losing record (7 or fewer wins out of 15 matches), his ranking goes down, as do the money and prestige that come with winning. The researchers wondered whether, going into the last match of the tournament, wrestlers with 7-7 records (needing only 1 more win to rise in the rankings) would have a better-than-expected win record against wrestlers with 8-6 records (those who already are guaranteed to rise in the rankings). Such a phenomenon might indicate cheating. One 7-7 wrestler (wrestler A), based on past matches against a given 8-6 opponent (wrestler B), was calculated to have won 48.7% of the time.

123

Testimonials and Harry Potter: Amazon and other online bookstores offer readers the opportunity to write their own book reviews, and many potential readers scour these reviews to decide which books to buy. Harry Potter books attract a great deal of these reader reviews. One Amazon reviewer, “bel 78,” submitted her review of Harry Potter and the Half-Blood Prince from Argentina. Of the book, she said, “It’s simply outstandingly good,” and suggested that readers of her review “run to get your copy.” Do these reviews have an impact? In this case, more than 900 people had read bel 78’s review, and close to 700 indicated that the review was helpful to them.

Horoscopes and predictions: People remember when their horoscopes had an uncanny prediction—say, the prediction of a problem in love on the exact day of the breakup of a romantic relationship—and decide that horoscopes are accurate. Munro & Munro (2000) are among those who have challenged such a conclusion. They reported that 34% of students chose their own horoscope as the best match for them when the horoscopes were labeled with the signs of the zodiac, whereas only 13% chose their own horoscope when the predictions were labeled only with numbers and in a random order. Thirteen percent is not statistically significantly different from 8.3%, which is the percentage we’d expect by chance.

Alcohol abuse interventions: Sixty-four male students were ordered, after they had violated university alcohol rules, to meet with a school counselor. Borsari and Carey (2005) randomly assigned these students to one of two conditions. Those in the first condition were assigned to undergo a newly developed brief motivational interview (BMI), an intervention in which educational material relates to the students’ own experiences; those in the second condition were assigned to attend a standard alcohol education session (AE) in which educational material is presented with no link to students’ experiences. Based on inferential statistics, the researchers concluded that those in the BMI group had fewer alcohol-related problems at follow-up, on average, than did those in the AE group.

124

Treatment for depression: Researchers conducted a study of 18 patients whose depression had not responded to treatment (Zarate, 2006). Half received one intravenous dose of ketamine, a hypothesized quick fix for depression; half received one intravenous dose of placebo. Far more of the patients who received ketamine improved, as measured by the Hamilton Depression Rating Scale, usually in less than 2 hours, than patients on placebo.