Chapter 3. ESP

3.1 Welcome

Think Like a Scientist
true
ESP (extrasensory perception)
Perception of information by some means other than through the normal processes of sensation.
extrasensory perception (ESP)
Perception of information by some means other than through the normal processes of sensation.
meta-analysis
A statistical technique that involves combining and analyzing the results of many research studies on a specific topic in order to identify overall trends.
statistically significant
A mathematical indication that research results are not very likely to have occurred by chance.

Think Like a Scientist

ESP

By:

Susan A. Nolan, Seton Hall University

Sandra E. Hockenbury

With the assistance of Scott Cohn, Western State Colorado University

REFERENCES

Fram, Alan. (2007). That’s the spirit: One-third of people believe in ghosts—and some report seeing one. Associated Press Archive, Record No. d8SGT3CG0. Associated Press, Washington, DC.

French, Chris. (2012). Halloween challenge: Psychics submit their powers to a scientific trial. The Guardian. Retrieved from http://www.theguardian.com/science/2012/oct/31/halloween-challenge-psychics-scientific-trial

Goldsmiths, University of London. (2012). Psychics fail ‘Halloween Challenge’ at Goldsmiths [Press release]. Retrieved from http://www.gold.ac.uk/news/pressrelease/?releaseID=972

Hyman, Ray. (2010). Meta-analysis that conceals more than it reveals: Comment on Storm et al. (2010). Psychological Bulletin, 136, 486-490.

Rouder, Jeffrey N.; Morey, Richard D.; & Province, Jordan M. (2013). A Bayes factor meta-analysis of recent extrasensory perception experiments: Comment on Storm, Tressoldi, and Di Risio (2010). Psychological Bulletin, 139, 241–247.

Storm, Lance; Tressoldi, Patrizio E.; & Di Risio, Lorenzo. (2010). Meta-analysis of free-response studies, 1992–2008: Assessing the noise reduction model in parapsychology. Psychological Bulletin, 136, 471–485.

FAQ

What is Think Like a Scientist?
Think Like a Scientist is a digital activity designed to help you develop your scientific thinking skills. Each activity places you in a different, real-world scenario, asking you to think critically about a specific claim.

Can instructors track your progress inThink Like a Scientist?
Scores from the five-question assessments at the end of each activity can be reported to your instructor. To ensure your privacy while participating in non-assessment features, which can include pseudoscientific quizzes or games, no other student response is saved or reported.

How is Think Like a Scientist aligned with the APA Guidelines 2.0?

The American Psychological Association’s “Guidelines for the Undergraduate Psychology Major” provides a set of learning goals for students. Think Like a Scientist addresses several of these goals, although it is specifically designed to develop skills from APA Goal 2: Scientific Inquiry and Critical Thinking. “ESP” covers many outcomes, including:

  • Interpret, design, and conduct basic psychological research: Describe the fundamental principles of research design [consider effect of manual versus computerized randomization]
  • Incorporate sociocultural factors in scientific inquiry: Relate examples of how a researcher’s value system, sociocultural characteristics, and historical context influence the development of scientific inquiry on psychological questions [acknowledge role of prior belief when performing research]

3.2 Introduction

This activity invites you to explore the existence of psychic abilities. First, you’ll identify the claim about extrasensory perception, or ESP. Then, you will explore evidence about the existence of ESP. You’ll also examine how your beliefs about ESP affect how you interpret research on this phenomenon. You will examine alternative explanations for why people believe in ESP despite a lack of evidence that it exists. Finally, you’ll consider the need to ask good questions about science, even when there is a high-quality source of available information, such as a peer-reviewed journal article.

3.3 Identify the Claim

1

Identify the Claim

3.3.1 Psychic Abilities

Have you ever had a hunch that turned out to be true? Do you believe in clairvoyance, telepathy, or other forms of extrasensory perception (ESP)? As you learned in your textbook chapter on sensation and perception, ESP is the acquisition of information outside of the normal processes of sensation like seeing or hearing. It’s kind of like Bluetooth for your brain. About half of Americans believe in ESP (Fram, 2007). Are you one of them? Later, you can complete a short survey to find out.

This illustration of two human beings indicates connection between the two through extrasensory perception, or ESP.
Pasieka/Science Source

3.3.2 Do You Think You Have Psychic Abilities?

If you are interested in paranormal phenomena, you may have looked for information about ESP online. Googling “Do I have ESP?” yields dozens of sites offering quizzes and exercises that claim to help you recognize or develop this “skill.” As an example, look at this mock Web site, which is similar to many “psychic phenomena” sites online. (You can click the image to enlarge.) On the next screen, you can take the quiz advertised on this site.

This is a mock psychic phenomenon website that claims to help its users determine if they have ESP.
Dundanim/Shutterstock

3.3.3 Take the Mock Quiz

hockenbury_tls_ch3_mock_quiz

Imagine you are intrigued by the Web site you just viewed, and you want to take the “psychic inclinations” test. The mock quiz that follows was developed based on actual quizzes available online, and you can try it now. You must complete all 15 questions in this quiz in order to receive your “result” and advance to the next screen of this Think Like a Scientist activity. (Note: Your responses to individual questions will not be saved and your “score” will not be reported to your instructor.)

Complete a short survey to find out if YOU have psychic inclinations! For each item, click either yes or no.

1. Do you feel energized by some people and exhausted after being around others?

Question

Rr2o7bm66VWWm0G9YC195Ye6EgfkN0NmaIC2CtRs1h56mM8GzEBX3UqHsUJfGloBWvyhzUv3dHCZgOBPq4u6UwVNtUGpWUtZYFC/yg==

3.3.4 Take the Mock Quiz

2. Do you ever feel like plants have energy or a presence?

Question

Rr2o7bm66VWWm0G9YC195Ye6EgfkN0NmaIC2CtRs1h56mM8GzEBX3UqHsUJfGloBWvyhzUv3dHCZgOBPq4u6UwVNtUGpWUtZYFC/yg==

3.3.5 Take the Mock Quiz

3. Have you ever dreamt about an event, or sequence of events, that later happened?

Question

Rr2o7bm66VWWm0G9YC195Ye6EgfkN0NmaIC2CtRs1h56mM8GzEBX3UqHsUJfGloBWvyhzUv3dHCZgOBPq4u6UwVNtUGpWUtZYFC/yg==

3.3.6 Take the Mock Quiz

4. Do you ever feel the presence of people shortly after they die?

Question

Rr2o7bm66VWWm0G9YC195Ye6EgfkN0NmaIC2CtRs1h56mM8GzEBX3UqHsUJfGloBWvyhzUv3dHCZgOBPq4u6UwVNtUGpWUtZYFC/yg==

3.3.7 Take the Mock Quiz

5. When meeting people for the first time, do you ever feel like you already know them from somewhere?

Question

Rr2o7bm66VWWm0G9YC195Ye6EgfkN0NmaIC2CtRs1h56mM8GzEBX3UqHsUJfGloBWvyhzUv3dHCZgOBPq4u6UwVNtUGpWUtZYFC/yg==

3.3.8 Take the Mock Quiz

6. Do you ever feel like there is another presence helping you? For example, when struggling to arrive at a decision, does an answer ever ‘come to you’ out of the blue?

Question

Rr2o7bm66VWWm0G9YC195Ye6EgfkN0NmaIC2CtRs1h56mM8GzEBX3UqHsUJfGloBWvyhzUv3dHCZgOBPq4u6UwVNtUGpWUtZYFC/yg==

3.3.9 Take the Mock Quiz

7. Are uncanny coincidences and serendipitous circumstances common in your life?

Question

Rr2o7bm66VWWm0G9YC195Ye6EgfkN0NmaIC2CtRs1h56mM8GzEBX3UqHsUJfGloBWvyhzUv3dHCZgOBPq4u6UwVNtUGpWUtZYFC/yg==

3.3.10 Take the Mock Quiz

8. Do you communicate well with small children and animals?

Question

Rr2o7bm66VWWm0G9YC195Ye6EgfkN0NmaIC2CtRs1h56mM8GzEBX3UqHsUJfGloBWvyhzUv3dHCZgOBPq4u6UwVNtUGpWUtZYFC/yg==

3.3.11 Take the Mock Quiz

9. Have you ever been alone and heard someone talking to you?

Question

Rr2o7bm66VWWm0G9YC195Ye6EgfkN0NmaIC2CtRs1h56mM8GzEBX3UqHsUJfGloBWvyhzUv3dHCZgOBPq4u6UwVNtUGpWUtZYFC/yg==

3.3.12 Take the Mock Quiz

10. Do you ever make decisions to avoid certain situations because of a ‘gut feeling’ that something will go wrong?

Question

Rr2o7bm66VWWm0G9YC195Ye6EgfkN0NmaIC2CtRs1h56mM8GzEBX3UqHsUJfGloBWvyhzUv3dHCZgOBPq4u6UwVNtUGpWUtZYFC/yg==

3.3.13 Take the Mock Quiz

11. Do you often think of people right before receiving a phone call or text from them?

Question

Rr2o7bm66VWWm0G9YC195Ye6EgfkN0NmaIC2CtRs1h56mM8GzEBX3UqHsUJfGloBWvyhzUv3dHCZgOBPq4u6UwVNtUGpWUtZYFC/yg==

3.3.14 Take the Mock Quiz

12. Did you have imaginary friends as a child that you could see, but were invisible to others?

Question

Rr2o7bm66VWWm0G9YC195Ye6EgfkN0NmaIC2CtRs1h56mM8GzEBX3UqHsUJfGloBWvyhzUv3dHCZgOBPq4u6UwVNtUGpWUtZYFC/yg==

3.3.15 Take the Mock Quiz

13. Have you ever seen or sensed an aura?

Question

Rr2o7bm66VWWm0G9YC195Ye6EgfkN0NmaIC2CtRs1h56mM8GzEBX3UqHsUJfGloBWvyhzUv3dHCZgOBPq4u6UwVNtUGpWUtZYFC/yg==

3.3.16 Take the Mock Quiz

14. If you stare at someone from behind, do they often turn around to look at you?

Question

Rr2o7bm66VWWm0G9YC195Ye6EgfkN0NmaIC2CtRs1h56mM8GzEBX3UqHsUJfGloBWvyhzUv3dHCZgOBPq4u6UwVNtUGpWUtZYFC/yg==

3.3.17 Take the Mock Quiz

15. When entering a space that you haven’t been in before, do you often get a feeling about the environment’s history that takes the form of an unexpected emotion or a physical sensation?

Question

Rr2o7bm66VWWm0G9YC195Ye6EgfkN0NmaIC2CtRs1h56mM8GzEBX3UqHsUJfGloBWvyhzUv3dHCZgOBPq4u6UwVNtUGpWUtZYFC/yg==

3.3.18 Are Some People Psychics?

hockenbury_tls_ch3_answer_student

What does your score on the “Psychic Abilities Quiz” mean in terms of your own “psychic inclinations”? Do you believe some people taking this quiz will learn that they have the potential for actual psychic abilities? In your own words, explain what a quiz like this tells you. We’ll come back to your answer later.

Question 3.1

snByA+gbH414Bn9ZPCHnGz5j+bJaRyfVU0UDU1U0BNJsReHmhwRRqokvbDf6tLrxGm9FkMaPPcTU2gfIR5eBEw==
If you believe the quiz results indicate an actual potential for psychic abilities, you are among the half of all Americans who believe in ESP.

Question 3.2

IPI4NpwYepbmCl4DxYsu/Ji5OlYPvO/HFenc/7geXhbO0dhjI8IqxLoxCgdxMfT4Nneh3PsCL7hpclAeTUUfSYvd33MHvcUNyU44pI7gqmXSFyR0oIwQAGal6OLI1K7hmhGQOqidcupfXGmNgpy7wa+kevOxJ22307ZXCbMdaJGXi+CCmNrUJAJeJp2DN072MCL6dkcRk1a6GAo7ZltNMybZDml5iLrfUJ0yBIix2Eg9WUR45mffTqHNrTOMVOW+n9usHFdc8Xjxkj9W5v0oQG+qLuXEn7n7UFJrbVFLc8CN8JTZOe3DZW67W0DFPy8MLcN2O2E5FvThf2dz1HNR7ycQRcdLsCBoAGfrtmiD0p//88VEznLDou4uGgp+8DkZbg47ZrNBIN0JIDocdlEA10CS69ZnY1hLvHzIgfr5wnGtxChQ2yv5AD5QXyHMMoz/wlzibcd70QSE9dNOlkuPZPgUSH1htu6a49qscgWTGE4=
Correct! These sites claim ESP is real, and that you might have the potential for such powers.
Actually, the answer is C. These sites claim ESP is real, and that you might have the potential for such powers.

3.4 Evaluate the Evidence

2

Evaluate the Evidence

3.4.1 So Is ESP Real?

Although the general public is split about whether ESP is real, only a small number of researchers are actively investigating this topic. The vast majority of psychologists agree that there is no convincing scientific evidence that ESP exists (Hyman, 2010).

So if the evidence is overwhelming, are you wondering why there is any debate at all? Whether or not you asked that question may have something to do with your response to the quiz you just took. We’ll look at the evidence for and against the existence of ESP. But first, let’s examine how scientists investigate ESP.

This photo shows a balance that may possibly indicate the scientific evidence available to prove and disprove ESP.
rommma/Shutterstock

3.4.2 Putting Psychics to the Test

The following small-scale example shows how ESP can be tested in the lab. Psychologists at the Anomalistic Psychology Research Unit, part of the University of London, put professional psychics Kim Whitton (shown below, at left) and Patricia Putt to the test. Whitton and Putt volunteered to provide psychic “readings” for five women (Goldsmiths, University of London, 2012). The entire test was conducted in silence. As you can see here, the women were separated by a screen from the psychics, who recorded their readings as written notes specific to each individual woman. The women were then asked to identify their own readings from among the five that Putt and Whitton had written. Although the psychics called the test “very fair and simple,” they both failed it. For Whitton, only one woman correctly guessed which reading described her—exactly what you’d expect by chance. And for Putt, not one woman guessed correctly.

Two young women in a laboratory undertake testing for ESP.
Dave Hughes

3.4.3 A Pro-ESP Meta-Analysis

Only a few psychology researchers study ESP. Researchers at the University of London regularly conduct experiments like the one just described, but have never found any support for claims about the existence of ESP. However, in a meta-analysis of 67 different studies, Australian psychologist Lance Storm and colleagues (2010) did find statistical support for the existence of ESP. In some of the studies that Storm and his colleagues examined, a “sender” was asked to telepathically broadcast information about an item to a “receiver” in a separate room. The “receiver” then attempted to choose that item from among a limited number of options.

The researchers did not explain how psychic abilities might work, usually an important part of a scientific write-up. And ESP cannot be explained by widely accepted principles of physics (Rouder & others, 2013). Nevertheless, findings from this meta-analysis (2010) provide evidence capable of convincing a skeptical but open-minded reader that ESP is real.

A young women experiments with ESP.
Susan Nolan

3.4.4 A Rebuttal Meta-Analysis

A few years later, U.S. psychologist Jeffrey Rouder and his colleagues performed their own meta-analysis, re-examining Storm and colleagues’ findings (2013). Focusing on studies that provided the most support for the existence of ESP, they found errors in many of them. Some studies, for example, did not use proper randomization procedures.

Ideally, researchers prevent any possibility of human interference—whether conscious or unconscious—by using a computer to determine relevant aspects of their experiment. For example, a computer can be used to randomly assign participants to groups or determine the order in which participants view items. However, in some studies included in the Storm team’s meta-analysis, researchers themselves made some decisions about the inclusion or order of items to be shown to the “sender.” Once human judgment enters the picture, the experiment is not truly random; there may be some pattern that is created, even unconsciously, that could change the results of a study.

These cards are used to test individuals’ powers of ESP.
originalpunkt/Shutterstock

3.4.5 The Effects of Randomization

Do you think it mattered that some studies used full randomization and others did not? Rouder and his colleagues compared these two different types of studies. They found that the experiments that used full computerized randomization showed many more results at or below the level of chance than the ones that did not use full computerized randomization. You can see this difference in the graph. When an outcome is at or below the level of chance, it does not provide support for the experiment’s hypothesis—in this case, the existence of ESP. So these studies provided less support for the existence of ESP than a meta-analysis including studies that used a manual randomization process.

The graph compares the ESP study results calculated by both manual randomization and computerized randomization.

3.4.6 Study Choice in Meta-Analysis

Once scientists track down all of the research in a particular area, how do they choose which studies to include in a meta-analysis? They can’t just flip a coin. A good scientist comes up with careful guidelines and only uses studies that meet those guidelines.

Rouder and his colleagues questioned the selection process for the studies that Storm and his colleagues included in their meta-analysis. Rouder’s team cited examples of studies that they believed should not have been included, as well as ones they believed should have been included. In one case, a research team conducted two experiments that both seemed to meet the criteria to be included in the meta-analysis. Yet, Storm’s team only included one of them—the one that provided more support for the existence of psychic abilities. And this is just an examination of the published studies. What about the ones that have never seen the light of day?

This photo shows a hand flipping a coin in the air.
Boule/Shutterstock

3.4.7 The Missing Research

hockenbury_tls_ch3_quiz_function

It is well known that research that finds an effect—say, that ESP works or that there is a difference between two groups—is more likely to be published. Even editors of academic journals and of news sources like magazines are swayed by the excitement of a statistically significant finding. Imagine you are an editor considering which articles to include in the next issue of your popular magazine. In each pair, indicate which fictional headline draws your attention.

Scientists Find Evidence for ESP
Scientists Fail to Find Evidence for ESP

Seven Scientific Tricks Found to Help with Weight Loss
Seven Scientific Tricks That Are Not Found to Help with Weight Loss

Drinking Coffee Shown to Raise IQ
Drinking Coffee Not Shown to Raise IQ
Photo shows a file drawer stuffed with many files indicating filed studies of ESP.
Thomas M Perkins/Shutterstock

3.4.8 So Has ESP Been Debunked?

The way that research studies work is that they either: 1) provide evidence for something, as Storm’s team did with respect to ESP, or 2) fail to find evidence for something, as Rouder’s team did with respect to ESP. What this means is that Rouder’s team’s findings don’t allow us to say definitively that ESP doesn’t exist. However, their meta-analysis does show that the evidence favoring ESP is far weaker than Storm’s team claimed.

Rouder’s team concluded that the Storm team’s meta-analysis is “certainly not sufficient to sway an appropriately skeptical reader” (2013).

PathDoc/Shutterstock

3.5 Consider Alternative Explanations

3

Consider Alternative Explanations

3.5.1 Do You Believe the Evidence Favoring ESP Is Still Compelling?

So, we now have a finding and what we might call a “non-finding”—study results that do not show support. How do you interpret the evidence? Your position in this “debate”—or even your thoughts on whether a debate really exists at all—may have something to do with your response to the quiz you completed earlier in this activity. After you took the quiz, you were asked what you thought the quiz could tell us. You said:

[Student answer]

Most people have prior beliefs about the existence of ESP—they either believe it is possible or they believe it is impossible. Your beliefs likely guided whether you thought the quiz could tell you something or you just thought it was a gimmick.

Two different groups of twelve people stand looking at each other perhaps to indicate that one group believes in the existence of ESP while the other group does not.
Monalyn Gracia/age fotostock

3.5.2 The Power of Prior Positive Belief

If on the quiz you were open to the belief that you might have psychic powers, it is more likely that the Storm team’s meta-analysis results will remain compelling to you. It can be challenging to respond objectively to evidence that conflicts with our personal beliefs. Remember the psychics who were tested in the British study we discussed earlier? They interpreted the outcome of that test—their failure to show psychic abilities—in light of their own prior beliefs. After the test, psychic Kim Whitton said, “I know what I do is very real, it's easy for me. I'm glad one of the [women] could recognise so many details about herself” (Goldsmiths, University of London, 2012). Remember, one out of five guessing her reading correctly is exactly what you’d expect by chance. That’s like rolling a die repeatedly, focusing on the single time you roll a six, and saying “See how good I am at rolling a six?” But Whitton was not swayed by this evidence.

Thus photo has multiple playing dice that correlate with the evidence available for the existence of ESP.
Viktor Jarema/Shutterstock

3.5.3 The Power of Prior Negative Belief

But what if you were skeptical of what it meant to score a five or higher on the quiz you took earlier? Did you recognize that some items on the quiz are linked to explainable phenomena like déjà vu or coincidence? If so, then you will likely also accept that this weakened support serves to discount the existence of ESP. For the half of all Americans who don’t believe in ESP, the results from Rouder and colleagues (2013) suggest that some form of systematic error may be at the root of studies purporting to show ESP.

A photo of a young man with a skeptical facial expression perhaps indicating his disbelief over the existence of ESP.
PathDoc/Shutterstock

3.6 Consider the Source of the Research or Claim

4

Consider the Source of the Research or Claim

3.6.1 Science and Skepticism

Thinking like a scientist also involves thoughtful consideration of the source of information. In this case both sources are excellent—peer-reviewed articles published in scholarly journals. This means that both Storm’s team (2010) and Rouder’s (2013) permitted their research to be reviewed by a panel of their peers prior to publication. Further, both teams published detailed explanations of the methods used and the results obtained rather than simply a summary of their findings. However, even peer-reviewed studies published in premier research journals are not beyond scrutiny.

Earlier we talked about how your prior beliefs affected what you think about scientific research. What about the scientists’ beliefs? Could they have played some role in the outcome of the meta-analyses? For example, Rouder and his colleagues pointed out systematic errors in Storm and colleagues’ meta-analysis. Imagine that Storm and his colleagues were open to the existence of ESP. Might their prior beliefs have unconsciously led to some of the choices they made about which studies to include?

This photo shows a pile of journals that can be used as source of information for studies on ESP.
Christian Muller/Getty Images

3.6.2 A Challenge

Thinking like a scientist means being aware of your own beliefs and possible biases—whether you’re reading about science or conducting an experiment. Wherever your personal beliefs lie in any scientific debate, it’s always important to think like a scientist to ensure that you weigh equally the findings that support your beliefs as well as the findings that don’t.

If, after learning about the evidence, you believe you could still have psychic abilities, here’s a challenge. James Randi (pictured below), the magician known as “The Amazing Randi” and a self-described “scientific skeptic,” has funded an educational foundation that issued “The Million Dollar Challenge” (http://web.randi.org/the-million-dollar-challenge.html). If you can demonstrate that you are psychic, while following the foundation’s scientific protocols, you can win $1,000,000. No joke! This offer has been on the table since 1964, and hundreds have applied. So far, no one has won.

James Randi, whose educational foundation funds the Million Dollar Challenge.
Mykel Nicolaou/REX/Newscom

3.7 Assessment

Assessment

3.7.1 Assessment

Question

M/Htd0k0SpNhLfNtu+R6SdpUngxDBw5M6x7rGTQycc/Uo0JI9TXgqvRetcMrIOp/AAFQw+CLaG8vy0J5GaCQpnT5WrZ8axNyMsnvNx/8wS151BxVUlFm+nzCilHeG1p6u5u02/UHFhGnaNTvfmn5DHGOatmok5ok10443Itav3yrkOFozdWphyBAVtkwlylJAxE8ONdnytbF4I6arOuC7fuU6PJDmYuI754WWn6IuaiYPAcrUajExfWcTKhpQgmBgSPIB8PyMWDCjG8X67GV656+wVdsr6G+QKCMzR3x4bB7Pg6FdmksT3UY2jubCtxMMt7Q7afLyYLVfS6hnFes1kSnfIt+FbMnFA+TIzmr2M5XjHSYcZqKJAouQ6kfS8EY1ourieW/75uoLWWOXRO/e51nfPkcbb/qUlyanXnmFrzfPEqHOZtwUZRISJfqwYEFFSbBRMd+U/6Fl1S0KZVTXFEKnfcawQT95P5bwG9BwFlkWrnvB9vb5f1aHBmSdUlaKtiG+jZ/rmT5HH9vg14dlewsrFWj0Uu0wDZZh0PUSB1vLrtPuyZq5lYu5aF+uZkBxkiTD/rk6AixH81/cFMdr0m1GSmiv4n8Ww3+wdrgEksD5uOgqxe6iAISAHodo33cw3oEgF6968HujJjNVEhykNCoECq9XnlAm1fqTFpGGLuZC8yL+tu8uuLTx8K13Uiub40+1OXLEHHcgVT4ocjqYU6Rg0zdU8lX/aQZBeWxS2gTz3Mu0s//GzVPZIgvjCzztq9/j95uutdZJbjBP5H9H8027Mmeso3nREpdr5JaMMDhcJ0LdhuUSATEAnrgL3L1gAz1XSGJYbO6zVyTTAqg+qai/y5bKt0EVZnFIA==
The correct answer is C. It has been empirically shown that people who are open to the existence of ESP tend to be more accepting of scientific studies that appear to show evidence of paranormal abilities.

Question

i9c1Sv5DuJ0eyILEBJaSCoFgGFfI4H0kNmyxNzBU2FTXjAMpIrjScaTXe6SOzwTXLvAZvU+BzFxIlhposiyHMwCWC6OWFysIUZfMRbgWFlsRYai2rXhMtK7PxTKR0wDpCtEKgZU8QuO80P447LG9hICWflwEy+2wYDDKhFd6WMD5vnu51Gsd39os/O3K/LbdmFt8WuVIEW+n4dkoQkJ7q+iwMJ54pbqi1GNC2aqGAr0EN195nL01yDe9Pwlo4FwBiBQg5GNZmcgU1ncHuW2amW4IHfN+aCYjbYVf7B7nBolO//0gS6yQPgIZ5xFcT5wEv/5avnXtlux/VMJ4eS/Ke568Zm+kiJVr185H5fXWRRp5o8aRLPMWkJW2x9Ji6pw+Hr5LKpL4uTkAVXwK3WoMBsG7HuCuKEQit+Y+H/qclRpcMMhERc7aPNGhDOBrTkrHqG+F009WLLO0gVCtFCqcIStcMoK0JUC2Sqm68CeXt498zK8Zmm47XAIWP61lt1/o++uD2PuoFRkWQATIw5erYmAJ5XWhZY/bezrFG9cgyRhJ0KR7bR7ahI8JhXpJD0qnoFQnjZkbtzAWpZs4KrKAFWkAHtN9SuecCPUlDAqV4XzDQVDHGtidCFynmfqEhpt5+pnX7+61hmyfthnfB2/KVw54t/AkewYPnwvLVgq3zbi3mltGwyO6qMfq6RQdc2ckKqMcO9/VsgEqUtzTvgi0OcHFKYpvqu+b+Cdehry3ko7nRlvk22BS8m88HgzvWrkrFtfnlgO/w8yDKI89Ggbrkw==
The correct answer is A. Storm and his colleagues conducted a meta-analysis of ESP studies that, in the aggregate, showed that there was statistical support for a certain type of ESP.

Question

zXUVAiL7MUbCdQWWO/yv7ypYWrPDwExB+7QMbPqeRV9ySYzrbT8CXk6RBD5fhTIwRoKiomu1NXLgn1zCbBL7eUOc42opY46LyyQhpOpsELXA3uEyAxxIp0ksQLA7G9p2GwzqbwzGIEMvTdUraT4Yu+cBXYYwVT9XhSjyHQipTmK+V2T+XoBeyKgO7ZLbRq4+zencT3TRr0MiwkYRkG0LvzAFeIMpj9mmA6+yfoJzAJRHZ7sIEw1vcsbT8cWnNCmf7CI4PBdiXuesZP1EKW99PGJLYXEv5hIRGQeadTUuag5W5+kDZWUP+/H2Uu54hlqbALrv71Wq+625OrIwW8c2F6/jpLNhc6anzOxtasekuP8UETbI8zJFzkYH9ZM5mFowfkfL8XQygRaih2XLAucx/Ws+nwpYJV4ZGUZlLzhlfcr/IYbVveblufoQYmhnILKhatoKfxr3NvjP+w3b9NxeKRsFmJxjQC3zHvU0CPU1rwNHfJ2gLjlDuhkMbrAFMtl5ahXZUlGu+Eh1kOPFqGEmTU3N5gM3P1ZhEa1lzMnqrIwWhv9y1MlZwD6SDY6l1qskqsm5xqx70ygzl5wTXjqQcvC+/4aBl95VSMCYsyeO9VS+XpNZih4kfQgbUPmavRkWiEJ2HuHSdZM7X0bW+NpkN16rfrLK4oHwOHqoHRgQ8LGPoQ07SULJp3DOZLbHHoXwYPGn9MG5EEViEtHwHk/oJEJR098YfKiQwQxAyEFeYCvSYS2FCs+i0TiFhBnj2pLXuOfDiyOG4JAYKsfGHzl0LNYvoIVPei+fM740uwwdzAlfCIOLl+fxiIcope9yTIP+dUNVdFGun7juVJYe2V3Xiain32vLx/8DcIcKkSoMrv2QdiQ1vWiCWbNM+ohyW/C7navlY7W/wecncVUrp6KYB0lmbJGiKkQ1PW64wLZ7uN4VLW9ZNx/KIePCrhwOXMh3ZkzM1qnKnI6QZq5MX6+zHdIBQxg8p1lgCJI1hDdUx36PYs+eHEfcWthe8E74KYJP8VK8GMOP424=
The correct answer is B. The “file drawer problem” refers to the fact that research studies that show no statistically significant effect are less likely to be published or to be noticed if they are published.

Question

F3MwJD8CEe6SngAdf+jQBoeq0+qXxi8XlTAs9L9EAnVH2VM8YWmAB18/EK+KHoovF6HhMTKyfR1Adb9ZE03TxxZV97RDSh5Xdb+g6Y0goo69249OhDClH2dZ2RjpHuCKgx1gtLUbtiMC0NL5wO1Qz9eLh7jdnTWWeZaD+iwFKoO634zqmeWYeU840cjyJw8SaA9oJocyXuPji2AFj1HHiFiUyyNIt01JwClwIjbkkNxScXiq6X7m4XLYzVltfFN+gNDju+Dw7+4Or8nVLTGmFxVfhU89WMtEi5lgky+y+j4RJ3vccECxAu2m6tY7tDlAuzFf5MJ80TbzO18ZA8CtmR8QMtKuMWugB15C60xFW7kZ8b7NqXjZpv72PFKFFoZtgeFe+31m01Jb+F++nYIB3J24h/5Nqmvqfpvn+Bdmkw78cIx2p73MVxU69atDO9Te1tY6atG63G8+W52VFBSwV2NLqUEFdJFX8R1ZXu7xbuLFXTdWmR2T0AYUG4J2bBeGQoJyl1GkTqQI0CItSmgnPCONese5TO6A4peYuBnnfKVIZSvfdL5oADmUuQpAwDoqEl4IiiUFJlwWGH98+g2eegDg/mBKmzTbsXNc7pvYTIEVax69ZMBIoNYn92Y98jo3etQRUFyHuFuEUy2meLt6Cq8O43lmWoO8dVGmwjUtSBcUOyyidqteQ4GMQ/SWBaJ9wVA+RwOzKVec7G+gJzJZyO0nzebufb6QTasTmZiAl3vQutD2rooJZzjdECY2zWqubwjPrkZzEALFqQzpgIxXe+w1BAfbuz8UjJHHiaZMOL4psQc2OaNxUpwWHA3E385HIj0m9pscajgNJ8zKsY+jEe/EOr+hfdFa6VEICY9I17P58FLWQ+ie6InzLStVoeTS9RAl3XiHOVU=
The correct answer is B. Rouder and his colleagues identified several flaws in the studies analyzed by Storm and his colleagues and concluded that the evidence, overall, was much weaker than Storm and his colleagues had suggested.

Question

nFAqv0nzk2O/EY0ZY2rsJDN3qNIOYeqRV/O3K7gEa9JUz+z6tHn0vdcSpmiKmeU9au4UoP45e8uzpkp7ZiriMoQjy0hkpwgSywRJ0a80CT4trNtC52fhEVLmHotXO58APbccATtl6JXXjG3RyUqKNN3jrMzOJULdjmBNbpBOOswThczt3+V4INuiZ6dptUQKHBhhkiIlLBOjnUGtv8IPX85gXTO1F1suoCXLYTpYi8MLtQP4P530uIm8D5fRbmkIaA6LSiNYRr4N2boeuYBUq+35+VA/SwJme52Y6QqF9QQAsUgvpoeZ2xrtVewtUhOfLdT2zUSIEj2EgE0KsHSH8KZo75MEfBCnyp57s1rZMowuHjruwflco5tsMAXVuKZo3vDI+Hx+TbWpx2RdQHZWHAfENdI8kGtBV3/ZQmK7x/jmIi4OgIwslpdnb3Ixu7dI3uwPIcyVeFoOcDsjYVe8eyZazCQ3FVKlXyS2PB1tyH4gesHdT0Wfiz6KwPSljXljEPpoiy2N9tFhclapR5hIFrvk/LG6Eu3cEH4V2mQ5BUgeMfODYWVmu6e0aAIKY8BVFwzzej52foeHQ7iSokqsV7NliKzugBw9t3Zlkkq9dRUODawXRcqx28BQWEfjX6UKFMBxAl2WOKgqz1m21wY3utbzyqyFmqeCpe7KILPEbGHpGXvRIdvKw8s3HCMTH1aliNAI8y66I9Bmvg4chmbQf3+1aiYW+8FZKWdFYrfajnuKu/e6GOX3YYSQ9al+CKCTAZr06x/2c78+gx5vNEToMVFiaGnEv767fbqxJOUwU793ew0OGwDYR82zl6OLM1RvfqQc0Fz3r9V6nKB3o2/BiHZsfIi3W01/FKMwCINFkz7pCb+k
The correct answer is D. Rouder and his colleagues identified many problems with the ESP studies, but dishonesty and fraud were not among those problems.