1.17: Pseudoscience and misleading anecdotal evidence can obscure the truth.

One of the major benefits of the scientific way of thinking is that it can prevent you from being taken in or fooled by false claims. There are two types of “scientific evidence” that frequently are cited in the popular media and are responsible for people erroneously believing that links between two things exist, when in fact they do not.

• 1. Pseudoscience, in which individuals make scientific-sounding claims that are not supported by trustworthy, methodical scientific studies
• 2. Anecdotal observations, in which, based on just one or a few observations, people conclude that there is or is not a link between two things

Pseudoscience is all around us, particularly in the claims made on the packaging of consumer products and food (FIGURE 1-22). Beginning in the 1960s, for example, consumers encountered the assertion by the makers of a sugarless gum that “4 out of 5 dentists surveyed recommend sugarless gum for their patients who chew gum.” Maybe the statement is factually true, but the general relationship it implies may not be. How many dentists were surveyed? If the gum makers surveyed only five dentists, then the statement may not represent the proportion of all dentists who would make such a recommendation. And how were the dentists sampled? Were they at a shareholders meeting for the sugarless gum company? What alternatives were given—perhaps gargling with a tooth-destroying acid? You just don’t know. That’s what makes it pseudoscience.

Pseudoscience capitalizes on a belief shared by most people: that scientific thinking is a powerful method for learning about the world. The problem with pseudoscience is that the scientific bases for a scientific-sounding claim are not clear. The claims generally sound reasonable, and they are persuasive in convincing people to purchase one product over another. But one of the beauties of real science is that you never have to just take someone’s word about something. Rather, you are free to evaluate people’s research methods and results, and then decide for yourself whether their conclusions and claims are appropriate.

We are all familiar with anecdotal evidence. Striking stories that we read or hear, or our own experiences, can shape our views of cause and effect. We may find compelling parallels between suggestions made in horoscopes and events in our lives, or we may think that we have a lucky shirt, or we may be moved by a child whose cancer went into remission after eating apricot seeds. Yet, despite lacking a human face, data are more reliable than anecdotes, primarily because they can illustrate a broader range of observations, capturing the big picture.

Anecdotal observations can seem harmless and can be emotionally powerful. But because they do not include a sufficiently large and representative set of observations of the world, they can lead people to draw erroneous conclusions, often with disastrous consequences. One important case of anecdotal evidence being used to draw general conclusions involves autism, a developmental disorder that impairs social interaction and communication, and the vaccination for measles, mumps, and rubella (commonly called the MMR vaccine) that is given to most children.

In 1998, the prestigious medical journal The Lancet published a report by a group of researchers that described a set of symptoms (diarrhea, abdominal pain, bloating) related to bowel inflammation in 12 children who exhibited the symptoms of autism. The parents or physicians of 8 of the children in the study said that the behavioral symptoms of autism appeared shortly after the children received MMR vaccination. For this reason, the authors of the report recommended further study of a possible link between the MMR vaccine, the bowel problems, and autism.

In a press conference, one of the paper’s authors suggested a link between autism and the MMR vaccine, recommending single vaccines rather than the MMR triple vaccine until it could be proved that the MMR vaccine did not trigger autism. This statement caused a major health scare that only recently has begun to subside. Noting that there had been a significant increase in the incidence of autism in the 1990s and early 2000s, the press and many people took the claims made by the researcher at the press conference as evidence that the MMR vaccine causes autism. Over the course of the next few years, the number of children getting the MMR vaccine dropped significantly, as parents sought to reduce the risk of autism in their children (FIGURE 1-23).

Unfortunately, this is a notable case of poor implementation of the scientific method, with numerous flaws leading to an incorrect conclusion. Most important among these is that the study was small (only 12 children), the sample was carefully selected rather than randomized (that is, the researchers selected the study participants based on the symptoms they showed), and no control group was included for comparison (for example, children who had been vaccinated but did not exhibit autism symptoms). On top of this, the study, even flawed as it was, did not actually find or even report a link between autism and the MMR vaccine. That purported link came only from the unsupported, suggestive statements made by one of the study’s authors at the press conference.

On their own, the design flaws in the study would be sufficient to invalidate the one author’s claims about the link between autism and the MMR vaccine, but there are even more weaknesses in the research. It later was discovered that before the paper’s publication, the study’s lead author (the one who spoke at the press conference) had received large sums of money from lawyers seeking evidence to use in lawsuits against the MMR vaccine manufacturers. It also came to light that he had applied for a patent for a vaccine that was a rival to the most commonly used MMR vaccine. In the light of these initially undisclosed biases, 10 of the paper’s 12 authors published a retraction of their original interpretation of their results. And in 2011, the BMJ (British Medical Journal) published an article declaring that the “article linking MMR vaccine and autism was fraudulent.”

In the years since the original paper was published, many well-controlled, large-scale studies have conducted critical experiments on this subject, and all have been definitive in their conclusion that there is no link between the MMR vaccine and autism. Here are a few of these studies.

• 1. A study of all children born in Denmark between 1991 and 1998 found no difference in the incidence of autism among the 440,655 children who were vaccinated with the MMR vaccine and the 96,648 children who were not vaccinated.
• 2. A 2005 study in Japan showed that after use of the MMR vaccine was stopped in 1993, the incidence of autism continued to increase.
• 3. A study of 1.8 million children in Finland who were followed up for 14 years after getting the MMR vaccine found no link at all between the occurrence of autism and the vaccine.

At this point, the consensus of the international scientific community is that there is no scientific evidence for a link between the MMR vaccine and autism.

So what explains the observation that there are more autism cases now than in the past? It seems that this is a function of the better identification of autism by doctors and changes in the process by which autism is diagnosed. Another reason for the perceived link between autism and the MMR vaccine was simply the coincidence that most children receive the vaccine at around 18-19 months of age, which happens to be the age at which the first symptoms of autism are usually noticed. In the end, what we learn from this is that we must be wary that we do not generalize from anecdotal observations or let poorly designed studies obscure the truth.