Experimental Research

26

KEY THEME

The experimental method is used to demonstrate a cause-and-effect relationship between two variables.

KEY QUESTIONS

In this chapter, we’ve noted a number of factors that are associated with higher or lower college grades. But all of these factors—such as time spent sending texts and on Facebook—are correlational, meaning that while they are linked, they do not necessarily indicate that the two factors are causally related.

In contrast to descriptive research and correlational studies, experimental research is used to demonstrate a cause-and-effect relationship between changes in one variable and the effect that is produced on another variable. Conducting an experiment involves deliberately varying one factor, which is called the independent variable, sometimes called the treatment variable. The researcher then measures the changes, if any, that are produced in a second factor, called the dependent variable, also called the outcome variable. The dependent variable is so named because changes in it “depend on” variations in the independent variable.

To the greatest degree possible, all other conditions in the experiment are kept exactly the same for all participants. Thus, when the data are analyzed, any changes that are measured in the dependent variable can be attributed to the deliberate manipulation of the independent variable. In this way, an experiment can provide evidence of a cause-and-effect relationship between the independent and dependent variables.

In designing experiments, psychologists try to anticipate and control for confounding variables. Also called extraneous variables, these factors are not the focus of the experiment. However, confounding variables might produce inaccurate experimental results by influencing changes in the dependent variable. Confounding variables in a psychology experiment could include unwanted variability in such factors as age, gender, ethnic background, race, health, occupation, personal habits, education, and so on.

To illustrate how experimental research works, let’s look at a topic of interest to most college students: What types of study strategies are most effective?

In most educational settings, learning is thought to take place during study, instruction, and practice. Tests, in contrast, are neutral experiences and simply assess what has been learned. But some studies seemed to suggest that being tested on new information helped students learn and remember it better than simply studying it (see Roediger & Butler, 2011). Psychologists Henry Roediger and Jeffrey Karpicke (2006) set out to investigate the effects of testing on learning and memory.

Experimental Design

STUDYING THE EFFECTS OF TESTING

image
FIGURE 1.5 The Testing Effect Experimental Design
Commercial Eye/Getty Images

How could you design an experiment to show a difference between learning due to studying and learning due to testing? Roediger and Karpicke (2006) designed an experiment that compared the effects of repeated testing with the effects of repeated study periods. They predicted that students who repeatedly took tests after studying would have better long-term memory of the new information than students who repeatedly studied, but were not tested on, the same material. The hypothesis, then, was that “repeated testing improves learning more than repeated studying.”

27

The participants were 60 college undergraduates, aged 18 to 24. The researchers used random assignment to assign participants to one of two groups: either the experimental group or the control group. Random assignment means that all the participants have an equal chance of being assigned to any of the experimental conditions. Random assignment helps ensure that any potential differences among the participants are spread out evenly across all experimental conditions. Random assignment also helps minimize the possibility of bias because the same criteria are used to assign all participants to the different experimental conditions.

In any well-designed experiment, there is at least one control group. The control group serves as a baseline against which changes in the experimental group can be compared. In a typical experiment, the participants assigned to the control group go through all the experimental phases but are not exposed to the independent variable. Only the participants in the experimental group are exposed to the independent variable. In this study, the independent variable was repeated testing. The dependent variable was the score earned on a final test.

Here’s how the experiment was conducted. All of the participants were given a short prose passage to study. Participants in the control group read the passage for five minutes, and took a two-minute break. They then studied the passage again for five minutes before taking another two-minute break. They repeated this process for a total of four consecutive study periods. Note that this is the standard test-preparation method: to repeatedly study the same material until you feel certain that you have mastered it.

Participants in the experimental group were given the same prose passage to learn. They were also allotted five minutes to study the passage, and then took a two-minute break. But rather than restudying the passage, they took a test on the material: They were given a blank sheet of paper and were allowed ten minutes to write down as much information from the prose passage as they could remember. After another two-minute break, without studying the material again, they were given the same test, followed by a two-minute break. This procedure was followed for a total of one study period and three test periods. Figure 1.5 shows the setup of the experiment.

At the end of the session, all of the participants filled out a short questionnaire asking them to predict how well they would remember the material in a week. A week later, all participants were tested on the material.

28

MYTH SCIENCE

Is it true that reading something over and over is the most effective way to prepare for a test?

How do you think the two groups would compare on a test of their retention of the material a week later? Conventional wisdom would suggest that the control group members, who studied the material in four periods for a total of twenty minutes, would have learned the material much better than the participants in the experimental group, who, after all, had only studied the material for a total of five minutes.

Figure 1.6 shows the results on the test one week later. Despite having studied the material only one-fourth as long as the control group, the experimental group trounced the control group: They remembered 61% of the material in the passage while the control group remembered only 40%. Interestingly, the control group participants were much more confident of their ability to remember the information than the experimental group.

image
FIGURE 1.6 Effects of Testing on Retention: Experimental Results One week after the experimental sessions concluded, participants were tested to see how much they retained from material they had studied. As you can see, the participants who were repeatedly tested remembered much more of the information than the students who had repeatedly studied the same material.
Source: Data from Roediger & Karpicke, 2006.

Were you surprised by the results? In fact, the basic results have been replicated by many different researchers (see Bjork & others, 2013; Rowland, 2014). Multiple studies have supported what has been dubbed the testing effect: the finding that practicing retrieval of information from memory produces better retention than restudying the same information for an equivalent amount of time. In other words, testing—rather than simply being a neutral assessment of what has been learned—is a powerful learning tool in its own right (Carpenter, 2012; Roediger & Nestojiko, 2015).

We’ll discuss some of the reasons for the testing effect in Chapter 6, Memory. In Psych for Your Life at the end of this chapter, we’ll describe some of the other ways in which the testing effect has been explored. And, we’ll discuss additional ways in which you can use psychological research to improve your own memory for new information.

Experimental Controls

image
Can Ginkgo Biloba Enhance Your Mental Abilities? The herbal supplement ginkgo biloba is marketed as a “cognitive enhancer” that supposedly improves memory, alertness, and concentration, especially in older adults. However, studies of ginkgo don’t support those claims (see Canter & Ernst, 2007; Daffner, 2010; Snitz & others, 2009).
Joe Raedle/Getty Images

Some experiments involve extra controls to increase the reliability of their findings. One important safeguard is the double-blind technique, which is often used when researchers are testing the effectiveness of a procedure or drug treatment. In a double-blind study, both the participants and the researchers interacting with them are blind, or unaware of the treatment or condition to which the participants have been assigned.

Using a double-blind technique helps guard against the possibility that the researcher inadvertently becomes an extraneous or confounding variable in the study. This can happen when a researcher, without realizing it, displays demand characteristics. These are subtle cues or signals that can bias the outcome of the study by communicating the behavior or response that is expected of the participants. A behavior as subtle as the researcher slightly smiling or frowning when dealing with some participants, but not others, could bias the outcome of a study.

29

Such studies also often involve the use of a placebo, which is a so-called “sugar pill” or other inactive substance or procedure. Although it is inactive, a placebo can produce real effects (see Wager & Atlas, 2015). A placebo effect is any change that can be attributed to beliefs and expectations rather than to an actual drug, treatment, or procedure.

For example, one student in our class asked us whether we believed that the herb ginkgo biloba could improve memory. To test that notion, psychologist Paul Solomon and his colleagues (2002) used a placebo in a double-blind study to test whether ginkgo biloba improved memory and concentration in older adults. Participants in the experimental group took the manufacturer’s recommended daily dosage of ginkgo biloba for six weeks, while those in the control group took an identical dosage of placebo capsules.

The researchers who interacted with the participants did not know which participants received the real and which received the fake ginkgo biloba. The researchers who did know the group assignments did not interact with or evaluate the participants. Memory and other cognitive abilities were assessed at the beginning and end of the six-week study.

Can you predict the results of the ginkgo biloba experiment? At the end of the six-week study, the test scores of both groups rose. However, there were no significant differences between the improvement in the ginkgo biloba and placebo groups. So why did both groups improve? The researchers concluded that it was probably due to the practice effect. The participants’ experience with the tests—the practice they got by simply taking the mental ability tests twice—was the most likely reason that test scores improved in both groups. This experiment illustrates the importance of the control group: Without a control group to compare, the improvement in the experimental group might have been attributed to the drug. But since the control group participants also improved, there must have been another explanation.

Limitations of Experiments and Variations in Experimental Design

A well-designed and carefully executed experiment can provide convincing evidence of a cause-and-effect relationship between the independent and dependent variables. But experiments do have limitations. Because experiments are often conducted in highly controlled laboratory situations, they are sometimes criticized for having little to do with actual behavior. That is, the results may not generalize well, meaning that the results cannot be applied to real-world situations or to more general populations beyond the participants in the study. To minimize this, experiments are sometimes carried out in natural settings, rather than in a laboratory. A second potential limitation is that the phenomena the researchers want to study may be impossible or unethical to control experimentally.

But researchers are sometimes able to take advantage of naturally occurring events or conditions. In a natural experiment, researchers carefully observe and measure the impact of a naturally occurring event or condition on their study participants (Rutter, 2008). Although not true experiments, psychologists can use natural experiments to study the effects of disasters, epidemics, or other events.

30

image
Using a Natural Experiment to Study the “Freshman Fifteen” Research shows that many college freshman do gain weight, although it is typically closer to five pounds than fifteen (Holm-Denoma & others, 2008). Kandice Kapinos and Olga Yakusheva (2011) used a natural experiment to study the effect of environmental factors on weight gain in college freshmen. Not surprisingly, freshman students who lived in a dorm with an on-site snack bar or cafeteria were more likely to eat more and gain weight than freshmen who had to walk across campus for meals and snacks.
Suzanne DeChillo/The New York Times/Redux

Sometimes natural experiments involve everyday settings. Psychologists Kandice Kapinos and Olga Yakusheva (2011) were interested in better understanding the relationship between environmental factors and weight gain. Previous research had shown that environmental factors, such as high concentrations of fast-food restaurants, are correlated with the average weight of nearby residents (Inagami & others, 2009). However, since the research was correlational, it wasn’t possible to conclude that proximity of fast-food restaurants caused weight gain. It could be that people with unhealthy eating habits were more likely to choose to live in neighborhoods with easy access to fast food.

image

Think Like a SCIENTIST

Could you have been part of an experiment without realizing it? Go to LaunchPad: Resources to Think Like a Scientist about Contagious Online Emotions.

image

Of course, researchers can’t randomly assign large numbers of participants to long-term living situations. But Kapinos and Yakusheva identified a naturally occurring situation in which people are randomly assigned to housing—college dormitories.

College freshmen do tend to gain weight during their first year away from home—the so-called “freshman fifteen”—although the weight gain is typically closer to five pounds (Holm-Denoma & others, 2008). Kapinos and Yakusheva took advantage of the naturally occurring conditions on their own college campus by comparing weight changes in freshmen who lived in dorms with on-site cafeterias and snack bars with weight changes in freshman who lived in dorms that did not have on-site food services. Since freshmen were randomly assigned to the dormitories, the researchers could safely assume that there wasn’t some other factor that might cause differences between the two groups.

What did Kapinos and Yakusheva (2011) find? Female students who were assigned to dormitories with on-site dining facilities gained more weight and exercised less than students who were assigned to dormitories without food services. Male students assigned to dormitories with food services reported eating more meals and more snacks, but did not report gaining weight. Kapinos and Yakusheva (2011) concluded that campus design did play a significant role in influencing healthy—and unhealthy—behaviors in college students.

Before leaving the topic of research methods, one contemporary trend deserves special mention: the increasing use of brain-imaging techniques in virtually every area of psychology. To help highlight the importance of neuroscience, every chapter includes a special “Focus on Neuroscience” feature. This chapter’s Focus on Neuroscience explores brain-imaging techniques and discusses their increasing use in psychological research (see pp. 34–35).

This brief introduction to research methods will give you some idea of how psychologists conduct research. But we hope it also illustrates some of the ways in which scientists—and others—evaluate claims and evidence. In the Critical Thinking box “Think Like a Scientist,” we offer several suggestions to help you evaluate claims that you encounter both inside and outside the classroom.

31

CRITICAL THINKING

How to Think Like a Scientist

Do violent video games make people aggressive? Are some people “right-brained” and others “left-brained”? What’s the best way to lose weight? Do “educational” DVDs and TV shows improve children’s language abilities or help them learn how to read?

How can you evaluate the claims you encounter? Both in class and out, it’s important to engage in critical thinking, actively questioning statements rather than blindly accepting them.

Critical thinkers are open to new information, ideas, and claims. However, this open-mindedness is tempered by a healthy sense of skepticism (J. C. Smith, 2010). The critical thinker consistently asks, “What evidence supports this claim?”

In this chapter, we’ve detailed the ways that psychologists conduct research, including the different ways they test hypotheses. You can think of the claims you encounter in print or in online media, on TV shows, or in conversation as hypotheses, too. In other words, when you encounter an idea or statement that is presented as factual, try to think like a scientist.

Like a scientist, you can follow these four steps to determine the validity of a particular claim:

  1. Identify the Claim

    Some claims are so vague that they are impossible to be tested scientifically. For example, take the statement that “you use only 10% of your brain.” Superficially, it sounds convincing, but can you imagine an experiment that would actually test this claim? Try to restate the claim in terms of a hypothesis that could be supported or disproved by empirical evidence. How would you define the variables that could be objectively measured?

  2. Evaluate the Evidence

    As you have learned, the scientific method includes key safeguards in experimental design, such as random assignment, the presence of a control group, and researchers who are “blind” to participants’ conditions. So when evidence is offered in support of a particular position, scrutinize it and look for those basic safeguards.

    MYTH SCIENCE

    Is it true that when two behaviors are “linked,” “related,” or tend to occur together it’s safe to assume that one behavior caused the other?

    Consider also the nature of the evidence that may be offered. When words like “link,” “tie,” or “association” are used, the evidence is probably correlational, rather than experimental. But remember the distinction between correlation and causation. As you have learned, just because two events are correlated does not mean that they are causally linked. For example, consider a recent study that found a positive correlation between the number of outdoor signs and billboards advertising food and non-alcoholic beverages and the rate of obesity in particular urban neighborhoods (Lesser & others, 2013). As the number of outdoor food ads increased, so did the obesity rate. One Internet headline read, “Billboards Make You Fat!” Some commentators advised that policy makers consider restricting outdoor food advertising as a way of reducing obesity in urban areas.

    But is such a conclusion justified? No. You cannot conclude that the correlation found between obesity and advertisements occurred because the higher prevalence of advertising caused higher rates of obesity. The evidence does not support that conclusion. In fact, it could just as likely be that advertisers are more likely to place ads for food in areas where they believe there are higher numbers of obese people (Chabris & Simons, 2013).

    Similarly, remember that testimonials are not evidence (Coltheart & McArthur, 2012). Distinguish between empirical evidence that can be objectively observed, measured, and shared—and private opinions, based on feelings or personal experience. For example, the fact that a relative’s child had a terrible experience in day care is not evidence that day care is generally bad for all children.

  3. Consider Alternative Explanations

    Especially if a claim is highly unusual, seems to contradict accepted scientific theories, or has no plausible explanation, consider alternative explanations. A claim demonstrating improvement in a condition or skill could, in fact, have many different explanations. For example, suppose a friend’s cold disappeared after he took a special herbal supplement recommended by another friend. The improvement could be due to the herbs. But it could also be due to the placebo effect (page 29) or to the natural healing that often takes place with the simple passage of time. A child’s improvement in paying attention in class might be due to the new, sugar-restricted diet she started. But it could also be the result of normal maturation, or the extra attention she received from her teacher after her parents expressed their concerns.

  4. Consider the Source of the Research or Claim

    Typically, scientific research is reported in a peer-reviewed scientific journal or at an academic conference before the results are released to the media. So, when research is reported in the popular media or on the Internet, consider the source of the research. While publication in a scientific journal is no guarantee that the results will prove valid over time, you can at least be certain that the research has been carefully evaluated by other scientists in the field. When a claim has no apparent ties to legitimate educational or scientific enterprises, you should be especially cautious. In general, it’s worth considering the researchers’ motives. If the people or company making the claim have the potential to profit from its use, the source may not be objective.

    Whether the claims you encounter come from friends, instructors, or pundits, remember these four steps—and think like a scientist before blindly accepting them.

CRITICAL THINKING QUESTIONS

  • Why might other people want to discourage you from thinking critically?

  • In what situations is it probably most difficult or challenging for you to exercise critical thinking skills? Why?

  • What can you do or say to encourage others to use critical thinking in evaluating questionable claims or assertions?

32

FOCUS ON NEUROSCIENCE

Psychological Research Using Brain Imaging

Brain-scan images have become so commonplace in news articles and popular magazines that it’s easy to forget just how revolutionary brain-imaging technology has been in the field of psychology (Mather & others, 2013a, b). Here, we’ll look at three commonly used brain-imaging techniques and examine how they’re used in psychological research.

Positron emission tomography, abbreviated PET, is based on the fact that increased activity in a particular brain region is associated with increased blood flow and energy consumption. A small amount of radioactively tagged glucose, oxygen, or other substance is injected into the person’s bloodstream. Then, the person lies in a PET scanner while performing some mental task. For several minutes, the PET scanner tracks the amounts of radioactive substance used in thousands of different brain regions. A computer analyzes the data, producing color-coded images of the brain’s activity.

image
Positron Emission Tomography (PET) PET scans provide color-coded images of the brain’s activity. This example shows the comparison between subjects learning a new language task (left) and performing the language task after it has been well learned (right). Red and yellow colors highlight areas with the highest level of activity, while green and blue colors indicate lower levels of brain activity. As you can see, the process of practicing and learning a new language task involves more and different brain areas before becoming established.
Dr. Marcus E. Raichle, Professor of Radiology & Neurology/Washington
University School of Medicine

Magnetic resonance imaging (MRI) does not involve invasive procedures such as injections of radioactive substances. Instead, while the person lies inside a magnetic tube, powerful but harmless magnetic fields bombard the brain. A computer analyzes the electromagnetic signals generated by brain-tissue molecules in response to the magnetic fields. The result is a series of digital images, each a detailed “slice” of the brain’s structures. MRI scans are routinely used to produce detailed images of other body parts, such as joints, spine, or organs.

image
Magnetic Resonance Imaging (MRI) MRI scans produce a highly detailed image of the brain, showing “slices” of the brain from different angles. The yellow dots highlight the brain region that was significantly larger in experienced London taxi drivers, known for their encyclopedic memory of London streets, as compared to control subjects (Maguire & others, 2000, 2006). This region, called the hippocampus, is known to be involved in forming new memories. This landmark study provided solid evidence for the once-revolutionary idea that structures in the adult brain change in response to experience and learning.
Hippocampus, Vol. 16, 2006, p. 1097. Reprinted with permission of Wiley-Liss, Inc., a subsidiary of John Wiley & Sons, Inc.

Functional MRI (fMRI) combines the ability to produce a detailed image of the brain’s structures with the capacity to track the brain’s activity or functioning (K. Smith, 2012). While the person lies in the MRI scanner, a powerful computer tracks the electromagnetic signals that are generated by changes in the brain’s metabolic activity, such as increased blood flow to a particular brain region. By measuring the ebb and flow of oxygenated blood in the brain, an fMRI produces a series of scans that show detailed moment-by-moment “movies” of the brain’s changing activity in specific structures or regions.

In the study of brain activity, fMRI has several advantages over PET scan technology. Because fMRI is a noninvasive procedure and the magnetic waves are harmless, research participants can safely undergo repeated fMRI scans. Also, fMRI produces a much sharper image than PET scans and can detail much smaller brain structures. Another advantage of fMRI is that it provides a picture of brain activity averaged over seconds rather than the several minutes required by PET scans.

image
Functional Magnetic Resonance Imaging (fMRI) fMRI combines highly detailed images of brain structures with moment-by-moment tracking of brain activity. Here, fMRI was used to record the brain activity of a 23-year-old woman who had been unresponsive to external stimuli for five months following an auto accident (Owen & others, 2006). Researchers asked her to first imagine playing tennis, and then to imagine walking through her house. The scans above compare her brain activity to that of normal volunteers (“controls”) performing the same tasks. In both the patient and the controls, regions known to be involved in movement and spatial navigation were active. The fMRI scans confirmed that the patient was conscious of her surroundings and able to respond to spoken commands.
Owen, A.M., et al. “Detecting awareness in the vegetative state.”
Science 313 (2006):1402. Reprinted with permission from AAAS

How Psychologists Use Brain-Imaging Technology

Brain imaging is used for both descriptive and experimental research. A descriptive study using brain scans might compare the brain structure or functioning of one carefully defined group of people with another.

For example, MRI scans were used to compare London taxi drivers with matched participants who were not taxi drivers (Maguire & others, 2000, 2006). In order to be licensed, London taxi drivers are required to have an encyclopedic knowledge of the city streets. The MRI scans showed that a brain structure involved in spatial memory, the hippocampus, was significantly larger in the experienced taxi drivers than in the control subjects (see MRI scans on the next page). And, the size of the hippocampus was also positively correlated with the length of time the participants had been driving taxis in London: The longer the individual had been driving a taxi, the larger the hippocampus (Woollett & others, 2009). In Chapter 2, Neuroscience and Behavior, we’ll look at how the brain changes in response to learning and environmental influences.

Brain-imaging technology can also be used in experimental research, such as a study of the effects of sleep deprivation that we’ll discuss in Chapter 4 (see pages 146–147). In a typical experiment, brain scans are taken while research participants are exposed to the experimental treatment or task. These scans are compared to scans taken of control group participants. The differences between the two sets of scans are assumed to be due to the experimental treatment or condition. When multiple participants are compared, researchers combine results to produce a composite scan showing the average differences among the experimental groups.

Limitations of Brain-Imaging Studies

Images are becoming even more detailed as brain-imaging technology advances. Nevertheless, brain-imaging research has several limitations (Miller, 2010; Satel & Lilienfeld, 2013). When you consider the results of brain-imaging studies, including those presented in this textbook, keep the following points in mind:

33

  1. Brain-imaging studies usually involve a small number of subjects. Because of the limited availability and the high cost of the technology, many brain-imaging studies have fewer than a dozen participants. With any research involving a small number of participants, caution must be exercised in generalizing results to a wider population (Button & others, 2013).

  2. MYTH SCIENCE

    Is it true that brain scans can pinpoint the exact, single part of the brain that causes a complex behavior?

    Brain imaging studies tend to focus on simple aspects of behavior. Even seemingly simple tasks involve the smooth coordination of multiple brain regions. As Jerome Kagan (2008) observes, “An event as simple as the unexpected sound of a whistle activates 24 different brain areas.” Thus, it’s naïve to think that complex psychological or behavioral functions can be mapped to a single brain center (Coltheart, 2013; Mather & others, 2013b).

  3. Brain imaging may not increase understanding of a psychological process. For example, although brain imaging might point to a particular brain structure as being involved in, say, fear or romantic love, knowing this may not advance our understanding of the psychological experience of fear or romantic love (Decety & Cacioppo, 2010).

  4. Brain imaging is not necessarily a more “scientific” explanation. As psychologist Paul Bloom (2006) points out, “Functional MRI seems more like ‘real’ science than many of the other things that psychologists are up to. It has all the trappings of work with great laboratory credibility: big, expensive, and potentially dangerous machines, hospitals and medical centers, and a lot of people in white coats.” To be truly useful, brain-activity snapshots of a particular behavior must be accurately interpreted within the context of existing psychological knowledge about the behavior (Beck, 2010; Kihlstrom, 2010).

Looking at Brain-Scan Images

What should you notice when you look at the brain-scan images in this text? First, read the text description so you understand the task or condition being measured. Second, read the brain-scan caption for specific details or areas to notice. Third, carefully compare the treatment scan with the control scan if both are shown. Fourth, keep the limitations of brain-scan technology in mind. Finally, remember that human experience is much too complex to be captured by a single snapshot of brain activity (Miller, 2010).