Classical conditioning is just one form of learning. A second is operant conditioning, in which behavior is modified by its consequences.

The core idea behind operant conditioning is that the consequences that follow any bit of behavior alter the likelihood that the behavior will be performed in the future. This, in principle, is true for virtually any behavior by any organism. For instance, if your dog sits up and there is a positive consequence (a dog treat), it will be more likely to sit up in the future. If you party the night before an exam and there is a negative consequence (you fail), you will be less likely to party pre-exam in the future. The environmental consequences shape future behavior. This is the essence of operant conditioning.

Examples of operant conditioning are abundant in everyday life. Suppose that you see a parent arguing with a small child and eventually giving in to the child’s demands. You may realize that the child, as a result, is more likely to argue in the future. The parent’s giving in is a positive consequence that raises the likelihood of the child repeating the behavior.

Alternatively, imagine you have a friend you’d like to know better. You hope the person will “open up” by talking more about personal thoughts and feelings. You know that, for this to occur, you need to pay attention and display interest whenever your friend shows the slightest sign of discussing personal topics. Your attention and interest are consequences that raise the likelihood that he or she will continue to open up. If, instead, you yawn and look at your watch, these consequences will make that behavior less likely.

Consequences alter behavior. The challenge for psychological science is to understand the exact processes through which learning from consequences occurs.

280

Operant conditioning and classical conditioning differ in two main ways: (1) the order of the stimulus and the behavior, and (2) the type of behavior that is learned.

Operant conditioning research, then, greatly expands the scope of learning processes beyond those studied by Pavlov. Let’s now look, in detail, at psychological processes in operant conditioning.

Research on operant conditioning has a long history in psychology. It can be traced to work by an American psychologist named Edward L. Thorndike (1874–1949).

281

At about the same time that Pavlov began to study classical conditioning in Russia, Thorndike started experiments on a different type of learning at Columbia University in New York. He placed cats in puzzle boxes, enclosures the cat could escape from by performing a relatively simple behavior (Figure 7.9). For example, the puzzle box might have a door to which a string was attached; the cat could escape by pulling the string, which opened the door. To provide the cats with an incentive to escape, Thorndike put them in the puzzle boxes when they were hungry and placed some food outside the puzzle box (Chance, 1999).

TRIAL-BY-TRIAL LEARNING. Once the cat figured out how to escape, the experiment was not over. Thorndike would put the cat back in the puzzle box, from which it would have to escape once again. This would happen repeatedly. On each trial—that is, each return to the puzzle box—Thorndike recorded how long it took the cat to figure out how to escape.

Thorndike discovered that the cats gradually escaped more quickly (Figure 7.10). Initially, it would take them a couple of minutes. They would waste time on behaviors that didn’t help them to get out of the box, such as clawing at the wall. But after a few trials, their behavior became more focused. They spent almost all their time performing actions (such as trying to pull the string connected to the door) that, on previous trials, had enabled them to escape. Eventually, they escaped the puzzle box in just a matter of seconds.

Thorndike found, then, that the cats rapidly learned. Their behavior changed as a result of their experience in the puzzle box. Over time, they learned to perform only those behaviors that, in the past, had been followed by successful escape.

282

THE LAW OF EFFECT. Thorndike realized that his discovery did not pertain merely to cats in boxes. He had discovered a general principle of learning that he called the law of effect. The law describes relations among (1) the situation the animal is in, (2) a behavior it performs, and (3) an outcome that follows the behavior. The law of effect states that when an organism performs a behavior that leads to a satisfying outcome in a given situation (i.e., an outcome that is pleasurable or beneficial), it will be more likely to perform that behavior when it encounters the same situation in the future. The likelihood of the behavior, then, goes up when the response consequence—the event that follows the behavior—is positive.

The law of effect explains the cat’s behavior. Each time the cat escapes the box, this satisfying outcome strengthens the connection between the stimulus (the puzzle box with its door closed) and the behavior that enabled the cat to escape (pulling the string attached to the door). When placed back in the puzzle box, the cat is thus likely to perform the behavior again.

After Thorndike, many psychologists explored how animals learn from experience (Bower & Hilgard, 1981); indeed, the study of learning came to dominate American psychology from the 1920s through the early 1960s. Among these many researchers, one became the most eminent psychologist of the twentieth century: B. F. Skinner (Haggbloom et al., 2002).

Skinner (1904–1990) began his career in psychology as a student at Harvard University, where he later served on the faculty. As a student, he developed a new research method (detailed below) for studying how environmental events shape behavior (Skinner, 1938). He continued this basic research on learning throughout his career, while also writing scientific books (Skinner, 1953, 1974) and even a novel, Walden II (Skinner, 1948), that explained to the public the principles of learning he had discovered and their implications for society.

CONSEQUENCES SHAPE BEHAVIOR. Skinner’s central insight was the same as Thorndike’s: Behavior is shaped by its consequences. When you do something, a consequence usually follows. Use a GPS, and you find the place you’re looking for. Tell dirty jokes to socially conservative friends, and you find they are not amused. These consequences not only influence your emotional reactions, as Pavlov noted; they also determine whether you perform these behaviors again in the future. Thanks to the positive consequence, finding your desired location, you start using your GPS more. Due to the negative consequence, the displeasure of your conservative friends, you tell them dirty jokes less often.

Although Skinner’s insight was the same as Thorndike’s, his overall contributions were far broader. Skinner not only generated experimental results showing that consequences influence behavior, but he also developed a new technology for conducting research on these influences (see Research Toolkit) that was widely adopted in psychology laboratories around the world. Furthermore, he advanced an entire school of thought, called behaviorism, which argues that all actions and experiences of all organisms must be explained by referring to the environmental influences that a given organism has encountered (Skinner, 1974). Skinner forcefully argued that behaviorism and the principles of operant conditioning could explain most of the phenomena in the field of psychology because most of psychology is concerned with behaviors that are followed by response consequences.

283

Central to Skinner’s theoretical system was his analysis of how two types of response consequences, reinforcers and punishments, shape behavior. Let’s examine these now.

REINFORCEMENT. In operant conditioning, the stimulus of central interest is called a reinforcer. A reinforcer is any stimulus that occurs after a response and raises the future probability of that response. Skinner’s research analyzed how reinforcers raise the probability that a reinforced behavior will occur again, thereby shaping behavior.

Skinner identified two types of reinforcement—positive and negative (Table 7.1). In positive reinforcement, the occurrence of a stimulus increases the likelihood that a given type of behavior occurs. If you diet, exercise, and lose weight and then friends say, “Wow, you look great!” their comments may serve as positive reinforcements that increase the future likelihood of you dieting and exercising. In negative reinforcement, the removal of a stimulus increases the likelihood that a given type of behavior occurs. If you take two aspirin for a headache and your headache goes away, then the removal of the headache is a negative reinforcer; its removal raises the probability that you’ll take aspirin the next time you have a headache.

Here’s another example; see if you can tell whether it’s a positive or negative reinforcer. Imagine that you have a neighbor who plays music too loudly. You yell, “Quiet!” and the neighbor turns the music off. What kind of reinforcer was the cessation of music? It was a negative reinforcer that makes it more likely that, the next time the music is too loud, you will yell “quiet” again.

Notice that “reinforcer” is defined in terms of the effects of a stimulus on the probability of behavior, not in terms of whether people like, or enjoy, the stimulus. Some stimuli that people do not like are effective positive reinforcers. Suppose you live in a region that periodically is struck with natural disasters (e.g., hurricanes or earthquakes). You don’t like the disasters, yet they are positive reinforcers. Their occasional occurrence powerfully raises the probability of some behaviors, such as purchasing electrical generators and stocking up on emergency water supplies.

284

PUNISHMENT. Skinner recognized that behavior is affected by not only reinforcement but also punishment. A punishment is a stimulus that lowers the future likelihood of behavior. Scolding a misbehaving child, ticketing a speeding driver, and booing an athlete who fails to hustle are all cases of punishment; these response consequences tend to lower the likelihood of the behavior they follow.

As with reinforcement, one can identify “positive” and “negative” versions of punishment. Positive punishment is the presentation of an aversive stimulus, and negative punishment is the taking away of a desired stimulus. Table 7.1 summarizes the two types of reinforcement and punishment.

An everyday setting that illustrates the distinction between positive and negative punishment occurs in child care. When a child misbehaves, a caretaker could opt for positive punishment by presenting an aversive stimulus (e.g., scolding the child). There is, however, a popular negative-punishment alternative: the time-out. As you might remember from your own childhood, children in time-out are socially isolated. They lose access to desired stimuli such as toys and playmates. Research indicates that this negative punishment can effectively increase children’s compliance with adults’ requests (Everett et al., 2007).

Punishment can get results. Yet Skinner devoted almost all his research efforts to the study of reinforcement rather than punishment. He did so for two reasons, involving effectiveness and ethics. While punishment works, he regarded it as an ineffective way to change behavior because people may act against someone who is punishing them. Second, Skinner judged that rewarding desirable behavior is ethically superior to punishing undesirable action (e.g., Skinner 1948). Let’s follow Skinner’s lead and focus in detail on reinforcement processes.

285

Skinner discovered a number of systematic, and sometimes surprising, relations between reinforcement and behavior. As you’ll see in this section, the discoveries involved four basic operant conditioning processes: schedules of reinforcement, extinction, discriminative stimuli, and the shaping of complex behavior.

Skinner believed that his research on each process was highly relevant to human behavior (Skinner, 1953). Note, however, that his studies were conducted on animals. Why animals? According to Skinner, just as principles of physics apply to all physical objects, principles of learning apply to all organisms. Thus, for ease of conducting research, he studied relatively simple organisms (especially pigeons and rats) in a relatively simple experimental setting, a Skinner box (see Research Toolkit).

RESEARCH TOOLKIT

287

SCHEDULES OF REINFORCEMENT. In both the laboratory and everyday life, the relation between behaviors and reinforcers varies. Sometimes reinforcers occur consistently and predictably. Every time you pay the cashier at a grocery store, the behavior is reinforced (you get a bag of food). Other times, however, reinforcers are sporadic and unpredictable. You always root, root, root for your home team, but often the behavior is not reinforced (sometimes they lose).

Skinner recognized that different schedules of reinforcement exist. A schedule of reinforcement is a timetable that indicates when reinforcers occur, in relation to the occurrence of behavior. Different schedules of reinforcement produce different patterns of behavior (Ferster & Skinner, 1957). Schedules of reinforcement differ in two ways: (1) the consistency of the reinforcement schedule and (2) whether time is a factor in the delivery of reinforcers (see Figure 7.12).

Regarding consistency, sometimes the schedule with which reinforcers are delivered remains the same, or is “fixed,” from one trial to the next. (A trial in Skinner’s experiments is any sequence in which a behavior or series of behaviors is followed by a reinforcer.) In fixed schedules of reinforcement, the delivery of reinforcers is consistent across trials. In variable schedules of reinforcement, the delivery of reinforcers is inconsistent; it changes unpredictably from one trial to the next.

Regarding time, sometimes an organism must wait for a period of time before a reinforcer is delivered. In interval schedules of reinforcement, a reinforcer is delivered subsequent to the first response the organism makes after a specific period of time elapses. There is nothing the organism can do to make the reinforcement occur more quickly. In another pattern, however, time is not a determining factor in the scheduling of reinforcements. In ratio schedules of reinforcement, the organism gets a reinforcer after performing a certain number of responses. If it responds faster, it is reinforced sooner.

As shown in Figure 7.12, these two distinctions combine to create four types of reinforcement schedules: fixed interval (FI), variable interval (VI), fixed ratio (FR), and variable ratio (VR). Let’s look at each one, examining both Skinner’s laboratory results and their relevance to everyday life.

In a fixed interval (FI) schedule, reinforcers occur after a consistent period of time has elapsed from one trial to the next. For example, suppose a rat is pressing a lever under an FI schedule, and the interval is 1 minute. This means that, once the rat presses the lever and is given a reinforcer, it cannot get another one for 60 seconds. After the full minute elapses, the first time the rat presses the lever, it receives another reinforcer.

FI schedules produce relatively low rates of behavior (Figure 7.13). Once a reinforcer is presented, an animal tends not to respond for awhile. The animal generally exhibits high rates of response only in the few seconds before the end of the interval. As a result, the animal’s overall level of behavior is low in comparison to that of animals reinforced under any of the other three reinforcement schedules.

FI schedules are common in everyday life. One example of a reinforcer that often occurs on an FI schedule is classroom exams; you might be in a class that has exams on a fixed schedule, maybe once every four weeks. (Exams are reinforcers; they reinforce studying. If there were no exams, most people would study less than they do in classes with exams.) So exams in your class are reinforcers that occur on a predictable, fixed schedule. What, then, would Skinner’s theory predict about the behavior of students in a course with regularly scheduled exams?

288

Because the exams occur on an FI schedule, Skinner would predict that students’ frequency of studying for the course would vary across time, just like the behavior of a rat in a Skinner box (see Figure 7.13). He would expect that, after one exam occurs, students would not do much studying for awhile. Rather, they would exhibit high rates of studying only in the few days and hours just prior to the next exam. Of course, for many students, this is exactly what occurs—it’s called cramming. Skinner would say that there’s nothing unique about this; it has nothing to do, specifically, with exams, or students, or even human beings. Cramming for a test is simply one example of the general pattern of behavior that occurs under an FI schedule of reinforcement.

In a variable interval (VI) schedule, reinforcements occur after a period of time that is unpredictable to the organism in the Skinner box. For example, the interval might be set to 1 minute on average, but could vary from 10 seconds to 110 seconds from one trial to the next. VI schedules produce patterns of behavior that occur at a higher rate, and more consistently, than behavior under an FI schedule (see Figure 7.13). A simple change in our cramming example above shows how this works in everyday life. Suppose you had pop quizzes in a course, instead of scheduled exams. You likely would study at a relatively consistent rate, always keeping up on readings and reviewing lecture notes, so that you’d always be ready for an exam. This type of consistency in behavior is exactly what occurs in Skinner boxes, under VI schedules.

The ratio schedules of reinforcement yield higher rates of response than either of the interval schedules (see Figure 7.13). In the fixed ratio (FR) schedule, a reinforcer is delivered after an organism performs a certain number of responses, and that number stays consistent from one trial to the next. For instance, a rat might be reinforced after every 20th lever press. With an FR schedule, the reinforcement occurs after a certain number of responses no matter how long, or short, a period of time the animal takes to perform these responses. Under an FR schedule, organisms get more reinforcements if they respond more quickly (unlike the FI schedule).

To see the power of an FR schedule, imagine two types of jobs: one in which you get paid by the hour, and the other—the FR job—in which you get paid more if you speedily accomplish more work. You would get paid by the hour, for example, in a fast-food restaurant. If you service more diners, you do not earn more money. On the other hand, you would be paid according to an FR schedule if you sold magazine subscriptions over the phone and were paid according to how many you sold. As business managers know (Lazear, 1986), the FR schedule of the magazine-sales job is likely to be more motivating. People are less likely to “slack off” when working under the ratio schedule of reinforcement.

In a variable ratio (VR) schedule, reinforcements are delivered after an organism performs a certain number of responses on average. The exact number of responses required on a given trial varies unpredictably from one trial to the next. For example, a rat’s lever-pressing behavior might be reinforced after 20 lever presses on average, but the rat might press the lever only 10 times before it is reinforced on one trial, then press the lever 30 times on the next trial.

Under a VR schedule, animals get reinforced only sporadically. An animal might, on some trials, have to press the lever 50 times or more just to receive one reinforcer. You might expect these schedules to produce low rates of response. But instead, the opposite occurs! VR schedules produce the highest response rates (see Figure 7.13). A remarkable finding from Skinner’s lab is that, while getting very few reinforcements, animals on a VR schedule will exhibit very high rates of response.

289

The power of the VR schedule explains the power of gambling devices, such as slot machines. Slot machines reinforce a sequence of behavior: putting money in the machine and pulling a lever (an action that, ironically, looks quite like the behavior of a rat pressing a lever in a Skinner box). Critically, they reinforce the behavior only sporadically, on a variable ratio schedule. Large payouts do occur, but only once every few hundred or thousand plays. Despite the low frequency of reinforcement, the device is addictive. Whether you’re a pigeon in a Skinner box or a human in a casino, the VR schedule can seize control of your behavior and cause you to lose your nest egg.

EXTINCTION. Earlier in this chapter, you learned about extinction in classical conditioning. In operant conditioning, extinction refers to the pattern of behavior that results when a behavior that previously was reinforced is no longer followed by any reinforcers. For instance, suppose an experimenter had reinforced the lever-pressing behavior of a rat, giving it a small piece of food every time it pressed on the lever. If the experimenter suddenly stopped reinforcing that behavior, that would be a case of extinction.

What happens to the animal’s behavior during extinction? The pattern of extinction depends on the schedule of reinforcement that the animal had experienced previously. If its behavior had been reinforced consistently (e.g., on an FR schedule with a reinforcement occurring after every few responses), extinction is rapid. Animals quickly stop responding when reinforcers cease. However, if the behavior had been reinforced on an irregular, random schedule (e.g., on a VR schedule with large numbers of behaviors sometimes being required to obtain one reinforcer), extinction is very slow. An animal might perform a large number of behaviors while receiving no reinforcement at all.

Extinction, then, is fast if an animal has been reinforced consistently but slow if an animal has been reinforced only intermittently. Compare the following two human cases in terms of extinction:

In the vending machine example, one trial in which you don’t get a reinforcer is enough: You stop putting money into that machine. In the lottery ticket example, one trial in which you don’t get a reinforcer has hardly any effect: Lottery players often buy tickets repeatedly, for months or years, without winning. The properly functioning vending machine reinforces behavior on an FR schedule, whereas the lottery reinforces behavior on a VR schedule, and—whether in the Skinner lab or in everyday life—VR schedules produce behavior that extinguishes very slowly.

290

DISCRIMINATIVE STIMULI. Your behavior commonly changes from one situation to the next. You’re talkative with friends, yet quiet in class or in a church. You work hard on some courses, but “blow off” others. You speak formally to your professor and casually to your roommate. Operant conditioning explains these variations in behavior in terms of discriminative stimuli. A discriminative stimulus is one that provides information about the type of consequences that are likely to follow a given type of behavior. Discriminative stimuli may signal that, in one situation, a behavior likely will be reinforced, whereas in another situation, that same behavior may be punished.

An example of a discriminative stimulus in social interactions is a “shhh” signal (see photo). The gesture indicates that the typical consequences of talking do not hold. Typically, if you talk about some topic, your behavior is reinforced by the attention of the people listening to you. The “shhh” signal indicates that, instead, this same behavior is likely to be punished; something in the typical environment has changed (e.g., someone nearby, who shouldn’t hear your conversation, is listening in), and the behavior will result in negative consequences.

Many of the everyday situations you encounter contain discriminative stimuli—signals indicating the type of behavior that is or is not acceptable. If you walk into a room in which everyone is quiet, their behavior is a discriminative stimulus telling you that loud, lively behavior will not be reinforced here. If you walk into a loud, lively party, the behavior of other people is a discriminative stimulus indicating that quiet, studious behavior will not be reinforced. Discriminative stimuli, then, explain why a person’s behavior can vary so much from one situation to another.

THIS JUST IN

THE SHAPING OF COMPLEX BEHAVIOR. A rat placed in a Skinner box does not immediately start pressing the lever. Lever pressing, from the rat’s perspective, is a somewhat complex behavior. It has to locate the lever, balance to reach it, and learn to press down with sufficient force. How can one teach the rat to do this complex act?

In operant conditioning, organisms learn complex behaviors through shaping. Shaping is a step-by-step learning process in which psychologists reinforce behavior that approximates some desired behavior. The approximations become closer and closer to the target behavior over a series of learning trials.

292

Did you ever wonder how animal trainers teach lions to jump through hoops, seals to balance balls on their noses, and parrots to ride bicycles? They do it by shaping the animals’ behavior through operant conditioning. Suppose you want to train a parrot to ride a bicycle. You can’t just sit around waiting for your parrot to hop on the bike and start pedaling, and only then give it a reinforcer. That’s never going to happen. Instead, you have to begin with something simple. You might at first reinforce the bird merely when it walks in the direction of the bike. Next, you might reinforce it only when it gets closer to the bike, then only when it touches the bike, and so forth. After a potentially painstaking series of shaping experiences, your parrot eventually will become a cyclist.

Much human learning also is shaped gradually. When, as a child, you learned proper manners for eating, adults first reinforced simple actions, such as picking up and using a fork. They didn’t wait until you made it through a formal meal, complete with proper use of salad fork and butter knife, before reinforcing your behavior. Similarly, in a foreign language class, the instructor’s goal is to teach you how to converse in complete sentences. But first, she reinforces something much simpler: the speaking of simple words and phrases. In both cases, behavior is gradually shaped. Over time, the behavior that is reinforced becomes more complex.

TRY THIS!

Earlier, in this chapter’s Try This! Activity, you attempted to teach a pigeon to peck on a wall. Try it again. Now that you have learned the principles of operant conditioning, you should be able to get a higher score than you did previously. In particular, apply what you have learned about schedules of reinforcement and how they can increase rates of response, to improve your score.

293

Skinner believed that his principles of operant conditioning would be comprehensive. In theory, they would explain all operant behavior, by all organisms, in all environments. Skinner made this claim explicitly. His first book (Skinner, 1938) was The Behavior of Organisms—a title that conveyed his belief that principles of learning discovered by studying rats in Skinner boxes would apply to all organisms and all forms of operant behavior.

Research has shown, however, that Skinner’s claim is not correct. His principles of learning are sometimes insufficient. Skinner failed to attend to some factors that have proven important to learning and behavior. Two such factors are (1) biological constraints on learning that are based in the evolutionary history of the species being studied, and (2) the way in which rewards alter not only people’s actions, but also their thoughts about their actions.

BIOLOGICAL CONSTRAINTS ON OPERANT CONDITIONING. Why did Skinner choose to conduct experiments on the lever-pressing behavior of rats and the pecking of pigeons? Convenience. It is relatively easy to maintain a lab full of rats or pigeons and to measure lever pressing and pecking. Skinner presumed the choice of organism and behavior wouldn’t matter; principles of learning, he thought, should apply across all species.

On this point, Skinner turned out to be wrong. The choice of organism and behavior sometimes does make a difference. The differences reflect animals’ innate biology, which produces biological constraints on learning (Domjan & Galef, 1983; Öhman & Mineka, 2001). A biological constraint on learning is an evolved predisposition that makes it difficult, if not impossible, for a given species to learn a certain type of behavior when reinforced with a certain type of reward.

Consider two operant conditioning experiments with a pigeon:

Skinner would have predicted that the two experiments should work the same way; the reinforcer, food, would raise the probability of responding. However, the experiments actually turn out differently (Rachlin, 1976). The pecking experiment works as Skinner expected, but the flapping experiment does not. Food reinforcers do not raise the probability of wing flapping. Why not?

The answer lies in the pigeons’ biological predispositions. For a pigeon, pecking and food naturally are biologically connected. Wing flapping and food, however, are not. In fact, wing flapping and eating were usually disconnected during pigeons’ evolution. For a pigeon in the presence of food, flapping its wings is a bad idea: It flies away and gets no food. Pigeons’ evolutionary history, then, constrains learning; they find it difficult to learn the connection between the behavior, wing flapping, and the reinforcer, food.

This is only one of many cases in which a species’ evolutionary past constrains its ability to learn through operant conditioning. In another (Breland & Breland, 1961), researchers reinforced a raccoon for picking up two coins and dropping them in a box. The raccoon picked up the coins. But instead of dropping them in the box, it rubbed them together, dipped them into the box, pulled them out, rubbed them some more, and never let go. The researchers had never reinforced these behaviors. Yet the raccoon performed them repeatedly. Why? Its actions were a case of instinctual drift, in which animals exhibit behavior that reflects their species’ evolved biological predispositions, rather than their individual learning experiences. Raccoons instinctively moisten their paws and food, and rub their food before eating, and in the experiment the raccoon instinctively did the same thing with the coins, in the absence of reinforcement.

294

Evolution also plays a role in human learning. Consider language. Shaping through operant conditioning cannot explain human language learning (Chomsky, 1959). Young children quickly learn not only the words of their language, but also the language’s syntax, that is, the rules through which sentences are formed. They do so despite the fact that parents rarely provide explicit reinforcements for syntax use (Brown, 1973). Humans are biologically predisposed to learn language; we inherit capabilities that make it easy for us to learn to understand and produce sentences (Pinker, 1994).

REWARDS AND PERSONAL INTEREST IN ACTIVITIES. Skinner thought that the effects of reinforcement on behavior were the same for all species. No matter who you are—rat, pigeon, or human—reinforcers are supposed to raise the probability of future behavior.

Yet there is a complicating factor to which Skinner gave little thought. Rats and pigeons do not think about themselves and the causes of their behavior. (“Am I really the sort of rat who should be stuck in a box pressing a lever all day long?”) People, on the other hand, think about themselves incessantly. Sometimes their thoughts can cause rewards to “backfire”—decreasing rather than increasing behavior.

In a classic study, researchers tested the effects of rewards on preschool children who were intrinsically interested in drawing, that is, who naturally liked to draw (Lepper, Greene, & Nisbett, 1973). In one experimental condition, the children were asked to draw pictures for a chance to win a “Good Player Award.” After they finished, the children received the award. In a second condition, children were asked to draw pictures but with no mention of an award. A few days later, there was a free period during which the children could do anything they wanted, including drawing. Did the Good Player Awards increase the probability of drawing during this period, as Skinner would have predicted? No. The very opposite occurred: Children in the no -award condition spent more time drawing. Why?

The rewards changed children’s thoughts about their behavior (Lepper, Greene, & Nisbett, 1973). At the start of the study, the children were interested in drawing; they knew they drew pictures out of personal interest. But the Good Player Awards caused them to think that they drew pictures to get external rewards. As a result, the children became less interested in drawing when no rewards were available.

A summary of more than 100 studies confirms that rewards can reduce children’s interest in activities (Deci, Koestner, & Ryan, 1999). The research shows that rewards can backfire—a result not anticipated by Skinner’s analysis of operant conditioning.

In summary, these two scientific findings—biological constraints on learning and the reduction of intrinsic interest in activities that is caused by external rewards—move beyond the understanding of reinforcement and behavior provided by Skinner.

295

Now that we’ve learned about psychological processes in operant learning, we can move to a biological level of analysis. What brain mechanisms allow organisms to learn through operant conditioning?

A first step in answering this question was taken more than a half-century ago, when researchers (Olds & Milner, 1954) discovered that the brain contains a reward center, a neural region whose stimulation is pleasurable to organisms. In this classic research, investigators implanted an electrode in a rat’s brain that allowed them to stimulate the limbic system electrically. The researchers then placed the rat in a Skinner box and stimulated its brain whenever the rat pressed the box’s lever (Figure 7.15). The consequence that followed the rat’s behavior was electrical stimulation, which was pleasurable and proved to be reinforcing. The rats repeatedly pressed the lever in order to gain stimulation of their reward center (Olds & Milner, 1954). The results showed, then, that the brain contains a region dedicated to producing pleasurable experiences.

Research has advanced considerably from these early days. Rather than identifying a general area of the brain that is involved in reward, researchers now search for specific neurons that are responsible for operant conditioning. This operant conditioning research has progressed, thanks to study of a simple organism you learned about earlier, Aplysia (Rankin, 2002). Aplysia eat by biting. Researchers (Brembs et al., 2002) reinforced biting by electrically stimulating a reward area of the Aplysia nervous system. This reinforcer changed the Aplysia ’s behavior; it increased their frequency of biting. The researchers then explored neuron-to-neuron connections in the Aplysia nervous system to identify the exact biological change responsible for the change in behavior. They discovered that, when the behavior changed, so did a specific neuron in Aplysia that controls the motor movement of biting. After reinforcement, this neuron became more excitable, in other words, more likely to fire (Brembs et al., 2002). The change in nerve cell firing, then, was responsible for the change in behavior.

Further research with Aplysia has shown that the precise cellular changes that occur in operant conditioning differ from those that occur in classical conditioning (Lorenzetti et al., 2006). This means that classical conditioning and operant conditioning, which differ psychologically, also differ biologically.

296

In more complex organisms, the situation is more complicated. In humans, for instance, rewards have a number of effects—including changing behavior, altering thoughts, and altering emotional states—each of which has different biological underpinnings. Different biological systems contribute, for example, to the experience of wanting a reward that may occur in the future, as opposed to experiencing a reward one has just received (Berridge & Robinson, 2003).

Despite this greater complexity, research conducted with animals (e.g., laboratory rats) has identified a biological substance in mammals that is key to their ability to learn through rewards: dopamine, one of the brain’s neurotransmitters. Different types of findings suggest that dopamine is central to reward processes. The presentation of rewarding stimuli (e.g., food, water, or opportunities for sex) increases levels of dopamine in the brain (Schultz, 2006; Wise, 2004). Furthermore, when the normal functioning of dopamine is blocked, rewards no longer control animals’ behavior.

In one study (Wise et al., 1978), rats were reinforced by food for lever pressing. After learning to press the lever, they were assigned to different experimental conditions. In one, reinforcement for lever pressing ceased; as you might expect, these rats gradually stopped pressing the lever. In a second condition, reinforcement continued; these rats kept pressing. Finally, the key experimental condition combined two features: (1) Lever pressing was reinforced, but (2) rats received an injection of a drug that blocks the neurotransmitter effects of dopamine. These rats stopped pressing the lever. Their behavior resembled that of the rats in the no-reinforcement condition. The results suggest, then, that at a biological level of analysis, reinforcers influence behavior through their effect on the dopamine system.