ASSOCIATING BEHAVIORS AND CONSEQUENCES

KEY THEME

Operant conditioning deals with the learning of active, voluntary behaviors that are shaped and maintained by their consequences.

KEY QUESTIONS

Classical conditioning can help explain the acquisition of many learned behaviors, including emotional and physiological responses. However, recall that classical conditioning involves reflexive behaviors that are automatically elicited by a specific stimulus. Most everyday behaviors don’t fall into this category. Instead, they involve nonreflexive, or voluntary, actions that can’t be explained with classical conditioning.

The investigation of how voluntary behaviors are acquired began with a young American psychology student named Edward L. Thorndike. A few years before Pavlov began his extensive studies of classical conditioning, Thorndike was using cats, chicks, and dogs to investigate how voluntary behaviors are acquired. Thorndike’s pioneering studies helped set the stage for the later work of another American psychologist named B.F. Skinner. It was Skinner who developed operant conditioning, another form of conditioning that explains how we acquire and maintain voluntary behaviors.

197

Edward L. Thorndike was the first psychologist to systematically investigate animal learning and how voluntary behaviors are influenced by their consequences. At the time, Thorndike was only in his early 20s and a psychology graduate student. He conducted his pioneering studies to complete his dissertation and earn his doctorate in psychology.

Thorndike’s dissertation focused on the issue of whether animals, like humans, use reasoning to solve problems (Dewsbury, 1998). In an important series of experiments, Thorndike (1898) put hungry cats in specially constructed cages that he called “puzzle boxes.” A cat could escape the cage by a simple act, such as pulling a loop or pressing a lever that would unlatch the cage door. A plate of food was placed just outside the cage, where the hungry cat could see and smell it.

Thorndike found that when the cat was first put into the puzzle box, it would engage in many different, seemingly random behaviors to escape. For example, the cat would scratch at the cage door, claw at the ceiling, and try to squeeze through the wooden slats (not to mention complain at the top of its lungs). Eventually, however, the cat would accidentally pull on the loop or step on the lever, opening the door latch and escaping the box. After several trials in the same puzzle box, a cat could get the cage door open very quickly.

Thorndike (1898) concluded that the cats did not display any humanlike insight or reasoning in unlatching the puzzle box door. Instead, he explained the cats’ learning as a process of trial and error (Chance, 1999). The cats gradually learned to associate certain responses with successfully escaping the box and gaining the food reward. According to Thorndike, these successful behaviors became “stamped in,” so that a cat was more likely to repeat these behaviors when placed in the puzzle box again. Unsuccessful behaviors were gradually eliminated.

Thorndike’s observations led him to formulate the law of effect: Responses followed by a “satisfying state of affairs” are “strengthened” and more likely to occur again in the same situation. Conversely, responses followed by an unpleasant or “annoying state of affairs” are “weakened” and less likely to occur again.

Thorndike’s description of the law of effect was an important first step in understanding how active, voluntary behaviors can be modified by their consequences. Thorndike, however, never developed his ideas on learning into a formal model or system (Hearst, 1999). Instead, he applied his findings to education, publishing many books on educational psychology (Mayer & others, 2003). Some 30 years after Thorndike’s famous puzzle-box studies, the task of further investigating how voluntary behaviors are acquired and maintained would be taken up by another American psychologist, B.F. Skinner (Rutherford, 2012).

From the time he was a graduate student in psychology until his death, the famous American psychologist B.F. Skinner searched for the “lawful processes” that would explain “order in behavior” (Skinner, 1956, 1967). Like John Watson, Skinner was a staunch behaviorist. Skinner strongly believed that psychology should restrict itself to studying only phenomena that could be objectively measured and verified—outwardly observable behavior and environmental events.

Skinner acknowledged that Pavlov’s classical conditioning could explain the learned association of stimuli in certain reflexive responses (Iversen, 1992). But classical conditioning was limited to existing behaviors that were reflexively elicited. To Skinner, the most important form of learning was demonstrated by new behaviors that were actively emitted by the organism, such as the active behaviors produced by Thorndike’s cats in trying to escape the puzzle boxes.

198

Skinner (1953) coined the term operant to describe any “active behavior that operates upon the environment to generate consequences.” In everyday language, Skinner’s principles of operant conditioning explain how we acquire the wide range of voluntary behaviors that we perform in daily life. But as a behaviorist who rejected mentalistic explanations, Skinner avoided the term voluntary because it would imply that behavior was due to a conscious choice or intention.

Skinner defined operant conditioning concepts in very objective terms and he avoided explanations based on subjective mental states (Moore, 2005b). We’ll closely follow Skinner’s original terminology and definitions.

INCREASING FUTURE BEHAVIOR

In a nutshell, Skinner’s operant conditioning explains learning as a process in which behavior is shaped and maintained by its consequences. One possible consequence of a behavior is reinforcement. Reinforcement is said to occur when a stimulus or an event follows an operant and increases the likelihood of the operant being repeated. Notice that reinforcement is defined by the effect it produces—increasing or strengthening the occurrence of a behavior in the future.

Let’s look at reinforcement in action. Suppose you put your money into a soft-drink vending machine and push the button. Nothing happens. You push the button again. Nothing. You try the coin-return lever. A shower of coins is released. In the future, if another vending machine swallows your money without giving you what you want, what are you likely to do? Hit the coin-return lever, right?

In this example, pushing the coin return lever is the operant—the active response you emitted. The shower of coins is the reinforcing stimulus, or reinforcer—the stimulus or event that is sought in a particular situation. In everyday language, a reinforcing stimulus is typically something desirable, satisfying, or pleasant. Skinner, of course, avoided such terms because they reflected subjective emotional states.

POSITIVE AND NEGATIVE REINFORCEMENT

There are two forms of reinforcement: positive reinforcement and negative reinforcement. Both affect future behavior, but they do so in different ways (see Table 5.1). It’s easier to understand these differences if you note at the outset that Skinner did not use the terms positive and negative in their everyday sense of meaning “good” and “bad” or “desirable” and “undesirable.” Instead, think of the words positive and negative in terms of their mathematical meanings. Positive is the equivalent of a plus sign (+), meaning that something is added. Negative is the equivalent of a minus sign (–), meaning that something is subtracted or removed. If you keep that distinction in mind, the principles of positive and negative reinforcement should be easier to understand.

Positive reinforcement involves following an operant with the addition of a reinforcing stimulus. In positive reinforcement situations, a response is strengthened because something is added or presented. Everyday examples of positive reinforcement in action are easy to identify. Here are some examples:

199

In each example, if the addition of the reinforcing stimulus has the effect of making you more likely to repeat the operant in similar situations in the future, then positive reinforcement has occurred.

It’s important to point out that what constitutes a reinforcing stimulus can vary from person to person, species to species, and situation to situation. While gold stars and stickers may be reinforcing to a third-grader, they would probably have little reinforcing value to your average high school student.

It’s also important to note that the reinforcing stimulus is not necessarily something we usually consider positive or desirable. For example, most teachers would not think of a scolding as being a reinforcing stimulus to children. But to children, adult attention can be a powerful reinforcing stimulus. If a child receives attention from the teacher only when he misbehaves, then the teacher may unwittingly be reinforcing misbehavior. The child may actually increase disruptive behavior in order to get the sought-after reinforcing stimulus—adult attention—even if it’s in the form of being scolded. To reduce the child’s disruptive behavior, the teacher would do better to reinforce the child’s appropriate behavior by paying attention to him when he’s not being disruptive, such as when he is working quietly.

Negative reinforcement involves an operant that is followed by the removal of an aversive stimulus. In negative reinforcement situations, a response is strengthened because something is being subtracted or removed. Remember that the word negative in the phrase negative reinforcement is used like a mathematical minus sign (–).

For example, you take two aspirin (the operant) to remove a headache (the aversive stimulus). Thirty minutes later, the headache is gone. Are you now more likely to take aspirin to deal with bodily aches and pain in the future? If you are, then negative reinforcement has occurred.

Aversive stimuli typically involve physical or psychological discomfort that an organism seeks to escape or avoid. Consequently, behaviors are said to be negatively reinforced when they let you either: (1) escape aversive stimuli that are already present or (2) avoid aversive stimuli before they occur. That is, we’re more likely to repeat the same escape or avoidance behaviors in similar situations in the future. The headache example illustrates the negative reinforcement of escape behavior. By taking two aspirin, you “escaped” the headache. Paying your electric bill on time to avoid a late charge illustrates the negative reinforcement of avoidance behavior. Here are some more examples of negative reinforcement involving escape or avoidance behavior:

In each example, if escaping or avoiding the aversive event has the effect of making you more likely to repeat the operant in similar situations in the future, then negative reinforcement has taken place.

200

PRIMARY AND CONDITIONED REINFORCERS

Skinner also distinguished two kinds of reinforcing stimuli: primary and conditioned. A primary reinforcer is one that is naturally reinforcing for a given species. That is, even if an individual has not had prior experience with the particular stimulus, the stimulus or event still has reinforcing properties. For example, food, water, adequate warmth, and sexual contact are primary reinforcers for most animals, including humans.

A conditioned reinforcer, also called a secondary reinforcer, is one that has acquired reinforcing value by being associated with a primary reinforcer. The classic example of a conditioned reinforcer is money. Money is reinforcing not because those flimsy bits of paper and little pieces of metal have value in and of themselves, but because we’ve learned that we can use them to acquire primary reinforcers and other conditioned reinforcers. Awards, frequent-flyer points, and college degrees are just a few other examples of conditioned reinforcers.

Conditioned reinforcers need not be as tangible as money or college degrees. Conditioned reinforcers can be as subtle as a smile, a touch, or a nod of recognition. Looking back at the Prologue, for example, Fern was reinforced by the laughter of her friends and relatives each time she told “the killer attic” tale—so she kept telling the story!

USING AVERSIVE CONSEQUENCES TO DECREASE BEHAVIOR

KEY THEME

Punishment is a process that decreases the future occurrence of a behavior.

KEY QUESTIONS

Positive and negative reinforcement are processes that increase the frequency of a particular behavior. The opposite effect is produced by punishment. Punishment is a process in which a behavior is followed by an aversive consequence that decreases the likelihood of the behavior’s being repeated. Many people tend to confuse punishment and negative reinforcement, but these two processes produce entirely different effects on behavior (see Table 5.2). Negative reinforcement always increases the likelihood that an operant will be repeated in the future. Punishment always decreases the future performance of an operant.

Skinner (1953) identified two types of aversive events that can act as punishment. Positive punishment, also called punishment by application, involves a response being followed by the presentation of an aversive stimulus. The word positive in the phrase positive punishment signifies that something is added or presented in the situation. In this case, it’s an aversive stimulus. Here are some everyday examples of punishment by application:

201

In each of these examples, if the presentation of the punishing stimulus has the effect of decreasing the behavior it follows, then punishment has occurred. Although the punishing stimuli in these examples were administered by other people, punishing stimuli also occur as natural consequences for some behaviors. Inadvertently touching a live electrical wire, a hot stove, or a sharp object (the operant) can result in a painful injury (the punishing stimulus).

The second type of punishment is negative punishment, also called punishment by removal. The word negative indicates that some stimulus is subtracted or removed from the situation (see Table 5.3). In this case, it is the loss or withdrawal of a reinforcing stimulus following a behavior. That is, the behavior’s consequence is the loss of some privilege, possession, or other desirable object or activity. Here are some everyday examples of punishment by removal:

In each example, if the behavior decreases in response to the removal of the reinforcing stimulus, then punishment has occurred. It’s important to stress that, like reinforcement, punishment is defined by the effect it produces. In everyday usage, people often refer to a particular consequence as a punishment when, strictly speaking, it’s not. Why? Because the consequence has not reduced future occurrences of the behavior. Hence, many consequences commonly thought of as punishments—being sent to prison, fined, reprimanded, ridiculed, or fired from a job—fail to reduce a particular behavior.

Why is it that aversive consequences don’t always function as effective punishments? Skinner (1953) as well as other researchers have noted that several factors influence the effectiveness of punishment (Horner, 2002). For example, punishment is more effective if it immediately follows a response than if it is delayed. Punishment is also more effective if it consistently, rather than occasionally, follows a response (Lerman & Vorndran, 2002; Spradlin, 2002). Though speeding tickets and prison sentences are commonly referred to as punishments, these aversive consequences are inconsistently applied and often administered only after a long delay. Thus, they don’t always effectively decrease specific behaviors.

202

Even when punishment works, its use has several drawbacks (see B. Smith, 2012). First, punishment may decrease a specific response, but it doesn’t necessarily teach or promote a more appropriate response to take its place. Second, punishment that is intense may produce undesirable results, such as complete passivity, fear, anxiety, or hostility (Lerman & Vorndran, 2002). Finally, the effects of punishment are likely to be temporary (Estes & Skinner, 1941; Skinner, 1938). A child who is sent to her room for teasing her little brother may well repeat the behavior when her mother’s back is turned. As Skinner (1971) noted, “Punished behavior is likely to reappear after the punitive consequences are withdrawn.” For some suggestions on how to change behavior without using a punishing stimulus, see the In Focus box, “Changing the Behavior of Others: Alternatives to Punishment.”

203

SETTING THE OCCASION FOR RESPONDING

Another component of operant conditioning is the discriminative stimulus—the specific stimulus in the presence of which a particular operant is more likely to be reinforced. For example, a ringing phone is a discriminative stimulus that sets the occasion for a particular response—picking up the telephone and speaking.

This example illustrates how we’ve learned from experience to associate certain environmental cues or signals with particular operant responses. We’ve learned that we’re more likely to be reinforced for performing a particular operant response when we do so in the presence of the appropriate discriminative stimulus. Thus, you’ve learned that you’re more likely to be reinforced for screaming at the top of your lungs at a football game (one discriminative stimulus) than in the middle of class (a different discriminative stimulus).

In this way, according to Skinner (1974), behavior is determined and controlled by the stimuli that are present in a given situation. In Skinner’s view, an individual’s behavior is not determined by a personal choice or a conscious decision. Instead, individual behavior is determined by environmental stimuli and the person’s reinforcement history in that environment. Skinner’s views on this point have some very controversial implications, which are discussed in the Critical Thinking box on the next page, “Is Human Freedom Just an Illusion?”

204

205

We have now discussed all three fundamental components of operant conditioning (see Table 5.4). In the presence of a specific environmental stimulus (the discriminative stimulus), we emit a particular behavior (the operant), which is followed by a consequence (reinforcement or punishment). If the consequence is either positive or negative reinforcement, we are more likely to repeat the operant when we encounter the same or similar discriminative stimuli in the future. If the consequence is some form of punishment, we are less likely to repeat the operant when we encounter the same or similar discriminative stimuli in the future.

Next, we’ll build on the basics of operant conditioning by considering how Skinner explained the acquisition of complex behaviors.

KEY THEME

New behaviors are acquired through shaping and can be maintained through different patterns of reinforcement.

KEY QUESTIONS

To scientifically study the relationship between behavior and its consequences in the laboratory, Skinner invented the operant chamber, more popularly known as the Skinner box. An operant chamber is a small cage with a food dispenser. Attached to the cage is a device that automatically records the number of operants made by an experimental animal, usually a rat or pigeon. For a rat, the typical operant is pressing a bar; for a pigeon, it is pecking at a small disk. Food pellets are usually used for positive reinforcement. Often, a light in the cage functions as a discriminative stimulus. When the light is on, pressing the bar or pecking the disk is reinforced with a food pellet. When the light is off, these responses do not result in reinforcement.

When a rat is first placed in a Skinner box, it typically explores its new environment, occasionally nudging or pressing the bar in the process. The researcher can accelerate the rat’s bar-pressing behavior through a process called shaping. Shaping involves reinforcing successively closer approximations of a behavior until the correct behavior is displayed. For example, the researcher might first reinforce the rat with a food pellet whenever it moves to the half of the Skinner box in which the bar is located. Other responses would be ignored. Once that response has been learned, reinforcement is withheld until the rat moves even closer to the bar. Then the rat might be reinforced only when it touches the bar. Step by step, the rat is reinforced for behaviors that correspond ever more closely to the final goal behavior—pressing the bar.

206

Skinner believed that shaping could explain how people acquire a wide variety of abilities and skills—everything from tying shoes to operating sophisticated computer programs. Athletic coaches, teachers, parents, and child-care workers all use shaping techniques.

THE PARTIAL REINFORCEMENT EFFECT

BUILDING RESISTANCE TO EXTINCTION

Once a rat had acquired a bar-pressing behavior, Skinner found that the most efficient way to strengthen the response was to immediately reinforce every occurrence of bar pressing. This pattern of reinforcement is called continuous reinforcement. In everyday life, of course, it’s common for responses to be reinforced only sometimes—a pattern called partial reinforcement. For example, practicing your basketball skills isn’t followed by putting the ball through the hoop on every shot. Sometimes you’re reinforced by making a basket, and sometimes you’re not.

Now suppose that despite all your hard work, your basketball skills are dismal. If practicing free throws was never reinforced by making a basket, what would you do? You’d probably eventually quit playing basketball. This is an example of extinction. In operant conditioning, when a learned response no longer results in reinforcement, the likelihood of the behavior’s being repeated gradually declines.

Skinner (1956) first noticed the effects of partial reinforcement when he began running low on food pellets one day. Rather than reinforcing every bar press, Skinner tried to stretch out his supply of pellets by rewarding responses only periodically. He found that the rats not only continued to respond, but actually increased their rate of bar pressing.

One important consequence of partially reinforcing behavior is that partially reinforced behaviors tend to be more resistant to extinction than are behaviors conditioned using continuous reinforcement. This phenomenon is called the partial reinforcement effect. For example, when Skinner shut off the food-dispensing mechanism, a pigeon conditioned using continuous reinforcement would continue pecking at the disk 100 times or so before the behavior decreased significantly, indicating extinction. In contrast, a pigeon conditioned with partial reinforcement continued to peck at the disk thousands of times! If you think about it, this is not surprising. When pigeons, rats, or humans have experienced partial reinforcement, they’ve learned that reinforcement may yet occur, despite delays and nonreinforced responses, if persistent responses are made.

207

In everyday life, the partial reinforcement effect is reflected in behaviors that persist despite the lack of reinforcement. Gamblers may persist despite a string of losses, writers will persevere in the face of repeated rejection slips, and the family dog will continue begging for the scraps of food that it has only occasionally received at the dinner table in the past.

THE SCHEDULES OF REINFORCEMENT

Skinner (1956) found that specific preset arrangements of partial reinforcement produced different patterns and rates of responding. Collectively, these different reinforcement arrangements are called schedules of reinforcement. As we describe the four basic schedules of reinforcement, it will be helpful to refer to Figure 5.5, which shows the typical pattern of responses produced by each schedule.

With a fixed-ratio (FR) schedule, reinforcement occurs after a fixed number of responses. A rat on a 10-to-1 fixed-ratio schedule (abbreviated FR-10) would have to press the bar 10 times in order to receive one food pellet. Fixed-ratio schedules typically produce a high rate of responding that follows a burst–pause– burst pattern. In everyday life, the fixed-ratio schedule is reflected in any activity that requires a precise number of responses in order to obtain reinforcement. Piecework—work for which you are paid for producing a specific items, such as being paid $1 for every 100 envelopes you stuff—is an example of an FR-100 schedule.

208

With a variable-ratio (VR) schedule, reinforcement occurs after an average number of responses, which varies from trial to trial. A rat on a variable-ratio-20 schedule (abbreviated VR-20) might have to press the bar 25 times on the first trial before being reinforced and only 15 times on the second trial before reinforcement. Although the number of responses required on any specific trial is unpredictable, over repeated trials the ratio of responses to reinforcers works out to the predetermined average.

Variable-ratio schedules of reinforcement produce high, steady rates of responding with hardly any pausing between trials or after reinforcement. Gambling is the classic example of a variable-ratio schedule in real life (Schüll, 2012). Each spin of the roulette wheel, toss of the dice, or purchase of a lottery ticket could be the big one, and the more often you gamble, the more opportunities you have to win (and lose, as casino owners are well aware).

On a fixed-interval (FI) schedule, a reinforcer is delivered for the first response emitted after the preset time interval has elapsed. A rat on a two-minute fixed-interval schedule (abbreviated FI-2 minutes) would receive no food pellets for any bar presses made during the first two minutes. But the first bar press after the two-minute interval had elapsed would be reinforced.

Fixed-interval schedules typically produce a scallop-shaped pattern of responding in which the number of responses tends to increase as the time for the next reinforcer draws near. For example, if your instructor gives you a test every four weeks, your studying behavior would probably follow the same scallop-shaped pattern of responding as the rat’s bar-pressing behavior. As the end of the four-week interval draws near, studying behavior increases. After the test, studying behavior drops off until the end of the next four-week interval approaches.

On a variable-interval (VI) schedule, reinforcement occurs for the first response emitted after an average amount of time has elapsed, but the interval varies from trial to trial. Hence, a rat on a VI-30 seconds schedule might be reinforced for the first bar press after only 10 seconds have elapsed on the first trial, for the first bar press after 50 seconds have elapsed on the second trial, and for the first bar press after 30 seconds have elapsed on the third trial. This works out to an average of one reinforcer every 30 seconds.

Generally, the unpredictable nature of variable-interval schedules tends to produce moderate but steady rates of responding, especially when the average interval is relatively short. In daily life, we experience variable-interval schedules when we have to wait for events that follow an approximate, rather than a precise, schedule. For example, parents often unwittingly reinforce a whining child on a variable interval schedule. From the child’s perspective, the whining usually results in the desired request, but how long the child has to whine before getting reinforced can vary. Thus, the child learns that persistent whining will eventually pay off.

209

The In Focus box on alternatives to punishment earlier in the chapter described how operant conditioning principles can be applied to reduce and eliminate problem behaviors. These examples illustrate behavior modification, the application of learning principles to help people develop more effective or adaptive behaviors. Most often, behavior modification involves applying the principles of operant conditioning to bring about changes in behavior.

Behavior modification techniques have been successfully applied in many different settings (see Kazdin, 2008). Coaches, parents, teachers, and employers all routinely use operant conditioning. For example, behavior modification has been used to reduce public smoking by teenagers (Jason & others, 2009), improve student behavior in school cafeterias (McCurdy & others, 2009), reduce problem behaviors in schoolchildren (Dunlap & others, 2010; Schanding & Sterling-Turner, 2010), and improve social skills and reduce self-destructive behaviors in people with autism and related disorders (Makrygianni & Reed, 2010).

Businesses also use behavior modification. For example, one large retailer increased productivity by allowing employees to choose their own reinforcers. A casual dress code and flexible work hours proved to be more effective reinforcers than money (Raj & others, 2006). In each of these examples, the systematic use of reinforcement, shaping, and extinction increased the occurrence of desirable behaviors and decreased the incidence of undesirable behaviors. In Chapter 14 on therapies, we’ll look at behavior modification techniques in more detail.

The principles of operant conditioning have also been used in the specialized training of animals, such as the Labrador shown at right, to help people who are physically challenged. Other examples are Seeing Eye dogs and capuchin monkeys who assist people who are severely disabled.