Your eyes receive light energy and transform it into neural messages that your brain then processes into what you consciously see. How does such a taken-for-granted yet remarkable thing happen?

The Stimulus Input: Light Energy

When you look at a bright red tulip, what strikes your eyes are not bits of the color red but pulses of energy that your visual system perceives as red. What we see as visible light is but a thin slice of the wide spectrum of electromagnetic energy shown in FIGURE 5.9. On one end of this spectrum are the short gamma waves, no longer than the diameter of an atom. On the other end are the mile-long waves of radio transmission. In between is the narrow band visible to us. Other portions are visible to other animals. Bees, for instance, cannot see what we perceive as red but they can see ultraviolet light.

Light travels in waves, and the shape of those waves influences what we see. Light’s wavelength is the distance from one wave peak to the next (FIGURE 5.10a). Wavelength determines hue—the color we experience, such as a tulip’s red petals. A light wave’s amplitude, or height, determines its intensity—the amount of energy the wave contains. Intensity influences brightness (FIGURE 5.10b).

Understanding the characteristics of the physical energy we see as light is one part of understanding vision. But to appreciate how we transform that energy into color and meaning, we need to know more about vision’s window—the eye.

The Eye

What color are your eyes? Asked this question, most people describe their iris, the doughnut-shaped ring of muscle that controls the size of your pupil. Your iris is so distinctive that an iris-scanning machine can confirm your identity. Your sensitive iris can also reveal some of your thoughts and emotions. When you feel disgust or are about to answer No, your iris constricts, making your pupil smaller (de Gee et al., 2014; Goldinger & Papesh, 2012). When you’re feeling amorous, your iris dilates, enlarging your pupil and signaling your interest. But the iris’ main job is controlling the amount of light entering your eye.

141

After passing through your pupil, light hits the lens in your eye. The lens then focuses the light rays into an image on your eyeball’s inner surface, the retina. For centuries, scientists knew that an image of a candle passing through a small opening will cast an upside-down, mirror image on a dark wall behind. They wondered how, if the eye’s structure casts this sort of image on the retina (as in FIGURE 5.11), can we see the world right side up?

Eventually the answer became clear: The retina doesn’t “see” a whole image. Rather, its millions of receptor cells behave like the prankster engineering students who make news by taking a car apart and rebuilding it in a friend’s third-floor bedroom. The retina’s millions of cells convert the particles of light energy into neural impulses and forward those to the brain. The brain reassembles them into what we perceive as an upright object. And along the way, visual information processing moves through increasingly more abstract levels, all at astonishing speed.

142

Imagine that you could follow a single light-energy particle after it entered your eye. First, you would thread your way through your retina’s sparse outer layer of cells. Then, reaching the back of the eye, you would meet the retina’s buried receptor cells, the rods and cones (FIGURE 5.12). There, you would see the light energy trigger chemical changes. That chemical reaction would spark neural signals in the nearby bipolar cells. You could then watch the bipolar cells activate neighboring ganglion cells, whose axons twine together like strands of a rope to form the optic nerve. After a momentary stopover at the thalamus, rather like changing planes in Chicago, the information will fly on to the final destination, your visual cortex, at the back of your brain.

The optic nerve is an information highway from the eye to the brain. This nerve can send nearly 1 million messages at once through its nearly 1 million ganglion fibers. We pay a small price for this high-speed connection, however. Your eye has a blind spot, with no receptor cells, where the optic nerve leaves the eye (FIGURE 5.13). Close one eye. Do you see a black hole? No, because, without seeking your approval, your brain will fill in the hole.

The retina’s two types of receptor cells, rods and cones, differ in where they’re found and in what they do (TABLE 5.1). Cones cluster in and around the fovea, the retina’s area of central focus. Many cones have their own hotline to the brain. One cone transmits a precise message to a single bipolar cell, which relays it to the visual cortex. Thanks to cones, you can see fine details and perceive color—but not at night. In dim light, cones don’t function well.

143

Rods, which are located around the outer regions (the periphery) of your retina, remain sensitive in dim light. If cones are soloists, rods perform as a chorus. They enable black-and-white vision. Rods have no hotlines to the brain. Several rods pool their faint energy output and funnel it onto a single bipolar cell, which sends the combined message to your brain.

Stop for a minute and experience the rod-cone difference. Pick a word in this sentence and stare directly at it, focusing its image on the cones in the center of your eye. Notice that words a few inches off to the side appear blurred? They lack detail because their image is striking your retina’s outer regions, where most rods are found. Thus, when driving or biking, rods help you detect a car in your peripheral vision well before you perceive its details.

So, cones and rods each provide a special sensitivity: Cones are sensitive to detail and color. Rods are sensitive to faint light and peripheral motion. But these receptor cells do more than simply pass along electrical impulses. They begin processing sensory information by coding and analyzing it. (In a frog’s eye, for example, the “bug detector” cells that fire when they respond to moving fly-like objects are found in the retina’s third neural layer.) After this round of processing, information travels up your optic nerve, headed toward a specific location in your visual cortex in the back of your brain. In an important stop on that journey, the optic nerve links up with neurons in the thalamus (FIGURE 5.14).

144

Color Processing

We talk as though objects possess color: “A tomato is red.” Recall the old question, “If a tree falls in the forest and no one hears it, does it make a sound?” We can ask the same of color: If no one sees the tomato, is it red?

The answer is No. First, the tomato is everything but red, because it rejects (reflects) the long wavelengths of red. Second, the tomato’s color is our mental construction. As the famous physicist Sir Isaac Newton (1704) observed more than three centuries ago, “The [light] rays are not colored.” Color, like all aspects of vision, lives not in the object but in the theater of our brain. Even while dreaming, we may perceive things in color.

One of vision’s most basic and intriguing mysteries is how we see the world in color. How, from the light energy striking your retina, does your brain construct your experience of color—and of so many colors?

Modern detective work on the mystery of color vision began in the nineteenth century, when Hermann von Helmholtz built on the insights of an English physicist, Thomas Young. The clue that led to their breakthrough was the knowledge that any color can be created by combining the light waves of three primary colors—red, green, and blue. Young and von Helmholtz reasoned that the eye must therefore have three types of receptors, one for each color.

Years later, researchers confirmed the Young-Helmholtz trichromatic (three-color) theory. By measuring the response of various cones to different color stimuli, they found that the retina does indeed have three types of color receptors. Each type is especially sensitive to the wavelengths of one of three colors, and those colors are, in fact, red, green, and blue. When light stimulates combinations of these cones, we see other colors. For example, the retina has no separate receptors especially sensitive to yellow. But when red and green wavelengths stimulate both red-sensitive and green-sensitive cones, we see yellow.

By one estimate, we can see differences among more than 1 million color variations (Neitz et al., 2001). At least most of us can. About 1 person in 50 is “colorblind.” That person is usually male, because the defect is genetically sex linked. Most people with color-deficient vision are not actually blind to all colors. They simply have trouble perceiving the difference between red and green. They don’t have three-color vision. Instead, perhaps unknown to them (because their lifelong vision seems normal), their retinas’ red- or green-sensitive cones, or sometimes both, don’t function properly (FIGURE 5.15).

But why do people blind to red and green often still see yellow? And why does yellow appear to be a pure color, not a mixture of red and green, the way purple is of red and blue? As Ewald Hering soon noted, trichromatic theory leaves some parts of the color vision mystery unsolved.

Hering, a physiologist, found a clue in afterimages. Stare at a green square for a while and then look at a white sheet of paper, and you will see red, green’s opponent color. Stare at a yellow square and its opponent color, blue, will appear on the white paper. (To experience this, try the flag demonstration in FIGURE 5.16.) Hering proposed that color vision must involve two additional processes: one responsible for red-versus-green perception, and the other for blue-versus-yellow.

A century later, researchers confirmed Hering’s proposal, now called the opponent-process theory. They found that color vision depends on three pairs of opponent retinal processes—red-green, yellow-blue, and white-black. As impulses travel to the visual cortex, some neurons in both the retina and the thalamus are “turned on” by red but “turned off” by green. Others are turned on by green but off by red (DeValois & DeValois, 1975). Like red and green marbles sent down a narrow tube, “red” and “green” messages cannot both travel at once. Red and green are thus opponents, so we see either red or green, not a mixture of reddish green. But red and blue travel in separate channels, so we are able to see a reddish-blue, or purple.

How does opponent-process theory help us understand afterimages, such as in the flag demonstration? Here’s the answer (for the green changing to red):

145

The present solution to the mystery of color vision is therefore roughly this: Color processing occurs in two stages.

Feature Detection

Scientists once compared the brain to a movie screen, on which the eye projected images. But along came David Hubel and Torsten Wiesel (1979), who showed that at an early stage, our visual processing system takes images apart and later reassembles them. Hubel and Wiesel received a Nobel Prize for their work on feature detectors, nerve cells in the brain that respond to a scene’s specific features—to particular edges, lines, angles, and movements. These specialized nerve cells in the visual cortex pass this specific information to other cortical areas, where teams of cells (supercell clusters) respond to more complex patterns, such as recognizing faces. The resulting brain activity varies depending on what’s viewed. Thus, with the help of brain scans, “we can tell if a person is looking at a shoe, a chair, or a face,” noted one researcher (Haxby, 2001).

One temporal lobe area by your right ear enables you to perceive faces and, thanks to a specialized neural network, to recognize them from many viewpoints (Connor, 2010). If this region is damaged, people still may recognize other forms and objects, but, like Heather Sellers, they cannot recognize familiar faces. How do we know this? In part because in laboratory experiments, researchers have used magnetic pulses to disrupt that brain area, producing a temporary loss of face recognition. The interaction between feature detectors and supercells provides instant analyses of objects in the world around us.

Parallel Processing

146

One of the most amazing aspects of visual information processing is the brain’s ability to divide a scene into its parts. Using parallel processing, your brain assigns different teams of nerve cells the separate tasks of simultaneously processing a scene’s movement, form, depth, and color (FIGURE 5.17). You then construct your perceptions by integrating the work of these different visual teams (Livingstone & Hubel, 1988).

Destroy or disable the neural workstation for a visual subtask, and something peculiar results, as happened to “Mrs. M.” (Hoffman, 1998). After a stroke damaged areas near the rear of both sides of her brain, she could not perceive movement. People in a room seemed “suddenly here or there but I [had] not seen them moving.” Pouring tea into a cup was a challenge because the fluid appeared frozen—she could not perceive it rising in the cup. Brain damage often reveals the importance of the parallel processing that operates, beyond our awareness, in our normal everyday life.

* * *

Think about the wonders of visual processing. As you read this page, the letters reflect light rays onto your retina, which then sends formless nerve impulses to several areas of your brain, which integrate the information and decode its meaning. The amazing result: we have transferred information across time and space, from our minds to your mind (FIGURE 5.18). That all of this happens instantly, effortlessly, and continuously is awe-inspiring.

It’s one thing to understand how we see colors and shapes. But how do we organize and interpret those sights (or sounds or tastes or smells) so that they become meaningful perceptions—a rose in bloom, a familiar face, a sunset?

Early in the twentieth century, a group of German psychologists noticed that people who are given a cluster of sensations tend to organize them into a gestalt, a German word meaning a “form” or a “whole.” As we look straight ahead, we cannot separate the perceived scene into our left and right fields of view. Our conscious perception is, at every moment, one whole, seamless scene. Consider FIGURE 5.19. The individual elements of this figure, called a Necker cube, are really nothing but eight blue circles, with three white lines meeting near the center. When we view all these elements together, however, we see a cube that sometimes reverses direction. The Necker cube nicely illustrates a famous saying of Gestalt psychologists: In perception, the whole may exceed the sum of its parts.

147

Over the years, the Gestalt psychologists demonstrated many principles we use to organize our sensations into perceptions. Underlying all of them is a basic truth: Our brain does more than register information about the world. Perception is not just opening a shutter and letting a picture print itself on the brain. We filter incoming information and we construct perceptions. Mind matters.

How Do We Perceive Form?

Imagine designing a video-computer system that, like your eye-brain system, could recognize faces at a glance. What abilities would it need? To start with, the video-computer system would need to perceive figure-ground—to separate faces from their backgrounds. In our eye-brain system, this is our first perceptual task—perceiving any object (the figure) as distinct from its surroundings (the ground). As you read, the words are the figure; the white space is the ground. This perception applies to our hearing, too. As you hear voices at a party, the one you attend to becomes the figure; all others are part of the ground. Sometimes, the same stimulus can trigger more than one perception, as in FIGURE 5.20, where the figure-ground relationship continually reverses. First we see the vase, then the faces, but we always perceive a figure standing out from a ground.

While telling figure from ground, we (and our video-computer system) must also organize the figure into a meaningful form. Some basic features of a scene—such as color, movement, and light-dark contrast—we process instantly and automatically (Treisman, 1987). Our mind brings order and form to other stimuli by following certain rules for grouping, also identified by the Gestalt psychologists. These rules, which we apply even as infants and even in our touch perceptions, illustrate how the perceived whole differs from the sum of its parts (Gallace & Spence, 2011; Quinn et al., 2002; Rock & Palmer, 1990). Three examples:

Proximity We group nearby figures together. We see not six separate lines, but three sets of two lines:

Continuity We perceive smooth, continuous patterns rather than discontinuous ones. This pattern could be a series of alternating semicircles, but we perceive it as two continuous lines—one wavy, one straight:

Closure We fill in gaps to create a complete, whole object. Thus, we assume that the circles on the left are complete but partially blocked by the (illusory) triangle. Add nothing more than little lines to close off the circles, and your brain stops constructing a triangle:

148

How Do We Perceive Depth?

Our brain performs many amazing tricks, but one of its best is depth perception. From the two-dimensional images falling on our retinas, our brain creates three-dimensional perceptions that let us estimate the distance of an oncoming car or the height of a faraway house. How do we acquire this ability? Are we born with it? Do we learn it?

As Eleanor Gibson picnicked on the rim of the Grand Canyon, her scientific curiosity kicked in. She wondered, Would a toddler peering over the rim perceive the dangerous drop-off and draw back? To answer that question and others, Gibson and Richard Walk (1960) designed a series of experiments using a visual cliff—a model of a cliff with a “drop-off” area that was covered by sturdy glass. They placed 6- to 14-month-olds on the edge of the “cliff” and had their mothers coax the infants to crawl out onto the glass (FIGURE 5.21). Most infants refused to do so, indicating that they could perceive depth.

Had they learned to perceive depth? Crawling, no matter when it begins, seems to increase an infant’s fear of heights (Adolph et al., 2014; Campos et al., 1992). But depth perception is also partly innate. Mobile newborn animals—even those with no visual experience (including young kittens, a day-old goat, and newly hatched chicks)—also refuse to venture across the visual cliff. Thus, biology prepares us to be wary of heights, and experience amplifies that fear.

If we were to build this ability into our video-computer system, what rules might enable it to convert two-dimensional images into a single three-dimensional perception? A good place to start would be the depth cues our brain receives from information supplied by one or both of our eyes.

BINOCULAR CUES People who see with two eyes perceive depth thanks partly to binocular cues. Here’s an example. With both eyes open, hold two pens or pencils in front of you and touch their tips together. Now do so with one eye closed. A more difficult task, yes?

We use binocular cues to judge the distance of nearby objects. One such cue is retinal disparity. Because your eyes are about 2½ inches apart, your retinas receive slightly different images of the world. By comparing these two images, your brain can judge how close an object is to you. The greater the disparity (the difference) between the two retinal images, the closer the object. Try it. Hold your two index fingers, with the tips about half an inch apart, directly in front of your nose, and your retinas will receive quite different views. If you close one eye and then the other, you can see the difference. (You can also create a finger sausage, as in FIGURE 5.22.) At a greater distance—say, when you hold your fingers at arm’s length—the disparity is smaller.

We could easily include retinal disparity in our video-computer system. Moviemakers do so by filming a scene with two cameras placed a few inches apart. Viewers then watch the film through glasses that allow the left eye to see only the image from the left camera, and the right eye to see only the image from the right camera. The resulting effect, as 3-D movie fans know, mimics or exaggerates normal retinal disparity, giving the perception of depth.

MONOCULAR CUES How do we judge whether a person is 10 or 100 yards away? Retinal disparity won’t help us here, because there won’t be much difference between the images cast on our right and left retinas. At such distances, we depend on monocular cues (depth cues available to each eye separately). See FIGURE 5.23 for some examples.

149

150

How Do We Perceive Motion?

Imagine that you could perceive the world as having color, form, and depth but that you could not see motion. You would be unable to bike or drive, and writing, eating, and walking would be a challenge.

Normally your brain computes motion based partly on its assumption that shrinking objects are moving away (not getting smaller) and enlarging objects are approaching. But you are imperfect at motion perception. In young children, this ability to correctly perceive approaching (and enlarging) vehicles is not yet fully developed, which puts them at risk for pedestrian accidents (Wann et al., 2011).

It’s not just children who have occasional difficulties with motion perception. Our adult brain is sometimes tricked into believing what it is not seeing. When large and small objects move at the same speed, the large objects appear to move more slowly. Thus, trains seem to move slower than cars, and jumbo jets seem to land more slowly than smaller jets.

Perceptual Constancy

So far, we have noted that our video-computer system must perceive objects as we do—as having a distinct form, location, and perhaps motion. Its next task is to recognize objects without being deceived by changes in their color, shape, or size. We call this top-down process perceptual constancy. This feat would be an enormous challenge for a video-computer system.

COLOR CONSTANCY Our experience of color depends on an object’s context. This would be clear if you viewed an isolated tomato through a paper tube over the course of the day. As the light—and thus the tomato’s reflected wavelengths—changed, the tomato’s color would also seem to change. But if you discarded the paper tube and viewed the tomato as one item in a salad bowl, its color would remain roughly constant as the lighting shifted. This perception of consistent color is known as color constancy.

We see color thanks to our brain’s ability to decode the meaning of the light reflected by an object relative to the objects surrounding it. FIGURE 5.24 dramatically illustrates the ability of a blue object to appear very different in three different contexts. Yet we have no trouble seeing these disks as blue. Paint manufacturers have learned this lesson. Knowing that your perception of a paint color will be determined by other colors in your home, many now offer trial samples you can test in that context. The take-home lesson: Comparisons govern our perceptions.

SHAPE AND SIZE CONSTANCIES Thanks to shape constancy, we usually perceive the form of familiar objects, such as the door in FIGURE 5.25, as constant even while our retinas receive changing images of them. Thanks to size constancy, we perceive objects as having a constant size even while our distance from them varies. We assume a car is large enough to carry people, even when we see its tiny image from two blocks away. This assumption also shows the close connection between perceived distance and perceived size. Perceiving an object’s distance gives us cues to its size. Likewise, knowing its general size—that the object is a car—provides us with cues to its distance.

Even in size-distance judgments, however, we consider an object’s context. This interplay between perceived size and perceived distance helps explain several well-known illusions, including the Moon illusion. The Moon looks up to 50 percent larger when near the horizon than when high in the sky. Can you imagine why? One reason is that monocular cues to an object’s distance make the horizon Moon appear farther away. If it’s farther away, our brain assumes, it must be larger than the Moon high in the night sky (Kaufman & Kaufman, 2000). Take away the distance cues—by looking at the horizon Moon through a paper tube—and the object will immediately shrink.

151

Mistaken judgments like these reveal the workings of our normally effective perceptual processes. The perceived relationship between distance and size is usually valid. But under special circumstances it can lead us astray.

Form perception, depth perception, and perceptual constancies illuminate how we organize our visual experiences. Perceptual organization applies to our other senses, too. Listening to an unfamiliar language, we have trouble hearing where one word stops and the next one begins. Listening to our own language, we automatically hear distinct words. We even organize a string of letters—THEDOGATEMEAT—into words that make an understandable phrase, more likely “The dog ate meat” than “The do gate me at” (McBurney & Collings, 1984). Perception, however, is more than organizing stimuli. Perception also requires what would be another big challenge to our video-computer system: interpretation—finding meaning in what we perceive.

The debate over whether our perceptual abilities spring from our nature or our nurture has a long history. To what extent do we learn to perceive? German philosopher Immanuel Kant (1724–1804) maintained that knowledge comes from our inborn ways of organizing sensory experiences. Psychology’s findings support this idea. We do come equipped to process sensory information. But British philosopher John Locke (1632–1704) argued that through our experiences we also learn to perceive the world. Psychology also supports this idea. We do learn to link an object’s distance with its size. So, just how important is experience? How much does it shape our perceptual interpretations?

Experience and Visual Perception

RESTORED VISION AND SENSORY RESTRICTION Writing to John Locke, a friend wondered what would happen if “a man born blind, and now adult, [was] taught by his touch to distinguish between a cube and a sphere.” Could he, if made to see, visually distinguish the two? Locke’s answer was No, because the man would never have learned to see the difference.

This question has since been put to the test with a few dozen adults who, though blind from birth, later gained sight (Gregory, 1978; Huber et al., 2015; von Senden, 1932). Most were born with cataracts—clouded lenses that allowed them to see only light and shadows, rather as a sighted person might see a foggy image through a Ping-Pong ball sliced in half. After cataract surgery, the patients could tell the difference between figure and ground, and they could sense colors. This suggests that we are born with these aspects of perception. But much as Locke supposed, they often could not visually recognize objects that were familiar by touch.

Seeking to gain more control than is provided by clinical cases, researchers have outfitted infant kittens and monkeys with goggles through which they could see only diffuse, unpatterned light (Wiesel, 1982). After infancy, when the goggles were removed, the animals’ reactions were much like those of humans born with cataracts. They could distinguish color and brightness but not form. Their eyes were healthy. Their retinas still sent signals to their visual cortex. But without early stimulation, the brain’s cortical cells had not developed normal connections. Thus, the animals remained functionally blind to shape.

152

Surgery on blind children in India reveals that children blind from birth can benefit from cataract surgery. Although their vision may never be as sharp as normal, the younger they are, the more they will benefit (Sinha, 2013). There is a critical period for normal sensory and perceptual development—a limited time when exposure to certain stimuli or experiences is required.

In humans and other animals, similar sensory restrictions later in life do no permanent harm. When researchers cover an adult animal’s eye for several months, its vision will be unaffected after the eye patch is removed. When surgeons remove cataracts that developed during late adulthood, most people are thrilled at the return to normal vision.

PERCEPTUAL ADAPTATION Given a new pair of glasses, we may feel a little strange, even dizzy. Within a day or two, we adjust. Our perceptual adaptation to changed visual input makes the world seem normal again. But imagine wearing a far more dramatic pair of new glasses—one that shifts the apparent location of objects 40 degrees to the left. When you toss a ball to a friend, it sails off to the left. Walking forward to shake hands with someone, you veer to the left.

Could you adapt to this distorted world? Not if you were a baby chicken. When fitted with such lenses, baby chicks continue to peck where food grains seem to be (Hess, 1956; Rossi, 1968). But we humans adapt to distorting lenses quickly. Within a few minutes, your throws would again be accurate, your stride on target. Remove the lenses and you would experience an aftereffect. At first your throws would err in the opposite direction, sailing off to the right. But again, within minutes you would adjust.

Indeed, given an even more radical pair of glasses—one that literally turns the world upside down—you could still adapt. Psychologist George Stratton (1896) experienced this when he invented, and for eight days wore, a device that flipped left to right and up to down, making him the first person to experience a right-side-up retinal image while standing upright. The ground was up; the sky was down.

At first, when Stratton wanted to walk, he found himself searching for his feet, which were now “up.” Eating was nearly impossible. He became nauseated and depressed. But he persisted, and by the eighth day he could comfortably reach for an object and walk without bumping into things. When Stratton finally removed the headgear, he readapted quickly. So did research participants who also wore such optical gear—while riding a motorcycle, skiing the Alps, or flying an airplane (Dolezal, 1982; Kohler, 1962). By actively moving about in their topsy-turvy world, they adapted to their new context and learned to coordinate their movements. Later, they readapted to normal life.

So do we learn to perceive the world? In part we do, as we constantly adjust to changed sensory input. Research on critical periods teaches us that early nurture sculpts what nature has provided. In less dramatic ways, nurture continues to do this throughout our lives. Experience guides, sustains, and maintains the pathways in our brain that enable our perceptions.