We have seen that vision and kinesthesia interact. Actually, all our senses eavesdrop on one another (Rosenblum, 2013). This is sensory interaction at work. One sense can influence another.

Consider how smell sticks its nose into the business of taste. Hold your nose, close your eyes, and have someone feed you various foods. You may be unable to tell a slice of apple from a chunk of raw potato. A piece of steak may taste like cardboard. Without their smells, a cup of cold coffee and a glass of red wine may seem the same. To savor a taste, we normally breathe the aroma through our nose—which is why eating is not much fun when you have a bad cold. Smell can also enhance our taste: A strawberry odor intensifies our perception of a drink’s sweetness. Even touch can influence taste. Depending on its texture, a potato chip “tastes” fresh or stale (Smith, 2011). Smell + texture + taste = flavor.

Hearing and vision may similarly interact. We can see a tiny flicker of light more easily if it is paired with a short burst of sound (Kayser, 2007). The reverse is also true: We can hear soft sounds more easily if they are paired with a visual cue (FIGURE 5.29). If I [DM], a person with hearing loss, watch a video with on-screen captions, I have no trouble hearing the words I see. But if I then decide I don’t need the captions, and turn them off, I will quickly realize I do need them. The eyes guide the ears.

161

So our senses interact. But what happens if they disagree? What if our eyes see a speaker form one sound but our ears hear another sound? Surprise: Our brain may perceive a third sound that blends both inputs. Seeing mouth movements for ga while hearing ba, we may perceive da. This is known as the McGurk effect, after one of its discoverers (McGurk & MacDonald, 1976). For all of us, lip reading is part of hearing.

We have seen that our perceptions have two main ingredients: Our bottom-up sensations and our top-down cognitions (such as expectations, attitudes, thoughts, and memories). But let’s return to our starting point in this chapter. In everyday life, sensation and perception are two points on a continuum. We think from within a body. It’s not surprising, then, that the brain circuits processing our bodily sensations may sometimes interact with brain circuits responsible for cognition. The result is embodied cognition. Here are some examples from a few playful experiments.

As we attempt to decipher our world, our brain blends inputs from multiple channels. For many people, an odor—perhaps of mint or chocolate—can evoke a sensation of taste. But in a few rare individuals, the brain circuits for two or more senses become joined in a condition called synesthesia, where one sort of sensation (such as hearing sound) produces another (such as seeing color). Thus, hearing music may activate color-sensitive cortex regions and trigger a sensation of color (Brang et al., 2008; Hubbard et al., 2005). Seeing the number 3 may evoke a taste sensation (Ward, 2003).

* * *

For a summary of our sensory systems, see TABLE 5.3.