Leonardo the robot may not be able to feel, but he sure can smile. And wink. And nod. Indeed, one of the reasons why people who interact with Leonardo find it so hard to think of him as a machine is that Leonardo expresses emotions that he does not actually have. An emotional expression is an observable sign of an emotional state, and while robots can be taught to exhibit them, human beings seem to do it quite naturally.

Our emotional states express themselves in a wide variety of ways. For example, they change the way we talk—from our intonation and inflection to the loudness and duration of our speech—and research shows that listeners can infer emotional states from vocal cues alone with better-than-chance accuracy (Banse & Scherer, 1996; Frick, 1985). Observers can also estimate emotional states from the direction of a person’s gaze, the rhythm of their gait, or even from a brief touch on the arm (Dael, Mortillaro, & Scherer, 2012; Dittrich et al., 1996; Hertenstein et al., 2009; Keltner & Shiota, 2003; Wallbott, 1998). In some sense, we are walking, talking advertisements for what is going on inside us.

Of course, no part of your body is more exquisitely designed for communicating emotion than your face. Underneath your face lie 43 muscles that are capable of creating more than 10 000 unique configurations, which enable you to convey information about your emotional state with an astonishing degree of subtlety and specificity (Ekman, 1965). Psychologists Paul Ekman and Wallace Friesen (1971) spent years cataloguing the muscle movements of which the human face is capable. They isolated 46 unique movements, which they called action units, and they gave each one a number and a name, such as “cheek puffer,” “dimpler,” and “nasolabial deepener” (which, coincidentally enough, are also the names of heavy metal bands). Research has shown that combinations of these action units are reliably related to specific emotional states (Davidson et al., 1990). For example, when we feel happy, our zygomatic major (a muscle that pulls our lip corners up) and our obicularis oculi (a muscle that crinkles the outside edges of our eyes) produce a unique facial expression that psychologists describe as “action units 6 and 12” and that the rest of us simply call smiling (Ekman & Friesen, 1982; Frank, Ekman, & Friesen, 1993; Steiner, 1986).

323

Why are our emotions written all over our faces? In 1872, Charles Darwin published The Expression of the Emotions in Man and Animals, in which he speculated about the evolutionary significance of emotional expression. Darwin noticed that human and nonhuman animals share certain facial and postural expressions, and he suggested that these expressions were meant to communicate information about internal states. It is not hard to see how such communications could be useful (Shariff & Tracy, 2011). For example, if a dominant animal can bare its teeth and communicate the message, “I am angry at you,” and if a subordinate animal can lower its head and communicate the message, “I am afraid of you,” then the two can establish a pecking order without actually spilling any blood. Darwin suggested that emotional expressions are a convenient way for one animal to let another animal know how it is feeling and therefore how it is prepared to act. In this sense, emotional expressions are a bit like the words of a nonverbal language.

Of course, a language only works if everybody speaks the same one, which is why Darwin advanced the universality hypothesis, which suggests that emotional expressions have the same meaning for everyone. In other words, every human being naturally expresses happiness with a smile, and every human being naturally understands that a smile signifies happiness.

There is some evidence for Darwin’s hypothesis. For example, people who have never seen a human face make the same facial expressions as those who have. Congenitally blind people smile when they are happy (Galati, Scherer, & Ricci-Bitt, 1997; Matsumoto & Willingham, 2009), and 2-day-old infants make a disgust face when bitter chemicals are put in their mouths (Steiner, 1973, 1979). In addition, people are fairly accurate when judging the emotional expressions of members of other cultures (Ekman & Friesen, 1971; Elfenbein & Ambady, 2002; Frank & Stennet, 2001; Haidt & Keltner, 1999). Not only do Chileans, Americans, and Japanese all recognize a smile as a sign of happiness and a frown as a sign of sadness, but so do members of preliterate cultures. In the 1950s, researchers took photographs of Westerners expressing anger, disgust, fear, happiness, sadness, and surprise (see FIGURE 8.7) and showed them to members of the South Fore, a people who lived a Stone Age existence in the highlands of Papua New Guinea and who at that point had had little contact with the modern world. Researchers asked these participants to match each photograph to a word (such as “happy” or “afraid”) and discovered that the South Fore made matches that were essentially the same as those made by Americans. (The one exception to this rule was that the Fore had trouble distinguishing expressions of surprise from expressions of fear, perhaps because for people who live in the wild, surprises are rarely pleasant.) Evidence of this sort has convinced many psychologists that facial displays of at least six emotions—anger, disgust, fear, happiness, sadness, and surprise—are universal. And a few other emotions—embarrassment, amusement, guilt, shame, and pride—may have universal patterns of facial expression as well (Keltner, 1995; Keltner & Buswell, 1996; Keltner & Haidt, 1999; Keltner & Harker, 1998; Tracy et al., 2013).

324

But not all psychologists are convinced. For example, recent research (Gendron et al., in press) shows that like the South Fore, members of an isolated tribe called the Himba can match faces to emotion words just as Americans do. But when the Himba are instead asked to match faces that are “feeling the same way” to each other, they produce matches that are quite unlike those produced by their American counterparts. Studies such as these suggest that the universality hypothesis may be stated too strongly. At present, we can say with confidence that there is considerable agreement among all humans about the emotional meaning of many facial expressions, but that this agreement is not perfect.

Members of different cultures express many emotions in the same ways, but why? After all, they do not speak the same languages, so why do they smile the same smiles and frown the same frowns? The answer is that words are symbols, but facial expressions are signs. Symbols are arbitrary designations that have no causal relationship with the things they symbolize. English speakers use the word cat to indicate a particular animal, but there is nothing about felines that actually causes this particular sound to pop out of our mouths, and we are not surprised when other human beings make different sounds—such as popoki or gatto—to indicate the same thing. In contrast, facial expressions are not arbitrary symbols of emotion. They are signs of emotion because signs are caused by the things they signify. The feeling of happiness causes the contraction of the zygomatic major; thus that contraction is a sign of that feeling in the same way a footprint in the snow is a sign that someone walked there.

Of course, just as a symbol (bat) can have more than one meaning (wooden club or flying mammal), so, too, can a sign. Is the woman in the photograph at right feeling joy or sorrow? In fact, these two emotions often produce rather similar facial expressions, so how do we tell them apart? Research suggests that one answer is context. When someone says, “The centrefielder hit the ball with the bat,” the sentence provides a context that tells us that bat means “club” and not “mammal.” Similarly, the context in which a facial expression occurs often tells us what that expression means (Aviezer et al., 2008; Barrett, Mesquita, & Gendron, 2011; Meeren, van Heijnsbergen, & de Gelder, 2005). It is difficult to tell what the woman in the photograph is feeling. But if you see the photograph in context, you will have no trouble. Indeed, when you return to this page, you may well wonder how you could ever have had any trouble.

Our emotional experiences cause our emotional expressions, but it also works the other way around. The facial feedback hypothesis (Adelmann & Zajonc, 1989; Izard, 1971; Tomkins, 1981) suggests that emotional expressions can cause the emotional experiences they signify. For instance, people feel happier when they are asked to make the sound of a long e or to hold a pencil in their teeth (both of which cause contraction of the zygomatic major) than when they are asked to make the sound of a long u or to hold a pencil in their lips (Strack, Martin, & Stepper, 1988; Zajonc, 1989) (see FIGURE 8.8). Similarly, when people are instructed to arch their brows they find facts more surprising, and when instructed to wrinkle their noses they find odours less pleasant (Lewis, 2012). These things happen because facial expressions and emotional states become strongly associated with each other over time (remember Pavlov?), and eventually each can bring about the other. These effects are not limited to the face. For example, people feel more assertive when instructed to make a fist (Schubert & Koole, 2009) and rate others as more hostile when instructed to extend their middle fingers (Chandler & Schwarz, 2009).

325

The fact that emotional expressions can cause the emotional experiences they signify may help explain why people are generally so good at recognizing the emotional expressions of others. Many studies show that people unconsciously mimic other people’s body postures and facial expressions (Chartrand & Bargh, 1999; Dimberg, 1982). When we see someone smile (or even when we read about someone smiling), our zygomatic major contracts ever so slightly—as yours almost surely is right now (Foroni & Semin, 2009). (By the way, the tendency to ape the facial expressions of our interaction partners is so natural that, yes, even apes do it [Davila Ross, Menzler, & Zimmermann, 2008].) Because facial expressions can cause the emotions they signify, mimicking another person’s facial expression allows us to feel what they are feeling and therefore identify their emotions.

326

What is the evidence for this? First, people find it difficult to identify other people’s emotions when they are unable to make facial expressions of their own, for example, if their facial muscles are paralyzed with Botox (Niedenthal et al., 2005). People also find it difficult to identify other people’s emotions when they are unable to experience emotions of their own (Hussey & Safford, 2009; Pitcher et al., 2008). For example, people with amygdala damage do not normally feel fear and anger, and are typically poor at recognizing the expressions of those emotions in others (Adolphs, Russell, & Tranel, 1999). On the flip side, people who are naturally quite good at figuring out what others are feeling tend to be natural mimics (Sonnby-Borgstrom, Jonsson, & Svensson, 2003), and their mimicry seems to pay off: Negotiators who mimic the facial expressions of their opponents earn more money than those who do not (Maddux, Mullen, & Galinsky, 2008). In the Consciousness chapter, you learned about mirror neurons: neurons that fire when an animal (or person) executes a particular action, and when the animal sees someone else perform the same action. You may not be surprised to know that mirror neurons are also thought to play a critical role in displaying and in understanding facial expressions of emotion.

Our emotional expressions can communicate our feelings truthfully—or not. When a friend makes a sarcastic remark about our haircut, we truthfully express our contempt with an arched brow and a reinforcing hand gesture; but when our boss makes the same remark, we swallow hard and fake a pained smile. Our knowledge that it is permissible to show contempt for a peer but not a superior is a display rule, which is a norm for the appropriate expression of emotion (Ekman, 1972; Ekman & Friesen, 1968). Obeying a display rule requires several techniques:

Although people in different cultures use many of the same techniques, they use them in the service of different display rules. For example, in one study, Japanese and American university students watched an unpleasant video of car accidents and amputations (Ekman, 1972; Friesen, 1972). When the students did not know that the experimenters were observing them, Japanese and American students made similar expressions of disgust, but when they realized that they were being observed, the Japanese students (but not the American students) masked their disgust with pleasant expressions. In many Asian countries it is considered rude to display negative emotions in the presence of a respected person, and so citizens of these countries tend to mask or neutralize their expressions. The fact that different cultures have different display rules may also help explain the fact that people are better at recognizing the facial expressions of people from their own cultures (Elfenbein & Ambady, 2002).

327

Our attempts to obey our culture’s display rules do not always work out so well. Darwin (1899/2007) noted that “those muscles of the face which are least obedient to the will, will sometimes alone betray a slight and passing emotion” (p. 64). Anyone who has ever watched the loser of a beauty pageant congratulate the winner knows that voices, bodies, and faces are “leaky” instruments that often betray a person’s emotional state. Even when people smile bravely to mask their disappointment, for example, their faces tend to express small bursts of disappointment that last just 1/5 to 1/25 of a second (Porter & ten Brinke, 2008). These micro-expressions happen so quickly that they are almost impossible to detect with the naked eye. Four other features that are more readily observable seem to distinguish between sincere and insincere facial expressions (Ekman, 2003).

Our emotions do not just leak on our faces: They leak all over the place. Research has shown that many aspects of our verbal and nonverbal behaviour are altered when we tell a lie (DePaulo et al., 2003). For example, liars speak more slowly, take longer to respond to questions, and respond in less detail than do those who are telling the truth. Liars are also less fluent, less engaging, more uncertain, more tense, and less pleasant than truth-tellers. Oddly enough, one of the telltale signs of a liar is that his or her performances tend to be just a bit too good. A liar’s speech lacks the little imperfections that are typical of truthful speech, such as superfluous detail (“I noticed that the robber was wearing the same shoes that I saw on sale last week at The Bay and I found myself wondering what he paid for them”), spontaneous correction (“He was six feet tall … well, no, actually more like six-two”), and expressions of self-doubt (“I think he had blue eyes, but I am really not sure”).

328

Given the reliable differences between sincere and insincere expressions, you might think that people would be quite good at telling one from the other. In fact, studies show that people are dreadful at this, and under most circumstances perform barely better than chance (DePaulo, Stone, & Lassiter, 1985; Ekman, 1992; Zuckerman, DePaulo, & Rosenthal, 1981; Zuckerman & Driver, 1985). One reason for this is that people have a strong bias toward believing that others are sincere, which explains why people tend to mistake liars for truth-tellers more often than they mistake truth-tellers for liars (Gilbert, 1991). A second reason is that people do not seem to know what they should attend to and what they should ignore (Vrij et al., 2011). For instance, people think that fast talking is a sign of lying when actually it is not, and that slow talking is not a sign of lying when actually it is. People are bad lie detectors who do not even know how bad they are: The correlation between a person’s ability to detect lies and the person’s confidence in that ability is essentially zero (DePaulo et al., 1997).

When people cannot do something well (e.g., adding numbers or picking up 10-tonne rocks), they typically turn the job over to machines (see FIGURE 8.11). Can machines detect lies better than we can? The answer is yes, but that is not saying very much. The most widely used lie detection machine is the polygraph, which measures a variety of physiological responses that are associated with stress, which people often feel when they are afraid of being caught in a lie. A polygraph can detect lies at a rate that is significantly better than chance, but its error rate is still too high to make it a reliable lie detector. For example, let us imagine that 10 of the 10 000 people coming through a particular airport are terrorists and that when hooked up to a polygraph, they all claim to be innocent. A polygraph that was set to maximum sensitivity would catch 8 of the 10 terrorists lying, but it would also mistakenly catch 1598 innocent people. A polygraph set to minimum sensitivity would mistakenly catch just 39 innocent people, but would only catch 2 of the 10 real terrorists. And these numbers assume that terrorists do not know how to fool a polygraph, which is something that people can, in fact, be trained to do. No wonder the American National Academy of Sciences (2003) warned, “Given its level of accuracy, achieving a high probability of identifying individuals who pose major security risks in a population with a very low proportion of such individuals would require setting the test to be so sensitive that hundreds, or even thousands, of innocent individuals would be implicated for every major security violator correctly identified” (p. 6). In short, neither people nor machines are particularly good at lie detection, which is why lying remains such a popular sport.

329

330

When someone is lying, he or she usually wants to conceal that fact from others. Individuals may wish to conceal their internal states in many other situations as well: feelings of disgust caused by food served at a dinner party, anxiety related to asking someone out on a date, or feelings of jealousy when a romantic partner talks enthusiastically about a new friend. Although it may feel like your innermost feelings are on public display at such awkward times, in fact, they are not. For example, public speakers tend to believe that their nervousness is obvious to their audience, and this makes them even more nervous. In fact, studies demonstrate that a speaker’s emotional state is not nearly as evident to the audience as the speaker believes (Savitsky & Gilovich, 2003). Reassuring speakers that the audience cannot perceive their nervousness leads to less anxiety and a better performance (Savitsky & Gilovich, 2003).

331