Microexpression

Microexpressions were first discovered by Haggard and Isaacs. In their 1966 study, Haggard and Isaacs outlined how they discovered these "micromomentary" expressions while "scanning motion picture films of psychotherapy for hours, searching for indications of non-verbal communication between therapist and patient"[6] Through a series of studies, Paul Ekman found a high agreement across members of diverse Western and Eastern literate cultures on selecting emotional labels that fit facial expressions. Expressions he found to be universal included those indicating anger, disgust, fear, happiness, sadness, and surprise. Findings on contempt are less clear, though there is at least some preliminary evidence that this emotion and its expression are universally recognized.[7] Working with his long-time friend Wallace V. Friesen, Ekman demonstrated that the findings extended to preliterate Fore tribesmen in Papua New Guinea, whose members could not have learned the meaning of expressions from exposure to media depictions of emotion.[8] Ekman and Friesen then demonstrated that certain emotions were exhibited with very specific display rules, culture-specific prescriptions about who can show which emotions to whom and when. These display rules could explain how cultural differences may conceal the universal effect of expression.[9]

In the 1960s, William S. Condon pioneered the study of interactions at the fraction-of-a-second level. In his famous research project, he scrutinized a four-and-a-half-second film segment frame by frame, where each frame represented 1/25th second. After studying this film segment for a year and a half, he discerned interactional micromovements, such as the wife moving her shoulder exactly as the husband's hands came up, which combined yielded rhythms at the micro level.[10]

Years after Condon's study, American psychologist John Gottman began video-recording living relationships to study how couples interact. By studying participants' facial expressions, Gottman was able to correlate expressions with which relationships would last and which would not.[11] Gottman's 2002 paper makes no claims to accuracy in terms of binary classification, and is instead a regression analysis of a two factor model where skin conductance levels and oral history narratives encodings are the only two statistically significant variables. Facial expressions using Ekman's encoding scheme were not statistically significant.[12] In Malcolm Gladwell's book Blink, Gottman states that there are four major emotional reactions that are destructive to a marriage: defensiveness which is described as a reaction toward a stimulus as if you were being attacked, stonewalling which is the behavior where a person refuses to communicate or cooperate with another,[13] criticism which is the practice of judging the merits and faults of a person, and contempt which is a general attitude that is a mixture of the primary emotions disgust and anger.[14] Among these four, Gottman considers contempt the most important of them all.[15]

Microexpressions are typically classified based on how an expression is modified. They exist in three groups:

• Simulated expressions: when a microexpression is not accompanied by a genuine emotion. This is the most commonly studied form of microexpression because of its nature. It occurs when there is a brief flash of an expression, and then returns to a neutral state.[16]
• Neutralized expressions: when a genuine expression is suppressed and the face remains neutral. This type of micro-expression is not observable due to the successful suppression of it by a person.[16]
• Masked expressions: when a genuine expression is completely masked by a falsified expression. Masked expressions are microexpressions that are intended to be hidden, either subconsciously or consciously.[17]

Microexpressions can be difficult to recognize, but still images and video can make them easier to perceive. In order to learn how to recognize the way that various emotions register across parts of the face, Ekman and Friesen recommend the study of what they call "facial blueprint photographs", photographic studies of "the same person showing all the emotions" under consistent photographic conditions.[18] However, because of their extremely short duration, by definition, microexpressions can happen too quickly to capture with traditional photography. Both Condon and Gottman compiled their seminal research by intensively reviewing film footage. Frame rate manipulation also allows the viewer to distinguish distinct emotions, as well as their stages and progressions, which would otherwise be too subtle to identify. This technique is demonstrated in the short film Thought Moments by Michael Simon Toon and a film in Malayalam Pretham 2016[19][20][21] Paul Ekman also has materials he has created on his website that teach people how to identify microexpressions using various photographs, including photos he took during his research period in New Guinea.[22]

Moods differ from emotions in that the feelings involved last over a longer period. For example, a feeling of anger lasting for just a few minutes, or even for an hour, is called an emotion. But if the person remains angry all day, or becomes angry a dozen times during that day, or is angry for days, then it is a mood.[23] Many people describe this as a person being irritable, or that the person is in an angry mood. As Paul Ekman described, it is possible but unlikely for a person in this mood to show a complete anger facial expression. More often just a trace of that angry facial expression may be held over a considerable period: a tightened jaw or tensed lower eyelid, or lip pressed against lip, or brows drawn down and together.[24] Emotions are defined as a complex pattern of changes, including physiological arousal, feelings, cognitive processes, and behavioral reactions, made in response to a situation perceived to be personally significant.[25]

Facial expressions are not just uncontrolled instances. Some may in fact be voluntary and others involuntary, and thus some may be truthful and others false or misleading.[26] Facial expression may be controlled or uncontrolled. Some people are born able to control their expressions (such as pathological liars), while others are trained, for example actors. "Natural liars" may be aware of their ability to control microexpressions, and so may those who know them well; they may have been "getting away" with things since childhood due to greater ease in fooling their parents, teachers, and friends.[27] People can simulate emotion expressions, attempting to create the impression that they feel an emotion when they are not experiencing it at all. A person may show an expression that looks like fear when in fact they feel nothing, or perhaps some other emotion.[28] Facial expressions of emotion are controlled for various reasons, whether cultural or by social conventions. For example, in the United States many little boys learn the cultural display rule, "little men do not cry or look afraid". There are also more personal display rules, not learned by most people within a culture, but the product of the idiosyncrasies of a particular family. A child may be taught never to look angrily at his father, or never to show sadness when disappointed. These display rules, whether cultural ones shared by most people or personal, individual ones, are usually so well-learned, and learned so early, that the control of the facial expression they dictate is done automatically without thinking or awareness.[29]

Involuntary facial expressions can be hard to pick up and understand explicitly, and it is more of an implicit competence of the unconscious mind. Daniel Goleman created a conclusion on the capacity of an individual to recognize their own, as well as others' emotions, and to discriminate emotions based on introspection of those feelings. This is part of Goleman's emotional intelligence. In E.I, attunement is an unconscious synchrony that guides empathy. Attunement relies heavily on nonverbal communication.[30] Looping is where facial expressions can elicit involuntary behavior. In the research motor mimicry there shows neurons that pick up on facial expressions and communicate with motor neurons responsible for muscles in the face to display the same facial expression. Thus displaying a smile may elicit a micro expression of a smile on someone who is trying to remain neutral in their expression.[31]

The amygdala is the emotion center of the brain

Through fMRI we can see the area where these Mirror neurons are located lights up when you show the subject an image of a face expressing an emotion using a mirror. In the relationship of the prefrontal cortex also known as the (executive mind) which is where cognitive thinking experience and the amygdala being part of the limbic system is responsible for involuntary functions, habits, and emotions. The amygdala can hijack the pre-frontal cortex in a sympathetic response. In his book Emotional Intelligence Goleman uses the case of Jason Haffizulla (who assaulted his high school physics teacher because of a grade he received on a test) as an example of an emotional hijacking in which rationality and better judgement can be impaired.[30] This is one example of how the bottom brain can interpret sensory memory and execute involuntary behavior. This is the purpose of microexpressions in attunement and how you can interpret the emotion that is shown in a fraction of a second. The microexpression of a concealed emotion that's displayed to an individual will elicit the same emotion in them to a degree, this process is referred to as an emotional contagion.[31]

Micro Facial expression training tools and subtle Facial expression training tools are software made to develop someone's skills in the competence of recognizing emotion. The software consists of a set of videos that you watch after being educated on the facial expressions. After watching a short clip, there is a test of your analysis of the video with immediate feedback. This tool is to be used daily to produce improvements. Individuals that are exposed to the test for the first time usually do poor trying to assume what expression was presented, but the idea is through the reinforcement of the feedback you unconsciously generate the correct expectations of that expression. These tools are used to develop rounder social skills and a better capacity for empathy. They are also quite useful for development of social skills in people on the autism spectrum.[31] Lie detection is an important skill not only in social situations and in the workplace, but also for law enforcement and other occupations that deal with frequent acts of deception. Microexpression and subtle expression recognition are valuable assets for these occupations as it increases the chance of detecting deception. In recent years it was found that the average person has a 54% accuracy rate in terms of exposing whether a person is lying or being truthful.[32] However, Ekman had done a research experiment and discovered that secret service agents have a 64% accuracy rate. In later years, Ekman found groups of people that are intrigued by this form of detecting deception and had accuracy rates that ranged from 68% to 73%. Their conclusion was that people with the same training on microexpression and subtle expression recognition will vary depending on their level of emotional intelligence.[32]

The sympathetic nervous system is one of two divisions under the autonomic nervous system, it functions involuntarily and one aspect of the system deals with emotional arousal in response to situations accordingly.[33] Therefore, if an individual decides to deceive someone, they will experience a stress response within because of the possible consequences if caught. A person using deception will typically cope by using nonverbal cues which take the form of bodily movements. These bodily movements occur because of the need to release the chemical buildup of cortisol, which is produced at a higher rate in a situation where there is something at stake.[34] The purpose for these involuntary nonverbal cues are to ease oneself in a stressful situation. In the midst of deceiving an individual, leakage can occur which is when nonverbal cues are exhibited and are contradictory to what the individual is conveying.[35] Despite this useful tactic of detecting deception, microexpressions do not show what intentions or thoughts the deceiver is trying to conceal. They only provide the fact that there was emotional arousal in the context of the situation. If an individual displays fear or surprise in the form of a microexpression, it does not mean that the individual is concealing information that is relevant to investigation. This is similar to how polygraphs fail to some degree: because there is a sympathetic response due to the fear of being disbelieved as innocent. The same goes for microexpressions, when there is a concealed emotion there is no information revealed on why that emotion was felt. They do not determine a lie, but are a form of detecting concealed information. David Matsumoto is a well-known American psychologist and explains that one must not conclude that someone is lying if a microexpression is detected but that there is more to the story than is being told.[36] Matsumoto was also the first to publish scientific evidence that microexpressions may be a key to detecting deception.[37]

Despite the prevailing belief among law enforcement and the public that micro-expressions are able to reveal whether a person is being deceitful,[38] there is a lack of empirical evidence to support this claim.[39][40] Research has shown that there is often a disconnect between displayed emotions and felt emotions; in short, deception does not necessarily produce negative emotions and negative emotions do not necessarily signal deception.[41] In addition, microexpressions do not occur often enough to be useful.[38] In one of the few studies of microexpressions, researchers found that only 2% of emotional expressions coded could be considered microexpressions and they appeared equally for truth-tellers and liars.[38][39] Other studies have found that liars and truth-tellers exhibit different responses than expected:[42][43] in a concealed information test, Pentland and colleagues found that liars showed less contempt and more intense smiles than truthful people.[43] This counters the fundamental idea behind microexpressions, which suggests that it is impossible for a liar to conceal their true nature, as evidence of their guilt "leaks" out through these expressions.[42][44] Taken together, their findings suggest that micro-expressions do not occur frequently enough to be detectable, neither are they consistent enough to distinguish liars from truth-tellers.

Universal Facial Expressions

A significant amount of research has been done in respect to whether basic facial expressions are universal or are culturally distinct. After Charles Darwin had written The Expression of the Emotions in Man and Animals it was widely accepted that facial expressions of emotion are universal and biologically determined.[45] Many writers have disagreed with this statement. David Matsumoto however agreed with this statement in his study of sighted and blind Olympians. Using thousands of photographs captured at the 2004 Olympic and Paralympic Games, Matsumoto compared the facial expressions of sighted and blind judo athletes, including individuals who were born blind. All competitors displayed the same expressions in response to winning and losing.[46] Matsumoto discovered that both blind and sighted competitors displayed similar facial expression, during winnings and loss. These results suggest that our ability to modify our faces to fit the social setting is not learned visually.[46] Matsumoto also has training tools he has created on his website that teaches people how to identify micro and subtle facial expressions of emotion.[47]

Microexpressions and associated science are the central premise for the 2009 television series Lie to Me, based on discoveries of Paul Ekman. The main character uses his acute awareness of microexpressions and other body language clues to determine when someone is lying or hiding something.

They also play a central role in Robert Ludlum's posthumously published The Ambler Warning, in which the central character, Harrison Ambler, is an intelligence agent able to recognize them. Similarly, one of the main characters in Alastair Reynolds' science fiction novel, Absolution Gap, Aura, can easily read microexpressions.

In The Mentalist, the main character, Patrick Jane, can often tell when people are being dishonest. However, specific reference to microexpressions is only made once in the 7th and final season.

In the 2015 science fiction thriller Ex Machina, Ava, an artificially intelligent humanoid, surprises the protagonist, Caleb, in their first meeting, when she tells him "Your microexpressions are telegraphing discomfort."

Though the study of microexpressions has gained popularity through popular media, studies show it lacks internal consistency in its conceptual formation.[48]

Maria Hartwig, professor of Psychology at John Jay College of Criminal Justice, argues that it has led to wrongful imprisonment of suspects who were aggressively interrogated due to perceived micro expressions. [49]

A 2016 article in Nature explains that it is possible to mask involuntary expressions with fake expressions, and that in real world situations, over 40% of the time we can not tell the difference. [50]

Judee K. Burgoon argues in a 2018 Frontiers in Psychology opinion that micro expressions theory presumes that people feel detectible emotions always connected to the same thoughts or motivations. But what if, for example, people feel happy rather than guilty about deceiving others? [51] Burgoon also cites studies showing that micro-expressions are rare:

In one of the very few investigations of microexpression frequency, Porter and ten Brinke (2008) coded 700 high-stakes genuine and falsified emotional expressions and found only 2% were microexpressions.

and that they seldom result in arrests when implemented at places like airports:

testimony to the U.S. Congress revealed that only 0.6% out of 61,000 passenger referrals to law enforcement in 2011 and 2012 resulted in arrests (U.S. Government Accountability Office, 2013)

• Interpersonal deception theory
• Nonverbal communication
• Microaggression
• Facecrime
• Silent Talker Lie Detector
• Tell (poker)

• Matthew Hertenstein (2015). The Tell: The Little Clues That Reveal Big Truths about Who We Are. Basic Books. ISBN 978-0465036592.

• Lying Is Exposed By Microexpressions We Can't Control, Science Daily, May 2006
• The Naked Face
• Facial Expressions Test based on "The Micro Expression Training Tool"
• "A Look Tells All" in Scientific American Mind October 2006
• Microexpressions Complicate Face Reading, by Medical News Today August 2007
• Deception Detection, American Psychological Association