Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Cultural difference of simplified facial expressions

1 The University of Nagasaki, Manabino 1-1, Nagayo, Nagasaki, Japan ↗
2 Nagasaki University, Bunkyomachi 1-14, Nagasaki, Japan ↗
3 Kyoto Institute of Technology, Matsugasaki, Sakyo, Kyoto, Japan ↗
Publication year
Digital Object Identifier10.54941/ahfe1001141

Abstract

In this research, the authors succeeded in creating facial expressions made with the minimum necessary elements for recognizing a face. The elements are two eyes and a mouth made using precise circles, which are transformed to make facial expressions geometrically, through rotation and vertically scaling transformation. The facial expression patterns made by the geometric elements and transformations were composed employing three dimensions of visual in-formation that had been suggested by many previous researches, slantedness, of the mouth, openness of the face, and slantedness of the eyes. The authors also surveyed cultural differences of impressions of such simplified faces.

Human Systems Engineering and Design (IHSED 2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8987-0

INTRODUCTION

When human beings communicate with each other, their faces convey the most important and richest information, and their facial expressions are by far the most essential element for understanding the other’s emotion. Various elements such as the eyes, mouth, and nose are contained in the face, and these are transformed to form facial expressions. For many primates including human beings, the eyes are one of the most important factor used to recognize a face as a face. Visual preference for facial figures and research on perceptions of upside-down faces are well known regarding facial recognition. These researches have also suggested that the eyes and the mouth are the most important elements for the primates. (Darwin 1872, Diamond et al. 1986, Ellis et al. 1979, Keating et al. 1975, Shepherd et al. 1981, Slater et al. 1993, Yin 1969, Young et al. 1985) There have been two theories regarding the recognition of facial expressions: the category perception theory and the dimension theory. The category perception theory states that human beings judge the meaning of facial expressions through 7 plus/minus 2 universal categories common to all human beings. This theory is based on an idea stemmed from evolutionary theory. This theory indicates that facial expression is a discrete (not continuous) status. In terms of emotional categories, the six basic emotions (happy, angry, sad, disgust, fear, and surprise) advocated by Ekman and many other researchers are the most typical. Basic emotions synchronize with physiological responses and signals to the body such as facial expressions, and it is proposed that facial expressions can be classified under one of the six basic emotions without exception irrespective of culture. (Ekman 2005, Smith et al. 1985, Yamada 2000, Yin 1969)

The dimension theory proposes that facial expressions are points in a continuous space of possible facial expressions, and people recognize the emotions after they locate the points. Dimension theory begins with Schlosberg’s theory of the dimension of emotion, for example the circular ring model comprising two dimensions, namely pleasant vs. unpleasant and attention vs. rejection, and the circular cone model comprising the previous two dimensions plus tension vs. sleep. Since this research, many researchers have discussed such affective meaning dimensions and have repeatedly encountered three dimensions: the pleasantness dimension (pleasant vs. unpleasant), the attention vs. rejection dimension, and the activeness (awareness) dimension (aware vs. asleep). The circumplex model comprising a pleasantness dimension and an awareness dimension suggested by Russell has been validated in terms of its universality and robustness by many previous researches. (Russel 1997, Schlosberg 1954, Shah et al. 2003, Smith et al. 1985, Takehara et al. 2001) Yamada conducted a study to clarify visual information (the physical variable) related to the cognition of facial expressions using a line-drawing figure in which eight points of the eyebrows, eyes and mouth are manipulated. From the results, two physical variables have been found: slantedness, meaning the curve and indication of face

Human Systems Engineering and Design (IHSED 2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8987-0 elements; and openness and curvedness, meaning the level of curve and openness of facial elements. Based on this knowledge, they proposed that there are three processes used to cognize facial expressions: (1) acknowledgement of visual information (the physical variable) of the face, (2) evaluation of affective meaning based on the physical variable, and (3) judgment of the emotional category based on affective meaning. (Watanabe 2001, Yamada 2000)

MINIMAL FACIAL EXPRESSIONS The authors validate the relationship between the physical variable and the affective variable discovered by previous researches using geometrical faces, and to apply the existing knowledge to robot facial design. Clarification of this relationship is one of the most important aspects for research on cognition of facial expression. The geometrical face used for this article is comprised of the minimum necessary elements for recognizing the face, and the elements are transformed geometrically to form the various facial expression patterns. The author introduces the physical variables, slantedness and openness, to the geometrical transformation. These facial patterns are classified by basic emotions, and evaluation of the relationship between the physical and affective variables is conducted by applying principal component analysis to the facial expression space centered on physical variables. Based on the results of the research, the author finally developed a model to create facial expressions.

By using simplified faical design, and employing geometrical transformation, the relationship between the facial expression will be shown clearly. In addition, the result will be easily used for design of humanodis and other human-like robotic designs.

Figure 1. Left: Basic face before transformation; Right: Parameters of eyes and mouth
Original PDF Figure 1.

Human Systems Engineering and Design (IHSED 2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8987-0

Figure 2. Left: Transformation of eyes by each parameter; Right: Transformation of mouth by each parameter The Design
Original PDF Figure 2.

The elements composing the face are limited to just three: the two eyes and the mouth, composed of precise circles and placed on the top of the upside-down triangle, the face figure (Fig. 1). Facial expression patterns are made by adding transformations to the three elements within the two parameters of slantedness and openness, comprising the physical variable for cognition of facial expressions. Slantedness of the eyes is a parameter that expresses the curve according to the opening of the eyes by their rotational deformation, while slantedness of the mouth expresses the rise or fall of the corners of the mouth. Openness is a parameter that expresses the change in the opening state of the eyes and mouth by the change in the vertically scaling transformation of the precise circle. Fig. 2 Left and Right show the changes to the eyes and mouth according to the value of the parameters. The eyes make 19 patterns and mouth makes 7 patterns as shown in Fig. 2 Left and Right, making a total of 133 expression patterns. The facial expression is assumed to be completely facing the observer, and all faces are symmetrical.

Thus, each facial expression is defined by four values: two parameters of the eyes and two of the mouth, namely slantedness and openness respectively. So, the coordinates of one face can be shown as in the following mathematical expression: fi = (Esi, Eoi, Msi, Moi)T.

The above mathematical expression shows the coordinates of the i-th facial expression, where Es and Eo represent the eyes' slantedness and openness respectively, and Ms and Mo represent the mouth's slantedness and openness respectively. By principal component analysis we would be able to reduce dimensions of the space that fi spans and get f’i ~ fi where dim f’ <dim f.

Human Systems Engineering and Design (IHSED 2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8987-0

Classification of Facial Expression Patterns The obtained expression patterns are classified by emotional category. For the emotional categories, the author uses the six basic emotions (happy, angry, sad, disgust, fear, and surprised) advocated by Ekman and many other researchers. In this article, the author additionally utilizes a neutral (pleasant) emotion, displaying no emotion but showing a pleasant expression, and a neutral (unpleasant) emotion, showing no emotion but displaying an unpleasant expression, in order to research the facial expressions of neutral emotions. Clarification of the neutral face is important for robot faces, especially for humanoid robots without the function of forming facial expressions. In total, the eight emotional categories are defined.

Classification of facial expression patterns is conducted through an identification task. There are three rules for this task as follows: (1) the answerer must choose one emotion category for one face; (2) the answerer may adopt the same emotion category for more than one face; (3) the answerer does not need to select an emotion category if no category fits the face.

Distributing Facial Expression Patterns in Mathematical Space As described in 2.1, each expression is defined by four-dimensional coordinates. Based on these coordinates, the author distributes the classified facial expression patterns in four-dimensional space. The facial patterns distributed in space are the faces selected by more than p/2 answerers (p is the maximum number of answerers selecting the face as the emotion category).

After creating the facial expression space, analysis is applied to the space and the dimensions of the space are reduced to render it perceptible to the human eye. Through this analysis and visualization, the author can observe the difference in spatial distribution and parametric values of each facial pattern.

RESULT OF CLASSIFICATION AND SPATIAL DISTRIBUTION OF FACIAL EXPRESSIONS From the results of the principal component analysis, the face distribution space is composed of three variables: openness of the mouth, slantedness of the eyes, and openness of the face. Moreover, this result almost completely concurs with expression distributions in the space composed of the emotional meaning dimension in previous research, and shows similarity to the results of expression cognition research using faces of actual human beings.

Human Systems Engineering and Design (IHSED 2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8987-0

In addition, the research of the past is consolidated, and 10 basic facial expressions— happy, angry, angry*, sad, disgust, fear, fear*, surprised, neutral (pleasant), and neutral (unpleasant)—are advocated.

Figure 3. Left: Projection plane of three-dimensional space for facial expression composed of slantedness of the mouth and openness of the face; Center: Projection plane of three-dimensional space for facial expressions composed of the slantedness of the eyes and the openness of the face; Right: Circumplex mode constructed by Takehara & Suzuki (2001).
Original PDF Figure 3.

Three-Dimensional Space for Distribution of Face The identification task was undertaken by 140 men and women ranging in age from their teens to their 60s. (average age in their 20s). The results of the principal component analysis of four-dimensional space are shown in Table 1. Four- dimensional space can be reduced to three- dimensional space from the result of the cumulative proportion from principal component-1 to principal component-2, shown to be 0.807. In addition, the value of each principal component is shown in Table 2. Principal component-1 was be judged to mean the slantedness of the mouth; principal component-2 shows the slantedness of mouth; and principal component-3 shows the openness of the face, meaning openness of both eyes and mouth, according to this value. This result corresponded to the three types of visual information (the visual variable) that had been obtained by previous researches. (Ekman 2005, Russel 1997)

To assess the relationship between the emotional meaning dimension obtained by this research and the visual information dimension obtained by previous researches, the author compared two planes, a projection plane of three- dimensional space obtained by this research comprising slantedness of mouth and openness of face, and a plane comprising the pleasantness dimension and the activeness dimension, which have strong relationships with the slantedness of the mouth and the openness of the face. The former plane is shown in Fig. 4 Left, and the latter in Fig. 4 Right. Each facial expression is assigned a weight according to the number of selections. An important point in Fig. 4 Center and Fig. 4 Right is that the average coordinates of each emotion category took these weights into consideration. (Ekman 2005, Yamada 2000) Comparing Fig. 4 Left and Fig. 4 Right, the distribution of happy, surprised, and fear can be seen to almost correspond, and in addition, angry, sad, and disgust were also

Human Systems Engineering and Design (IHSED 2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8987-0 seen to correspond in terms of closeness of distribution. Moreover, fear could be seen to separate into two clusters in terms of distribution, which could be read in Fig. 4 Left. As mentioned above, the author determined that the distribution of the facial expressions in three-dimensional space obtained through this article was valid.

In addition, angry, sad and disgust were separately observed by constructing and observing a projection plane of three-dimensional space comprising slantedness of the eyes and openness of the face (Fig. 4 Right). From this result, by considering the third dimension, the slantedness of the eyes, the distribution of each facial expression was easy to separate and read. Thus the author discovered that the visual information dimension (physical variable) comprises three variables for cognizance of facial expressions.

Table 1: Result of principal component analysis Principal component PC1 PC2 PC3 PC4
Original PDF Table 1:
Table 2: Meaning of each principal component Principal component PC1 PC2 PC3
Original PDF Table 2:

Eight Facial Expressions and Two Neutral Facial Expressions In addition to the observation of face distributions in space, the author found 10 basic facial expressions, eight basic facial expressions: happy, angry, angry*, sad, disgust, fear, fear*, and surprised; and two neutral facial expressions: neutral (pleasant) and neutral (unpleasant), according to the parametric values and actual facial patterns. The difference between angry and angry* is especially apparent in terms of the openness of the eyes and mouth, so angry can be separated in terms of the facial expression showing the anger emotion. Moreover, it can be seen that fear and fear* can be separated because the slantedness of the eyes and the mouth indicate an opposite value.

Human Systems Engineering and Design (IHSED 2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8987-0

DISCUSSION AND CONCLUSION Through the facial expression pattern presented in this article, the visual variable dimension was obtained, namely, the slantedness of the eyes, the slantedness of the mouth, and the openness of eyes and mouth. It was found that there is a strong relationship between the slantedness of the mouth and the pleasantness dimension, and the openness of the eyes and mouth and activeness.

Based on observation of the distribution in three-dimensional space, the slantedness of the eyes is an effective means of discerning the distribution areas of the facial expressions, especially angry, sad and disgust. Thus the third effective variable's dimension serves to aid cognition of facial expression, and bears a strong relationship with the judgment of angry, sad and disgust. The neutral (pleasant) and neutral (unpleasant) expressions were added to the classification of facial expressions. These expressions were found even though they do not show a specific emotion.

The authors succeeded in creating facial expressions made with the minimum elements for recognizing a face. The elements used for the face were two eyes and a mouth made by geometrically transformed circles through rotation and slanting. The facial expressions comprised three dimensions of visual information that had been suggested by major previous researches. The results of this research indicate that human beings can classify expression patterns of minimal faces to particular emotional categories just as they would with an actual human face.

ACKNOWLEDGMENTS

This work was supported by Grant-in-Aid for Scientific Research on Innovative Areas No.18H04203 “Construction of the Face-Body Studies in Transcultural Conditions.”

REFERENCES

Charles Darwin. (1872). The expression of the emotions in man and animals. London: John Murray, London. Rhea Diamond and Susan Carey. (1986). Why faces are and are not special: An effect of

expertise. Journal of Experimental Psychology, 115. Paul Ekman. (2005). Basic Emotions. New York: John Wiley & Sons, Ltd, 45–60. Hadyn Ellis, John Shepherd, and Graham Davies. (1979). Identification of familiar and unfamiliar faces from internal and external features: Someimplications for theories of face recognition. Perception 8, 431–9. Caroline F. Keating and E. Gregory Keating. (1975). Visual scan patterns of rhesus monkeys

viewing faces. Perception 11, 133–140. James A. Russell. (1997). Reading emotions from and into faces: Resurrecting a dimensionalcontextual perspective. Cambridge University Press, 295–320.

Human Systems Engineering and Design (IHSED 2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8987-0

Harold Schlosberg. (1954). Three dimensions of emotion. Psychological Review 10, 5, 81–

88. Rebecca Shah and Michael Lewis. (2003). Locating the neutral expression in the facialemotion space. Visual Cognition 10, 5, 549–566. John W. Shepherd, Graham Davies, and Hayn D. Ellis. (1981). Studies of cue saliency. In G.

Davies, H. Ellis, & J. Shepherd (Eds.). Perceiving and remembering faces. Alan Slater, Mark H. Johnson, and John Morton. (1993). Biology and cognitive development: The case of face recognition. Oxford: Blackwell Scientific Publications. Craig A. Smith and Phoebe C. Ellsworth. (1985). Patterns of cognitive appraisal in emotion.

Journal of Personality and Social Psychology 48, 4, 813–838. Takuma Takehara and Naoto Suzuki. (2001). Robustness of the two-dimensional structure of recognition of emotionality. Perceptual and Motor Skills 93, 3, 739–753. Nobuyuki Watanabe. (2001). Research of distribution of facial expressions in emotional

space by using affect grid method [in Japanese]. Japanese Psychological Research 65, 274. Hiroshi Yamada. (2000). Expression model for process of perceptive estimation of facial expression [in Japanese]. Japanese Psychological Review 43, 2, 245–255. Robert K. Yin. (1969). Looking at upside-down faces. Journal of Experimental Psychology

81, 141–145. Andrew W. Young, Dennis C. Hay, Kathryn H. McWeeny, Brenda M. Fluid, and Andrew W.

Ellis. (1985). Matching familiar and unfamiliar faces on internal and external features. Perception 14, 737–746.

Human Systems Engineering and Design (IHSED 2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8987-0

Cover of the proceedings volume Human Systems Engineering and Design (IHSED2021): Future Trends and Applications

Explore the complete digital volume

Human Systems Engineering and Design (IHSED2021): Future Trends and Applications

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-7923-8987-0

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →