People communicate smoothly through emotional expressions. Therefore, we consider that emotional expression by robots is necessary for human-robot coexistence as well. Various studies have been conducted on emotional expression by robots. In our previous study, we created a robot named "Tilting Robot" that tilts back and forward, and investigated the effect of the color of the clothing worn by the robot on its emotional expression. The results showed that the intensity of the emotional expression may change depending on the color of the clothing and the speed of the robot's motion. In this paper, we investigate the effect of using light-emitting EL sheets as a material for the robot's clothing. Many robots have been developed that express emotional expression by emitting light from their eyes and cheeks, but there is no research on emotional expression by emitting light from clothing. In addition, light-emitting clothing is easy to implement and does not cause discomfort when worn by a robot, so if light-emitting clothing is effective in emotional expression, it has great potential for application. In the experiment, the robot's clothing was colored red, green, blue, and white, and a light-emitting material (EL sheet) and a non-light-emitting material (felt) were prepared for each of the four colors. Also, for the light-emitting material, two conditions were set: one was to emit light continuously during the robot's motion (constant light-emitting condition), and the other was to emit light in the middle of the robot's motion (midway light-emitting condition). Therefore, there are three conditions for the clothing: the felt condition, the constant light-emitting condition, and the midway light-emitting condition. The robot made a total of five motions, including back and forward tilting motions, fast and slow motions, and no motion while in an upright posture. Subjects observed a total of 60 materials that were a combination of all conditions and motions on a monitor, and evaluated each material.The results of the experiment showed that, first of all, when the robot leaned backward quickly, subjects expressed an emotional expression of surprise, regardless of the type of clothing. In addition, when the motion was slow, the robot also expressed surprise in the mid-lighting condition. These results indicate that the intensity of instantaneous emotional expressions such as surprise depends on the speed of motion, and that the midway light may be used to express the emotion. In addition, it was found that the forward-tilting motion can express the emotional expression of sadness regardless of the type of clothing.Next, when we focused on the color of the clothing, we found that the color red evoked an emotional expression of anger, and that the two luminous conditions evoked anger more strongly than the felt condition, in which no luminescence was emitted. The blue color showed the same level of emotional expression of sadness in all clothing conditions. These results indicate that emotional expression can be achieved by changing the color of clothing, as in previous studies, but that the luminescence of the clothing may make it more pronounced.
In psychological communication, the suppression of emotional expression is considered to have a negative impact. Bulter et al. (2003) showed that suppressing emotional expression during social interactions hinders com- munication and increases the blood pressure response of the suppressor’s interlocutor. Gross and John (2003) showed that suppressing emotional expression was associated with lower interpersonal relationships and nega- tively correlated with happiness. Balzarotti et al. (2016) showed that positive reappraisal and refocus on planning were positively correlated with hap- piness, while negative effects, such as rumination, catastrophizing, and self-blame, were negatively correlated with happiness. Suppressing negative emotional expressions like rumination, catastrophizing, and self-blame, was associated with lower levels of happiness. Thus, it is considered undesirable to suppress emotional expressions in human communication, and robots must express a variety of emotions in situations where humans and robots coexist.
Various studies have long been conducted on robot emotion expression. Sugano and Ogata (1996) developed a robot that can change its head color in accordance with its emotion. Ariyoshi et al. (2004) developed a robot that uses LEDs for emotional expression and demonstrated that LED-induced facial color change improves emotion recognition accuracy. Terada et al. (2011) proposed the use of the HSV color space, which relates Plutchik’s emotion circle model (Plutchik, 2002) to color, and applied it to robot emo- tional expressions. This approach showed the potential for classifying basic emotions on the basis of hue values. These studies are mainly based on the use of coloration to express emotions. Many robots have been developed to
express emotional expressions by emitting light from their heads, such as eyes and cheeks.
On the other hand, while there have been studies focusing on robot cloth- ing rather than robot heads, Hindriks et al. (2022) found that clothing can affect human-like qualities, changing first impressions of expertise and likability. Hurtienne and Arnold (2020) investigated whether financial invest- ments affect outcomes when information is presented by a robot dressed differently. However, their results showed that formal, casual, or undressed clothing had no effect on the results. Similarly, Yoshiwara and Kobayashi (2021) conducted a study where participants watched videos of robot-like agents performing household tasks and evaluated their performance. The study demonstrated that wearing appropriate clothing corresponding to the household task led to the perception that the robot was performing the task adequately. Thus, while studies on robot clothing have discussed the robot’s capability to perform in various fields, there have been no investigations into whether different types of clothing can enable different emotional expres- sions. We (Sugiyama and Kanoh, 2022) therefore created a robot with a back and forward tilting motion, the “Tilting Robot,” and studied its emotional expression depending on the clothing color it wears. The results suggested that the robot’s emotional expression may change depending on the cloth- ing color it wears. Additionally, it was also suggested that the intensity of the emotional expression may vary depending on the speed of the robot’s motion. However, previous study (Sugiyama and Kanoh, 2022) has exper- imented with clothing based on fabrics similar to those worn by humans, and did not examine the effect of light-emitting (LE) clothing on emotional expression in the same way that LE the robot’s head is used. If wearing LE clothing does indeed have an effect on the robot’s emotional expression, we believe it holds great potential for applications. Moreover, LE clothing is less likely to cause discomfort and is easy to implement. In this paper, we compare LE clothing with clothing made of ordinary fabric and explore the emotional expression effects of each type of clothing on the basis of the six basic emotion categories.
Figure 1: Robot appearance. An example of red light-emitting clothing.
Investigating Emotional Expressivity in Robots Wearing Light-Emitting Clothing 143
TILTING ROBOT 2 Mechanism of Tilting Robot 2
We developed a robot called “Tilting Robot 2”that can tilt back and forward with variable motion speed for the experiments described in the follow- ing. Figure 1 shows Tilting Robot 2, based on Babyloid, a robot developed by Kano et al. (2015). The robot is 320 mm high, 191 mm wide, and weighs 914 g. Its appearance is designed to be infantile to avoid evoking socially complex emotions like “shame” and “embarrassment” (Ben-Ze’ev and Oatley, 2007). Socially complex emotions are considered a combination of basic emotions and may influence the interpretation of these basic emo- tions (Bridges, 1932). To ensure the experiment’s accuracy in evaluating basic emotions, it was essential to account for such effects. Therefore, Tilting Robot 2 was designed to have infant-like features (Lorenz, 1943). The appearance of Tilting Robot 2 adopted the features of the baby schema (Lorenz, 1943), making it less likely to evoke complex emotions.
Clothing of Tilting Robot 2 According to Wiedemann et al. (2015), red gives an aggressive impression and is associated with anger. On the other hand, Kato and Yamashita (2016) sug- gested that blue is associated with sadness, and in particular, using blue as a background color may intensify the robot’s expression of sadness. While pre- vious studies (Sugiyama and Kanoh, 2022) have suggested the possibility of expressing the robot’s emotional expression by wearing red or blue clothing, we believe that emotional expression can be even more strongly expressed when the clothing is LE. In this experiment, we developed a uniformly light-emitting robot garment using an inorganic EL sheet.
EXPERIMENT Experimental Setting
The experiment was conducted to verify whether the emotional expression of Tilting Robot 2 wearing a specific clothing color changes when it makes a motion, and whether the LE clothing changes the emotional expression com- pared with the non-LE clothing. Four clothing colors were selected: red, blue, green, and a neutral white color. In addition, previous research (Sugiyama and Kanoh, 2022) suggested that the robot’s motion speed may influence emotional expression. In this experiment, the robot has two motion speeds: high and low. The robot’s motion behavior involves tilting back and forward from an upright posture and returning to an upright posture. An additional behavior where the robot remained still in an upright posture was included, resulting in a total of five behaviors for the experiment. Thus, there are 20 combinations of clothing and behavior. To compare the effects of LE and non- LE emotional expressions, EL sheets are used for the LE behavior while felt is used for the non-LE behavior. For the EL sheet, two conditions were set: one where the sheet continuously emitted light during the robot’s motion (con- stant LE condition), and another where the sheet only emitted light during the motion (intermediate LE condition). Therefore, there are three conditions
144 Sugiyama and Kanoh
for the clothing: constant LE, intermediate LE, and felt, resulting in a total of 60 possible combinations. Since the luminance of the EL sheet decreases with increasing LE time, the evaluation will be conducted by taking movies that were filmed in advance and presenting the movies as materials.
In the experimental procedure, we first give the participants an overview of the experiment. Following this, the participants are asked to respond to the Interpersonal Reactivity Index (IRI) as a preliminary questionnaire. Next, one of the 60 materials was presented to the participants. They were then asked to evaluate their feelings toward the presented material and respond to the questionnaire (Figure 2). The questionnaire items consisted of six basic emo- tions (anger, fear, surprise, pleasure, sadness, and disgust), and participants responded to each item on a 7-axis Likert scale (1: not applicable at all to 7: applicable very often). This process was repeated 60 times. The materials were presented in random order. Ten male and ten female university students (mean age 20.5 years ± S.D. 1.2 years) participated in the experiment in a within-subjects design.
Figure 2: Experiment environment.
Interpersonal Reactivity Index In this experiment, participants observe the robot’s motion and clothing and judge what kind of emotional expressions they perceive the robot to be expressing. Due to the subjective nature of such judgments, there is a pos- sibility that the emotions perceived by participants may vary depending on their individual empathy characteristics. Considering this possibility, it is nec- essary to measure the empathy characteristics of the participants in advance using the IRI (Davis, 1980). The IRI is a 28-item scale that assesses empathy across four aspects: Personal Distress, Empathic Concern, Perspective Tak- ing, and Fantasy. As all the participants in this experiment were Japanese, the Japanese version of the IRI was used (Himichi et al., 2017).
RESULTS
Figure 3 shows the results of the IRI obtained from the preliminary question- naire. Because this experiment followed a within-subjects design, it was not
Investigating Emotional Expressivity in Robots Wearing Light-Emitting Clothing 145
possible to compare the empathy characteristics. Therefore, a comparison was made with the IRI results from a previous study (Sugiyama and Kanoh, 2022). A one-way ANOVA was conducted, and it revealed no significant differences for all items. Thus, we proceeded with the analysis assuming that there were no significant differences in the empathy characteristics compared with the previous study (Sugiyama and Kanoh, 2022).
Impressions Per Motion To examine how the impression changes on the basis of the robot’s motion, the scores for each motion were summed, and the average of these sums was calculated as the score for each participant. A one-way ANOVA was per- formed on the total scores, and Bonferroni’s multiple comparison test was conducted when significant differences were found. As shown in Figure 4, the emotional expression of surprise was significantly greater for the low-speed backward-leaning motion and the high-speed backward-leaning motion compared with the low-speed forward-leaning motion and the high-speed forward-leaning motion, regardless of the clothing type. In addition, the high-speed backward-leaning motion showed significantly higher emotional expression than the low-speed backward-leaning motion. A one-sample t-test was conducted on the backward-leaning motion with a score of 4, which indicated an evaluation of “undecided.” As shown in Table 1, significant differences were observed among all three conditions for the high-speed backward-leaning motion and only in the intermediate LE condition for the low-speed backward-leaning motion.
For the emotional expression of sadness, as shown in Figure 5, the low- speed forward-leaning motion and the high-speed forward-leaning motion significantly expressed this emotion compared with the low-speed backward- leaning motion and the high-speed backward-leaning motion, regardless of the clothing type.
Figure 3: Results of IRI.
146 Sugiyama and Kanoh
Figure 4: Motion - result of surprise.Table 1. Backward-leaning motion - result of surprise.
Table available in the original PDF paper.
Item Ave. S.D. t (4,19) p Constant LE (High) 5.00 1.19 3.76 0.001 p< 0.01 Intermediate LE (High) 5.43 1.24 5.16 0.001 p< 0.01 Felt (High) 5.21 1.04 5.20 0,001 p< 0.01 Constant LE (Low) 3.93 1.22 0.28 0.786 N.S. Intermediate LE (Low) 4.44 0.93 2.11 0.049 p< 0.05 Felt (Low) 4.38 1.19 1.27 0.221 N.S.
Figure 5: Motion - result of sadness.
Investigating Emotional Expressivity in Robots Wearing Light-Emitting Clothing 147
Impressions Per Clothing Next, we investigate how the impression of the robot changes depending on its clothing color. The scores for each clothing color were summed, and the average score for each participant was calculated. A one-way ANOVA was performed on the total scores, and Bonferroni’s multiple comparison test was applied when significant differences were found. As shown in Figure 6, the results indicated that participants perceived the emotional expression of anger more significantly from the robot wearing red clothing compared with the other clothing colors, regardless of motion. Furthermore, when the same test was conducted among the three conditions, it became clear that the emo- tional expression of the robot wearing red clothing was significantly weaker in the felt condition than in the two LE conditions. Additionally, a one- sample t-test was conducted with a score of 4, which indicated an evaluation of “undecided.” As shown in Table 2, significant differences were observed between the two LE conditions, whereas no significant differences were found in the non-LE felt condition.
Next, as shown in Figure 7, the emotional expression of sadness was sig- nificantly higher for the robot wearing blue clothing compared with the other clothing colors, regardless of motion.
Figure 6: Clothing - result of anger.Table 2. Red clothing (anger): results of a one-sample t-test for a score of 4.
Table available in the original PDF paper.
Item Ave. S.D. t (4,19) p Constant LE 4.99 1.01 4.34 0.001 p< 0.01 Intermediate LE 4.98 1.14 3.82 0.001 p< 0.01 Felt 3.75 1.26 0.89 0.387 N.S.
DISCUSSION
Motion The results in Figure 4 show that regardless of the clothing color and motion speed, a backward-leaning motion expresses surprise. Such a motion that causes the head to bend backward may give the impression of surprise. Fur- thermore, the faster motion speed strengthened the impression of surprise.
148 Sugiyama and Kanoh
According to Yamane (2005), surprise is an instantaneous emotion triggered by sudden motion. This may explain why quick motions lead to the impres- sion of surprise. The results in Table 1 show that only the intermediate LE condition received a higher score for the motion of leaning backward at a low speed. This is thought to be due to the fact that the sudden emission of light, as well as the motion speed, can give the impression of surprise.
Next, Figure 5 shows that the motion of leaning forward expresses sad- ness, regardless of clothing and regardless of motion speed. Thus, the motion of nodding one’s head forward is considered to convey the impression of sad- ness. Unlike surprise, there is no difference in the speed of the forward-leaning motion. The reason for this is that sadness is a continuous emotion. Philippe and Lavrijsen (2015) investigated the duration of each emotion and showed that sadness lasts for 120 hours, longer than other emotions. This suggests that sadness is a continuous emotion and that the emotional expression of sadness did not change with the speed of motion.
Figure 7: Clothing - result of sadness.
Clothing Next, Figure 6 shows that, regardless of motion, red clothing is an expres- sion of anger. The color red is known to evoke an aggressive impression (Wiedemann et al., 2015), and we believe that the use of red clothing led to the perception of anger. Furthermore, Table 2 shows that the felt condi- tion produced lower scores for anger, with scores below the median score of 4 on the scale. This suggests that the red clothing created with felt may only produce an impression of vitality and not anger, consistent with our previous study (Sugiyama and Kanoh, 2022). Plutchik’s model of the emotional cir- cle (Plutchik, 2002) suggests that the emotion of aggression is a combination of anger and vigilance, giving a similar impression to others. As it has been reported that red LE colors make people more alert (Meijden et al., 2018), it is possible that the LE color from the EL sheet strongly conveys the impres- sion of anger. Therefore, we believe that red clothing created with LE has the effect of enhancing the impression of anger.
Investigating Emotional Expressivity in Robots Wearing Light-Emitting Clothing 149
Next, Figure 7 shows that blue clothing expresses sadness, regardless of motion. Interestingly, the felt condition did not significantly lower the sadness score. While several studies have shown that blue light emission can have a relaxing effect on people (Minguillon et al., 2017), there are no reports suggesting that blue light emission enhances specific emotions. Thus, it can be inferred that there is almost no effect of blue LE on emotional expression and that there was no difference in the evaluation of blue color depending on whether the LE was used.
CONCLUSION
In this paper, we compared LE clothing and clothing made of ordinary fabrics to investigate the effects of each type of clothing on the emotional expression of a robot. The results showed that the red LE garment conveyed the impres- sion of anger. On the other hand, the blue garments, whether LE or not, conveyed the impression of sadness.
In the future, we will investigate how the intensity of the emotional expres- sion changes with motion speed. In addition, since the robot’s motions have not been designed for each emotion in previous studies, we will investigate whether changing the clothing color of the robot when it expresses a specific emotion will change the emotion it receives.
REFERENCES
T. Ariyoshi, K. Nakadai and Hiroshi Tsujino, “Effect of facial colors on humanoids in emotion recognition using speech,” IEEE International Workshop on Robot and Human Interactive Communication, 59–64, (2004). S. Balzarotti, F. Biassoni, D. Villani, A. Prunas and P. Velotti, “Individual differences in
cognitive emotion regulation: Implications for subjective and psychological well- being,” Journal of Happiness Studies, 17, 125–143, (2016). A. Ben-Ze’ev and K. Oatley, “The intentional and social nature of human emotions: reconsideration of the distinction between basic and non-basic emotions,” Journal for the Theory of Social Behavior, 26, 81–94, (2007). K. M. B. Bridges, “Emotional development in early infancy,” Child Development, 3,
324–341, (1932). E. A. Butler, B. Egloff, F. H. Wilhelm, N. C. Smith, E. A. Erickson and J. J. Gross, “The social consequences of expressive suppression,” Emotion, Vol. 3, 48–67, (2003). M. H. Davis, “A multidimensional approach to individual differences in empathy,”
JSAS Catalog of Selected Documents in Psychology, 10, 85–103, (1980). J. J. Gross and O. P. John, “Individual differences in two emotion regulation processes: Implications for affect, relationship, and well-being,”Journal of Personality and Social Psychology, 85, 348–362, (2003). T. Himichi, H. Osanai, T. Goto, H. Fujita, Y. Kawamura and M. H. Davis, M.
Nomura, “Development of a Japanese version of the Interpersonal Reactivity Index,” The Japanese journal of psychology, 88, 61–71, (2017). K. V. Hindriks, M. Hagenaar and A. L. Huckelba, “Effects of robot clothing on first impressions, gender, human-likeness, and suitability of a robot for occu- pations,” 31st IEEE International Conference on Robot and Human Interactive Communication, 428–435, (2022). J. Hurtienne and D. Arnold, “The naked truth?” ACM/IEEE International Confer-
ence on Human-Robot Interaction, 269–271, (2020).
150 Sugiyama and Kanoh
M. Kanoh, “A robot as “receiver of care”in symbiosis with people,” Journal of Japan Society for Fuzzy Theory and Intelligent Informatics, 27, 6, 193–201, (2015). M. Kato and T. Yamashita, “Effects of colors on emotional recognition from facial
expressions by line drawing,” Journal of Japan Society for Fuzzy Theory and Intelligent Informatics, 28, 2, 576–582, (2016). K. Lorenz, “Die angeborenen Formen möglicher Erfahrung [The innate conditions of the possibility of experience],”Zeitschrift fur Tierpsychologie, 5, 235–409, (1943). W. P. Meijden, B. H. W. Lindert, J. R. Ramautar, Y. Wei, J. E. Coppens, M. Kamer-
mans, C. Cajochen, P. Bourgin and E. J. W. Someren, “Sustained effects of prior red light on pupil diameter and vigilance during subsequent darkness,” Proceedings of the Royal Society B: Biological Sciences, 285, 1883, (2018). J. Minguillon, M. A. Lopez-Gordo, D. A. RenedoCriado, M. J. Sanchez-Carrion and F. Pelayo, “Blue lighting accelerates post-stress relaxation: Results of a preliminary study,” PLOS ONE, 12, 10, (2017). R. Plutchik, “Emotions and life: Perspectives from psychology, biology, and evolu-
tion,” Amer Psychological Assn, (2002). S. Sugano and T. Ogata, “Emergence of mind in robots for human interface-research methodology and robot model,” IEEE International Conference on Robotics and Automation, 2, 1191–1198, (1996). T. Sugiyama and M. Kanoh, “Meaning of clothing colors in interactive robots,” 2022
Joint 12th International Conference on Soft Computing and Intelligent Systems and 23rd International Symposium on Advanced Intelligent Systems, (2022) K. Terada, A. Yamauchi and Akira Ito, “Artificial emotion expression for a robot by dynamic color change,” IEEE International Workshop on Robot and Human Interactive Communication, 314–321, (2011). P. Verduyn and S. Lavrijsen, “Which emotions last longest and why: The role of event
importance and rumination,” Motivation and Emotion, 39, 119—127, (2015). D. Wiedemann, D. M. Burt, R. A. Hill and R. A. Barton, “Red clothing increases perceived dominance, aggression and anger,” Biology Letters, 11, (2015). I. Yamane, “Phenomenology of surprise,” Sugiyama Women’s University Research
Paper No. 36 (Humanities), 13–28, (2005), (in Japanese). K. Yoshiwara and K. Kobayashi, “Effect on user impression of robot’s Task Dependent Uniform,” International Conference on Computer-Human Interaction Research and Applications, 1, 90–97, (2021).
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Emerging Technologies and Future of Work
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-93-3
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional