Yufei Liu, Danhua Zhao (2022). In-car sound design under the framework of emotional design. In: Ergonomics In Design. AHFE Open Access, vol. 47, pp. 413–419. https://doi.org/10.54941/ahfe1001964
In the in-car interactive system, sound is an efficient and indispensable interactive element. As the in-car interaction design focuses more on the experience improvement brought about by the user's emotional changes, the in-car sound design has gradually introduced emotional design methods into the design process. The literature on in-car sound design is increasing, but there is no overview of the in-car sound design under the framework of emotional design.This paper aims to provide a literature review on the subject of the emotional design of in-car sounds. Based on keywords searching, the paper screened the topic-related papers since 2011 to sort out the development trend of in-car sound design in the past ten years and refined the application of emotional design methods in sound design. The paper summarizes the research trends of in-car emotional sound interaction as follows: (1) Personification; (2) Emotional trigger mechanism; (3) Speech and sound effects.
Sound design has always been an important part of car design. Early sound research in car focused on analyzing noise in the driving environment, such as modifying car structures to reduce noise (Nefske et al., 1982) or explo- ring the impact of noise on driver mood (Liang et al., 2016). Non-noise sounds are mainly used as prompts or simple feedback for user operations in human-computer interaction systems (Tan, 2018). In the past ten years, as human-computer interaction has become an important part of car design, sound design has highlighted its advantages in information communication because it does not occupy the driver’s visual resources (Tan, 2012). Design studies are also increasing year by year.
On the other hand, the user’s emotion while driving is also the focus of the research on car human-computer interaction design. Research has shown that emotions while driving can have a serious impact on road safety (Braun et al., 2020), and that good emotions can lead to significant expe- rience improvements. Effective emotion detection and emotion intervention
through interaction design is very necessary. Sound is capable of both infor- mation transfer and emotional communication (Tan, 2018), so emotional intervention can be well done through emotional sound design.
At present, compared with the emotional design of visual interface, the research of emotional sound design in the car is not mature enough. This paper mainly sorts out the literature related to emotional sound design in the car in the ten years from 2011 to 2021, and summarizes the current design progress from three aspects: (1) Research on sound design based on information transmission. (2) Research on sound design based on emotional communication. (3) Directions for future research.
IN-CAR EMOTIONAL SOUND DESIGN Since ancient times, human beings have developed language to communicate and transmit information. Emotional expression in sound is also an impor- tant part of human communication. Not only human speech, but also music and ambient sounds carry emotional information (Weninger et al., 2013). By taking into account both information transmission and emotional expres- sion, in-car sound design can effectively assist the driver’s in-car interaction and enhance the driving experience.
Most of the early sound research focused on the analysis and classifica- tion of the concept of sound, as well as the information transmission level of sound. With the development of human-computer interaction, the design with “human experience” as the core makes the user’s emotions more and more important, and the emotional method is gradually introduced to study the emotional expression of sound (Beale & Peter, 2008).
Classification of Sounds In daily life, people can perceive a wide variety of sounds. When designing, it is also necessary to classify and select sounds in a purposeful and methodical manner. At present, the classification of acoustic materials in product design can be divided into analog sound (including speech, natural sound, electronic sound and pure music), mechanical noise, ultrasonic, infrasound, etc., among which analog sound is mainly used (Sun & Ma, 2011). For example, using speech sounds to make artificial intelligence assistants, or using electronic sounds for action feedback, prompting users, etc.
The sound classification method for human-computer interaction design believes that according to their natural properties, the sounds used in the in- car auditory interface can be divided into: natural sounds, musical sounds and speech. If we distinguish from the perspective of our cognition of sound, natural sound belongs to concrete sound (see Figure 1), while musical sound and speech belong to abstract sound. Another division is divided into speech and non-speech: the former two are non-speech, the latter is speech (Li et al., 2001). The division of sounds used in-car auditory interface in this paper is mainly based on this classification method. The auditory display content in sound design also contains complex elements, like the ontological properties
In-Car Sound Design Under the Framework of Emotional Design – A Review 415
Figure 1: Common sound classification for HCI. (Adapted from Li et al., 2001).Table 1. The focus of in-car sound information transmission.
Table available in the original PDF paper.
Focus Point Sound Type Design Practice Reference Pietila & Lim, 2012 Eliminate information interference Noise reduction, simulate natural sound
Noise, Natural sound Speech Reduce driver’s visual load, use multimodal interaction Accurate and efficient presentation of information
Lin & Yin, 2016
Brodsky & Kizner, 2012; Tabata et al., 2011; Tan et al., 2012 Optimized expression of information
Musical sound
Highlights brand uniqueness, pleasing to the ear, and quick sound editing
of the sound (such as sound intensity, pitch, timbre, etc.), and the expression of the sound (such as duration, rhythm, imagery, etc.) (Tan et al., 2012).
Research on Sound Design Based on Information Transmission As said before, information transmission is the basic function of sound. During the driving process, noise, ambient sound, and in-car prompt sound are all transmitting information. Sound systems in automobiles are complex and require very careful consideration in their design. This paper summari- zes three focus points of the existing research on the use of in-car sound for information transmission (see Table 1).
Sound design is involved in every step of the car from sale to use. The most basic task of sound design is to eliminate unnecessary information interfere- nce. For example, the sound of a product during operation can influence a consumer’s purchasing decision. When buying a car, a test drive that makes a strange noise can give consumers the impression of poor quality (Pietila & Lim, 2012).
On the other hand, beautiful and well-designed sounds will also affect the user’s brand impression. Therefore, it is also a clever design idea to establish a unique sound system or an iconic sound. In a 2016 study, the designer proposed a sound effect solution for flexibly editing sound effects, so as to add pleasant prompts to embedded systems (Lin & Yin, 2016).
416 Liu and Zhao
Table 2. The focus of in-car emotional communication.
Table available in the original PDF paper.
Focus Point Sound Type Design Practice Reference Emotion Recognition Mechanisms Noise, Natural sound, Musical sound, Speech Explore the sound mechanisms/ sound elements that trigger positive/negative emotions
Schuller et al., 2012;
Natural sound, Musical sound
Weninger et al., 2013; Desmet, 2012; Çano et al., 2016; Fakhrhosseini & Jeon, 2016; Sound and affective computing
Develop sound descriptors and emotional computing models Musical Sound, Speech
Sound- emotion regulation system
Alternative in-car background sound, anthropomorphic voice assistant
During driving, the sounds received by the driver can be divided into the noises passively generated during driving and the sounds actively emitted by the car (such as natural sound, musical sound, and speech). The design of in-car sound can not only influence consumers’ impression of the car but also has safety considerations. Driver distraction can be caused by excessive noise and aggressive music while driving. (Brodsky & Kizner, 2012). Unclear sound and overlapping sounds can interfere with the accuracy of information transmission, leading to misjudgment by the driver. These will directly or indi- rectly endanger driving safety (Tabata et al., 2011). In the process of sound design, noise needs to be properly intervened so that it does not interfere with the driver’s concentration and the communication of important information.
Sound design also needs balance. The auditory display is time-sensitive, and generally, the information transmitted should not be too much. If com- plex information is to be transmitted, it needs to be processed in stages, such as the two-stage processing of “calling attention - transmitting information”. The various prompt sounds existing in the car should be clear in priority and secondary, and information should be conveyed accurately and efficiently to ensure driving safety.
Research on Sound Design Based on Emotional Communication Regarding the research on the emotional conveyance properties of sound, one research focus is to analyze the human emotion recognition mechanism for sound and to understand what kind of sound properties can trigger what specific emotion; another is focus on how to influence the driving behavior of the driver (Hsieh et al., 2010), and how to intervene the sound to achieve the effect of regulating the driver’s behavior. Ultimately, these studies can be integrated into the framework of sound-emotion regulation systems to provide guidance for the design of emotional sounds in car (see Table 2).
In-Car Sound Design Under the Framework of Emotional Design – A Review 417
Almost any sound that humans receive every day contains emotional information (Weninger et al., 2013), and emotion plays a central role in human-human interaction (Sears & Jacko, 2002). Therefore, the sound con- taining emotion has also received extensive attention in the HCI field. In 2012, Desmet’s research defined 25 positive emotions that users can feel when they experience a product, which can be used as a starting point for emotio- nal sound design. By calibrating the aim emotions, the sound that influence emotions can be designed accordingly (Desmet, 2012).
Noise has an impact on people’s driving mood, and the impact is mainly negative (Chan & Singhal, 2015). For example, when the driver is depres- sed and anxious, the negative attitude may lead to a potential accident, so it is very important to analyze and assist the driver to control his emotions (Bankar et al., 2018). The researchers put forward the concept of “noise annoyance” very early, and found that the annoyance is related to various parameters such as human psychological factors (Liang et al., 2016). Cer- tain musical sounds can also have a negative impact on driving mood (Çano et al., 2016) (Fakhrhosseini & Jeon, 2016). One study attempted to develop in-car music as an alternative to dangerous, emotionally stimulating music. They report the intervention of alternative music on mood and the effective improvement of driving safety (Brodsky & Kizner, 2012).
The emotions evoked by the analog sounds emitted by the product are complex. As early as 1936, there were experimental studies demonstrating the correlation between music and emotions (Hevner, 1936). In related rese- arch over the past decade, researchers have explored factors that affect users’ perception of sound, such as loudness and pitch (Tan & Yang, 2018). Based on the development in the field of affective computing, some acoustics studies have attempted to propose a holistic computational model for sound design (Zentner et al., 2008), and use valence and arousal to evaluate emotion in sound (Schuller et al., 2012) (Eyben et al., 2010).
In-car sound design can use the emotional conveyance properties of sound to form an emotion regulation system (Gusikhin, 2011) that impro- ves road safety by catering to user emotions (Holman & Popu¸soi, 2020) (Fakhrhosseini et al., 2014). The focus is on emotion recognition and response. A 2018 study reported real-time analysis of drivers’ emotions through EEG waves and used an automatically responding music system to adjust the driver’s mood (Bankar et al., 2018). The application of voice assi- stant is a research hotspot of emotion regulation system. Reports of voice assistant said it can trigger a user’s tendency to personification conversati- ons, which in turn increases user satisfaction (Lopatovska & Williams, 2018) (Yarosh et al., 2018). Results of a simulator study suggest that an emotio- nal voice assistant that resonates with users best improves driver negativity (Braun et al., 2019).
CONCLUSION
By collecting and summarizing research in recent years, the following three major design trends in the field of emotional sound design in cars can be derived.
418 Liu and Zhao
(1) Voice assistant with anthropomorphic speech and close to natural communication habits.With the development of artificial intelligence technology, voice assistants that are more intelligent and anthropomor- phic will receive more attention. Natural communication helps reduce the cost of understanding and communication; through anthropomor- phic speech, voice assistants can effectively convey emotions and assist drivers in emotional regulation. (2) Synaesthesia of speech and sound effects. Synesthesia of sound and other perceptual modalities, such as touch, sight, taste, and even memory, can be potential mechanisms for the effective regulation of emotions. By making adjustments to sound attributes, combined with metaphorical meanings, information and emotions can be more effectively conveyed in the car through sound. (3) More accurate emotional trigger mechanism. The development of in- car sound interaction requires researchers to explore more accurate emotion triggering mechanisms. Through various forms such as quan- titative calculation and subjective analysis of emotion and sound, a more standardized and traceable sound-emotion trigger mechanism can be summarized, forming the framework for in-car emotional sound design.
REFERENCES
Bankar, C. Bhide, A. Kulkarni, A. Ghube, C. and Bedekar, M. (2018). Driving Control Using Emotion Analysis Via EEG. 2018 IEEE Punecon, pp. 1–7. Beale, R. & Peter, C. (2008). The role of affect and emotion in HCI. In Affect and
emotion in human-computer interaction, pp. 1–11. Braun, M. Schubert, J. Pfleging, B. and Alt, F. (2019). Improving Driver Emotions with Affective Strategies. Multimodal Technologies and Interaction, 3(1), 21. Braun, M. Weber, F. and Alt, F. (2020). Affective Automotive User Interfaces –
Reviewing the State of Emotion Regulation in the Car. ACM Comput. Surv. Brodsky, W. and Kizner, M. (2012). Exploring an alternative in-car music background designed for driver safety. Transportation Research Part F: Traffic Psychology and Behaviour, 15(2), 162–173. Çano, E. Coppola, R. Gargiulo, E. Marengo, M. & Morisio, M. (2016). Mood-
based on-car music recommendations. In International Conference on Industrial Networks and Intelligent Systems, 154–163. Chan, M. & Singhal, A. (2015). Emotion matters: Implications for distracted driving.
Safety science, 72, 302–309. Desmet, P. M. A. (2012). Faces of product pleasure: 25 positive emotions in humanproduct interactions. International Journal of Design, 6(2), 1–29. Eyben, F. Wöllmer, M. Poitschke, T. Schuller, B. Blaschke, C. Färber, B. & Nguyen-
Thien, N. (2010). Emotion on the road—necessity, acceptance, and feasibility of affective computing in the car. Advances in human-computer interaction. Fakhrhosseini, M. & Jeon, M. (2016). The effects of various music on angry drivers’ subjective, behavioral, and physiological states. In Adjunct Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, pp. 191–196.
In-Car Sound Design Under the Framework of Emotional Design – A Review 419
Fakhrhosseini, S. M., Landry, S., Tan, Y. Y., Bhattarai, S., & Jeon, M. (2014). If you’re angry, turn the music on: Music can mitigate anger effects on driving performance. In Proceedings of the 6th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, pp. 1–7. Gusikhin, O. Klampfl, E. Filev, D. & Chen, Y. (2011). Emotive driver advisor system
(EDAS). In Informatics in Control, Automation and Robotics. Hevner, K. (1936) Experimental studies of the elements of expression in music.
American Journal of Psychology, 48:246–268. Holman, A. C., & Popu¸soi, S. A. (2020). How you deal with your emotions is how you drive. Emotion regulation strategies, traffic offenses, and the mediating role of driving styles. Sustainability, 12(12), 4929. Hsieh, L.Seaman, S.and Young, R. (2010). Effect of emotional speech tone on driving
from lab to road: fMRI and ERP studies. In Proceedings of the 2nd International Conference on Automotive User Interfaces and Interactive Vehicular Applications. Li, Q. Fang, Z. Shen, M. and Chen, Y. (2001). Sound usage for auditory interfaces.
Chinese Journal of Ergonomics. 7(4), 41–44. Liang, Y. Chen, K. Zhang, B. and Qiang, T. (2016). Timbre representation of vehicle noise annoyance. ACTA ACUSTICA, 41(3), 442–448. Lin, P. and Yin, S. (2016). Design and realization of advanced prompt sound in
embedded systems. Software and Algorithms, 35(21), 20–23. Lopatovska, I. and Williams, H. (2018). Personification of the Amazon Alexa: BFF or a Mindless Companion. Proceedings of the 2018 Conference on Human Information Interaction & Retrieval - CHIIR ‘18, 265–268. Nefske, D. J. Wolf, J. A. and Howeli, L. J. (1982). The Automobile Passenger
Compartment: A Review of Current Practice. 20. Pietila, G. and Lim, T. C. (2012). Intelligent systems approaches to product sound quality evaluations – A review. Applied Acoustics, 73(10), 987–1002. Sears, A. and Jacko, J. A. (2002). Emotion in Human–Computer Interaction Scott
Brave and Clifford Nass. In The Human-Computer Interaction Handbook. Schuller, B. Hantke, S. Weninger, F. Han, W. Zhang, Z. and Narayanan, S. (2012).
Automatic recognition of emotion evoked by general sound events. 2012 IEEE ICASSP, 341–344. Sun, X. and Ma, C. (2011). Analysis of Cognitive Psychological Properties in Sound Design of Products. Packaging Engineering, 32(24), 84–94. Tabata, T. Konet, H. & Kanuma, T. (2011). Development of Nissan Approaching
Vehicle Sound for Pedestrians: How to solve the trade off between Quietness and Pedestrian safty of the Electric vehicles? SAE Technical Paper. Tan, H. Zhao, D. and Zhao, J. (2012). Research on Automotive Human Machine Interface Design Based on Complex Interaction Context. Packaging Engineering, 33(18), 26–30. Tan, Z. and Yang, W. (2018). Sound Synaesthesia Design Based on the Natural
Interactions. Packaging Engineering, 39(8), 68–73. Weninger, F. Eyben, F. Schuller, B. W. Mortillaro, M. and Scherer, K. R. (2013). On the Acoustics of Emotion in Audio: What Speech, Music, and Sound have in Common. Frontiers in Psychology, 4. Yarosh, S. Thompson, S. Watson, K. Chase, A. Senthilkumar, A. Yuan, Y. and
Brush, A. J. B. (2018). Children asking questions: Speech interface reformulati- ons and personification preferences. Proceedings of the 17th ACM Conference on Interaction Design and Children, 300–312. Zentner, M. Grandjean, D. and Scherer, K. R. (2008). Emotions evoked by the sound of music: Characterization, classification, and measurement. Emotion.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Ergonomics In Design
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-23-0
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional