Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

User Emotional Experience Assessment Method of Product's Intentional Sound

School of Design, Hunan University, China ↗
Publication year
Digital Object Identifier10.54941/ahfe1002750

Abstract

Product sound plays an important role in the multi-sensory user experience of home appliances. The sound effects that are given meaning by designers (intentional sounds) in home appliances have three contributions to the user experience: semantic conformity to make a satisfactory contribution to the overall product experience; brand impression; and bringing pleasantness and emotional experience. Based on the three aspects of the impact of intentional sounds on product experience, combined with the Semantic Differential method in the field of Kansei engineering and the Hevner adjective table commonly used in music sentiment analysis, this research will design a set of intentional sound evaluation methods from the perspective of user experience.

INTRODUCTION

In the field of human-computer interaction, the product impression through multi-sensory channels will affect the user’s overall product experience (Schifferstein and Desmet, 2008). Sound is an important factor in these multi- sensory product evaluations (Erkut et al. 2015). Desmet et al. divide the user product experience into three components: the aesthetic experience that sati- sfies one or more senses of the user; the experience of meaning that enables users to identify metaphors, demonstrate personality, or evaluate product value; and the emotional experience that brings different emotional pheno- mena to users (Desmet and Hekkert, 2007). The impact of product sound on the user’s product experience also runs through these three directions. For aesthetic experience, product sound can assist in the formation of user’s visual experience, affecting users’ judgment on product features (Van Egmond, 2008); for the experience of meaning, some features of the product’s hea- ring have a semantic impact on the user’s product experience (Blauert and Jekosch, 1997; Özcan and van Egmond, 2012; Spence and Zampini, 2006); for emotional experience, product sound affects the user’s emotions in the daily interaction between the user and the product (Desmet and Hekkert, 2007; Özcan and van Egmond, 2008), and ultimately affects the user’s daily behavior subtly. In recent years, more and more researchers have focused on improving the sound quality of products to improve the experience of products (Özcan and van Egmond, 2008; Lyon, 2000).

© 2022. Published by AHFE Open Access. All rights reserved. 330

User Emotional Experience Assessment Method of Product’s Intentional Sound 331

Sound design activities are inherently multidisciplinary (Özcan and van Egmond, 2012; Langeveld et al., 2013). A large number of independent varia- bles (rhythm, pitch, etc.) have been shown to affect a wide range of dependent variables (attitude, pleasure, etc.) (Graakjær and Bonde, 2018). The com- plex correspondence between variables increases the difficulty of product sound design and evaluation. Several interviews with sound design practi- tioners have shown that there are large gaps in design, development, and evaluation among stakeholders (engineers, musicians, and interaction desi- gners) (Hug and Misdariis, 2011). Users and designers lack the vocabulary to express sound design ideas, while the professional vocabulary possessed by acoustic engineers is also difficult to communicate with non-professionals (Langeveld et al. 2013). Traditional product sound design and evaluation methods are therefore unsuitable for future interactive products that enable people to express their skills, emotions, and moods. (Erkut et al. 2015). Under such circumstances, although people are increasingly interested in the sound design and research of household appliances, most of the research focuses on consequential sounds (Takada et al. 2009; Martin et al. 2021) that are produced by the operation of certain functions of the product, such as the wind sound when the air conditioner is running, which is often understood as “noise”. Intentional sounds that carry product functional information and are endowed with emotions by designers are rarely mentioned, and there are few studies on intentional sounds from the perspective of user experience and designers. For example, a microwave oven sounds when food is ready, or a washing machine sounds a warning when the washing process is over (Özcan and van Egmond, 2012). In product design, intentional sounds can convey abstract meanings or provide information (feedback) about the results of a process or activity, combine user behavior with the function of the pro- duct, and allow sounds to establish a conceptual connection with a certain function of the product. Because of its relationship to carrying abstract infor- mation, intentional sounds usually appear in the form of “music,” and under the subconscious reminder that they carry information, listeners will also feel the need to pay attention to such sounds. The purpose of this study is to deve- lop a method for evaluating intentional sounds from the perspective of user experience, which starts with understanding the role of intentional sounds in user experience. Through the literature review, we divide the contribution of intentional sounds to user experience into three categories: 1) Semantic ali- gnment to make a satisfying contribution to the overall product experience; 2) conveying brand impression; 3) Bring joy and emotional experience.

Semantic Alignment to Make a Satisfying Contribution to the Overall Product Experience Intentional sounds carry information related to product functions, which will assist users to form an impression of the product concept and supple- ment users’ expectations (Özcan and van Egmond, 2008). One of the current concerns about product sound design is whether sound is suitable for the pro- duct’s concept (Blauert and Jekosch, 1997; Özcan and van Egmond, 2006). We expect the sound emitted by the product to conform to the product type

itself (Lyon, 2003), just as it is inappropriate to use the starting sound of a car as the starting sound of a household air conditioner, which may bias the user’s perception of the product type. At the same time, the design of product sound should be in line with the product value. Products that do not match their sound and value may cause user dissatisfaction (Özcan and van Egmond, 2006). Compared with the common “beep” sound with button feedback, some elegant music clips can effectively shape the product sound. In addition to the “advanced” perception of aspects in the process of using the product, users will also experience whether the sound produced by the product is consistent with the function of the product (Blauert and Jekosch, 1997). For example, when the product reports an error, the use of soft music may make it difficult for users to perceive the information conveyed by the product’s sound.

Convey Brand Impression People have become increasingly aware of the importance of hearing bran- ding to businesses in recent years (Lindstrom, 2006). A few simple notes in a unique musical style are enough to attract a target customer and associ- ate social and cultural values with a product (Cook, 2000). When this trait is applied to product sound design, it can help consumers find what they’re loo- king for faster when their visual attention is diverted (Knoeferle et al. 2014), and thus attract the brand’s core users.

Intentional sounds can enhance the brand value through careful design to make it more refined and trustworthy (Carron et al. 2014; Franinovi´c and Salter, 2013), and some iconic product sounds can also imply certain cha- racteristics of the brand, and even strengthen the emotional appeal of the product (Özcan et al. 2017; Miller, Mills, 2012). For example, the famous Harley-Davidson motorcycle engine sound is so loved by consumers for its unique sense of power that its manufacturer has attempted to register this sound as its own trademark (Sapherstein, 1998). This harmonious relation- ship between brand personality and vocal expressiveness is a decisive factor in delivering a brand message consistently and reliably (Westermann, 2008).

Bring Joy and Emotional Experience Apparently, people’s psychoacoustic experiences affect their sense of ple- asure (Zwicker and Fastl, 2013). For example, when the sound becomes sharper and louder, the sound becomes less pleasant for the listener. But on the other hand, just like the roar of a car’s engine, when the sound of machinery becomes louder, people’s sensory pleasure decreases, and at the same time, they obtain an emotional experience similar to “power” or “control of power” (Bisping, 1997). Some studies have also shown that audi- tory input appears to dominate specific parts of the affective and cognitive experience (such as impressing and remembering users), so perhaps product sounds can be developed for more engaging product experiences (Özcan et al. 2017).

User Emotional Experience Assessment Method of Product’s Intentional Sound 333

According to Ludden (Ludden and Schifferstein, 2007), “People have expe- ctations about how a product will sound.” If designers have enough insight into these expectations, they can cater to them. Similarly, the design of intenti- onal sounds can bring users a multi-level pleasant experience and a multi-type emotional experience. For example, making a “cute” sound to make users think the product itself is “cute” (Ludden and Schifferstein, 2007); or conve- ying a certain abstract emotional perception to the user, just as some high-end lighter manufacturers intentionally design the sound of operating the ignition to maximize their “luxury” positioning (Lageat et al. 2003).

From the perspective of user experience, the development of the evaluation method for product meaning sound can take the three types of contributions of product meaning sound to user experience as the evaluation dimension, and the performance of product meaning sound in these three types of con- tributions as the evaluation basis. It is worth noting that in the dimension of “bringing pleasure and emotional experience”, traditional psychoacoustic analysis has been able to reveal people’s perception of sound attributes (such as loudness, frequency, etc.), including pleasure and comfort (Zwicker and Fastl, 2013). This research will focus on the aspect of emotional experience in this dimension.

METHOD

Based on the above-mentioned three types of contributions of product mea- ning to user experience and the emotion evaluation test of kitchen appliance “food completion sound” (Due to word limit, it will be presented in the conference presentation), we summarize two evaluation methods.

Method One: Starting from the sound material (referring to the sound to be evaluated) itself, evaluate whether the existing musical emotion of the material is consistent with certain attributes of the product. Comparing the assessor’s perception of a product characteristic (e.g., appearance/brand value) with the perception of the sound material.

Method Two: Starting from the audience, evaluate whether the informa- tion carried by the sound material is effectively conveyed to the assessor (usually the target user of the product). comparing assessors’ perceptions of functional or emotional information (e.g., “start” or “satisfied”) before and after listening to the audio material.

For Method ONE, the semantics of the product’s feature, the type of the product, or the values of the product brand can all be regarded as attributes of the product. This approach is to judge whether the evalua- tion material “sounds” like it conflicts with a certain characteristic of the product, or to better express the characteristic. For Method TWO, there are generally two types of information carried by the sound of product meaning: 1) Expression function: convey information about a specific pro- duct function (for example, startup) or express a specific product status (for example, error), etc. 2) Expressing abstract emotions while expressing functions’ information: expressing product functions while at the same time conveying some abstract idea or emotion that the designer/product mana- ger wants to express (for example, using a soft and pleasant bird song as

the wake-up ringtone for an alarm clock). This method’s core purpose is to assess how much information the assessor can obtain from the assessment material.

Both evaluation methods require some kind of “medium” to quantify the user’s subjective evaluation (such as perception). For this purpose, this study chose to use the Semantic Differential (SD), which is widely used in Kansei Engineering (Osgood, 1957; Osgood, 1962), by using Kansei words or adje- ctives to describe the semantics and emotions carried by intentional sounds, as well as the emotions and moods users perceive from intentional sounds, and by using the Likert scale to guide assessors to express their subjective feelings.

Figure 1: Evaluate method in three types of intentional sounds’ contribution.
Original PDF Figure 1:

In addition, in many studies that use SD to analyze users’ perceptions of a certain feature of a product, the sources of Kansei words are mostly obtained from conversations with experts and users in related fields, including inte- rviews, telephone questionnaires, focus groups, etc. (Lindberg et al. 2013; Huang et al. 2012). However, for Method ONE, it is difficult for users with non-music engineering backgrounds to describe their perception of music in words (Langeveld et al. 2013). Therefore, a “music description vocabu- lary” that can be widely used in most music types and can be understood by ordinary users is needed. This study proposes to use Hevner’s adjective list, which is the most widely recognized in the study of the relationship between music and emotion (Hevner, 1935; Hevner, 1936), because the adjective list is established based on the specific analysis of music, poetry, and other arti- stic expressions, which is in line with the emotional connotation of music. It is frequently used in related research, such as music psychology and com- puter music sentiment analysis. However, due to the large number of words in the Hevner adjective list, in order to ensure that the evaluator’s attention is not distracted and the evaluation results are more accurate, the evaluators can be divided into two groups. The first group is responsible for evaluating the “title words” that best represent the word family (Namely Dignified, Sad ect.), selecting 1-2 most representative “title words” from the sound mate- rial according to the results (such as average ranking), and finally, the second group will analyze the word family represented by these “title words”to rate.

User Emotional Experience Assessment Method of Product’s Intentional Sound 335

Figure 2: Hevner music emotional adjective list (Adapted from Hevner, 1936).
Original PDF Figure 2:

Method ONE Preparation Take the “title word” of each adjective family in Hevner’s adjective list (Figure 2) as the content, prepare a 7-point scale (Figure 3, scale B), The scale uses a uniform bipolar system for evaluating Kansei words (from step 1). In this system, at one extreme of the scale indicates that there is no such feelings (Kansei words) at all, while the other extreme denotes very strong feeling (Kansei words). And prepare multiple scales of the same type (see Figure 3, scales C1-C8) based on the specific vocabulary of each word family (see Figure 2).

Step 1 Get Kansei words. For the characteristics of the product to be compared with the sound material (such as the appearance of the product, showing the product pictures or real objects to the assessor), Kansei words are obtained from different sources for different purposes. As described in Figure 1, in order to evaluate the effect of certain information that the desi- gner/brand wants to express to the user, the source of the vocabulary can be the designer/brand. To compare the difference between the user’s cognition of a product attribute and the cognition of the sound material, the source of the vocabulary can be the user itself.The specific acquisition methods can be questionnaires, interviews, focus groups, etc., but the vocabulary selected according to the semantic space description needs to include three aspects: evaluation (such as: good, elegant, modern); potency (e.g. strong, soft); and activity (e.g. passive, lively) (Osgood et al. 1957). The acquired Kansei words with obviously similar word meanings were combined, the obviously wrong words were screened, and the Kansei words were used as the content to make a 7-point scale of the same type as the “preparation stage”(Figure 3, scale A).

Step 2 Select assessors. Selecting proper assessors according to different purposes, divide them into groups A and B equally according to the number of people.

Step 3 Evaluating product characteristics using SD. Describe or present the information to be evaluated to the assessor, such as showing a picture of the product’s appearance or asking its perception of a certain emotion. All evaluators (from step 2) were invited to rate the Kansei words on Scale A without giving specific explanations for the Kansei words. After comple- ting the calculation of Kansei mean values and variances, the Kansei words with high variance can be considered that the evaluators have different under- standings of the words, which can reduce the attention paid to such Kansei words. The mean values can reflect the user’s understanding of the product characteristics in a simple and intuitive way.

Figure 3: Scale A (left); Scale A (middle); Scale C1-Scale C8 (right).
Original PDF Figure 3:

Step 4 Evaluating music perception using SD. After playing the sound mate- rial for group A, invite group A to rate it on scale A by “using the following words to describe your feeling of sound effect”, then calculate the mean values and variances. Omit the “title word”with large variance and sort them by mean values. By using the top two “title words,” choose two scales from C1 to C8. Then let group B rate on scale C by “using the following words to describe your feelings of sound effect.”

Step 5: Comparison of mean values. Compare and rank the mean values of the two groups of cognitive scores (product feature perception and sound material perception) and observe whether the higher-ranked Kansei words in the two groups have similar meanings. If it is similar, it means that the user’s perception of the sound material is similar to the perception of the product feature. If the word meaning is not similar, the relationship between the two cognitions can be judged according to the specific word meaning, such as conflict, mutual promotion, etc.

Step 6: Other Calculations. The data in Step 3 and Step 4 can be proces- sed and calculated according to different evaluation objectives. For example, using principal components analysis to extract comprehensive “word family” from the two sets of Kansei word scores and naming the “word family”accor- ding to the meanings of the variables (Kansei words in “words family”). Then compare the two groups of “word family” names, to more concisely reflect the differences between the two.

Method Two step 1 Get Kansei words. The specific method is similar to Method ONE- Step 1, but the target word type describes a certain function or emotion in people’s cognition, such as using the interview method to ask the subject: “What kind of sound do you think should be used as a warning?" or "What kind of emotion do you think the food brings to you?” Finally, refine the Kansei words and make a 7-point scale (same type as Figure 3).

Step 2 Select an assessor. Selecting proper assessors according to different purposes.

User Emotional Experience Assessment Method of Product’s Intentional Sound 337

Step 3 Before listening to the sound material, evaluate the functional or emotional perception of the sound material using the SD. Similar to Method ONE-Step 3, before playing the sound material to the evaluators, describe to them the functional/emotional information to be evaluated and invite them to rate the Kansei words on a scale (from Step 1) against this information.

Step 4 After listening to the sound material, evaluate the functional or emotional perception of the sound material using the SD. Rating the scale made in Step 1 again.

Step 5 Comparison of mean values. Comparing and ranking the mean values of the two groups of cognitive scores (the cognition of the evalua- tion information before and after listening to the sound material). Observe the ranking of the Kansei words in the two groups to judge whether there is a deviation in the transmission of information and the specific performance of the deviation.

Step 6: Other Calculations. Same as Method TWO-Step 6. It’s worth noting that paired T-tests may perform well in Method TWO, which can test whether there is a significant difference in the assessor’s perception of functi- onal or emotional cognition before and after listening to the sound, so as to judge the effectiveness of such information.

CONCLUSION

The two intentional sound evaluation methods can evaluate the impact of sound materials on user product experience from multiple levels and can fle- xibly adapt to different evaluation goals and objects, but there is still a lot of development space for their application scope. Besides the calculation link in the method, a variety of calculation methods can provide more in-depth insi- ght into the characteristics of the sound material and the relationship between the information carried in the sound material and the user’s perception. This part still needs to be explored.

REFERENCES

Bisping, R., 1997. Car interior sound quality: Experimental analysis by synthesis.

Acta Acustica united with Acustica, 83(5), pp. 813–818. Blauert, J. and Jekosch, U., 1997. Sound-quality evaluation–a multi-layered problem.

Acta acustica united with acustica, 83(5), pp. 747–753. Cook, N., 2000. Analysing musical multimedia. Desmet, P. and Hekkert, P., 2007. Framework of product experience. International journal of design, 1(1). Erkut, C., Serafin, S., Hoby, M. and Sårde, J., 2015. Product sound design: Form,

function, and experience. In Proceedings of the Audio Mostly 2015 on Interaction With Sound (pp. 1–6). Franinovi´c, K. and Salter, C., 2013. The experience of sonic interaction. Sonic interaction design: Fresh perspectives, pp. 39–76. Graakjær, N.J. and Bonde, A., 2018. Non-musical sound branding–a conceptualiza-

tion and research overview. European Journal of Marketing. Hevner, K., 1935. Expression in music: a discussion of experimental studies and theories. Psychological review, 42(2), p. 186.

Hevner, K., 1936. Experimental studies of the elements of expression in music. The American Journal of Psychology, 48(2), pp. 246–268. Huang, Y., Chen, C.H. and Khoo, L.P., 2012. Products classification in emotional

design using a basic-emotion based semantic differential method. International Journal of Industrial Ergonomics, 42(6), pp. 569–580. Hug, D. and Misdariis, N., 2011, September. Towards a conceptual framework to integrate designerly and scientific sound design methods. In Proceedings of the 6th audio mostly conference: A conference on interaction with sound (pp. 23–30). Knoeferle, K.M., Knoeferle, P., Velasco, C. and Spence, C., 2016. Multisensory brand

search: How the meaning of sounds guides consumers’ visual attention. Journal of Experimental Psychology: Applied, 22(2), p. 196. Lageat, T., Czellar, S. and Laurent, G., 2003. Engineering hedonic attributes to generate perceptions of luxury: Consumer perception of an everyday sound. Marketing Letters, 14(2), pp. 97–109. Langeveld, L., van Egmond, R., Jansen, R. and Özcan, E., 2013. Product sound

design: Intentional and consequential sounds. Advances in industrial design engineering, 47(3). Lindberg, S., Roos, A., Kihlstedt, A. and Lindström, M., 2013. A product semantic study of the influence of the sense of touch on the evaluation of wood-based materials. Materials & Design (1980-2015), 52, pp. 300–307. Lindstrom, M., 2006. Brand sense: How to build powerful brands through touch,

taste, smell, sight and sound. Strategic Direction. Ludden, G.D. and Schifferstein, H.N., 2007. Effects of visual-auditory incongruity on product expression and surprise. International Journal of Design, 1(3). Lyon, R., 2000. Designing for product sound quality. CRC Press. Lyon, R.H., 2003. Product sound quality-from perception to design. Sound and

vibration, 37(3), pp. 18–23. Miller, K.W. and Mills, M.K., 2012. Probing brand luxury: A multiple lens approach.

Journal of Brand Management, 20(1), pp. 41–51. Martin, N., Lehmann, J. and Wadle, L.M., Relationship between acoustic perception and overall user experience in vacuum cleaners. Osgood, C.E., 1962. Studies on the generality of affective meaning systems. American

Psychologist, 17(1), p.10. Osgood, C.E., Suci, G.J. and Tannenbaum, P.H., 1957. The measurement of meaning (No. 47). University of Illinois press. Osgood, C.E., Suci, G.J. and Tannenbaum, P.H., 1957. The measurement of meaning

(No. 47). University of Illinois press. Özcan, E. and van Egmond, R., 2006, August. Product sound design and application: An overview. In Proceedings of the fifth international conference on desing and emotion, Gothenburg. Özcan, E. and van Egmond, R., 2008. Product sound design: An inter-disciplinary

approach? Özcan, E. and van Egmond, R., 2012. Basic semantics of product sounds. International Journal of Design, 6(2). Özcan, E., Cupchik, G.C. and Schifferstein, H.N., 2017. Auditory and visual con-

tributions to affective product quality. International Journal of Design, 11(1), p. 35. Sapherstein, M.B., 1998. The trademark registrability of the Harley-Davidson roar: A multimedia analysis. BC Intell. Prop. & Tech. F, 101101, p. 1998. Schifferstein, H.N. and Desmet, P.M., 2008. Tools facilitating multi-sensory product

design. The Design Journal, 11(2), pp. 137–158.

User Emotional Experience Assessment Method of Product’s Intentional Sound 339

Spence, C. and Zampini, M., 2006. Auditory contributions to multisensory product perception. Acta acustica united with acustica, 92(6), pp. 1009–1025. Susini, P., Houix, O. and Misdariis, N., 2014. Sound design: an applied, experimental

framework to study the perception of everyday sounds. The New Soundtrack, 4(2), pp. 103–121. Takada, M., Arase, S., Tanaka, K. and Iwamiya, S.I., 2009. Economic valuation of the sound quality of noise emitted from vacuum cleaners and hairdryers by conjoint analysis. Noise Control Engineering Journal, 57(3), pp. 263–278. Van Egmond, R., 2008, February. Impact of sound on image-evoked emotions. In

Human Vision and Electronic Imaging XIII (Vol. 6806, pp. 162–173). SPIE. Westermann, C.F., 2008. Sound branding and corporate voice–strategic brand management using sound. In Usability of speech dialog systems (pp. 147–155). Springer, Berlin, Heidelberg. Zwicker, E. and Fastl, H., 2013. Psychoacoustics: Facts and models (Vol. 22).

Springer Science & Business Media.

Cover of the proceedings volume Human Interaction & Emerging Technologies (IHIET 2022): Artificial Intelligence & Future Applications

Explore the complete digital volume

Human Interaction & Emerging Technologies (IHIET 2022): Artificial Intelligence & Future Applications

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-44-5

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →