Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

The Application of Visual Translation in Synaesthesia to Product Design

Beijing Institute of Technology, Beijing 100081, China ↗
Publication year
Digital Object Identifier10.54941/ahfe1002027

Abstract

The purpose of the study is to extract and construct the development and method of synaesthetic visual translation in product design, provide new design thinking, and enrich user experience. This paper employs a mixed method of case study, practice, fuzzy evaluation, etc. Through case analysis, the manifestation of visual translation in product design is summarized, the translation law between vision and other senses is abstracted, a method to achieve visual translation is built, and research conclusions are verified through design practice. Based on fuzzy evaluations, the development of synaesthetic visual translation is established, and a reusable visual translation design method is proposed, which enriches product connotation while improving comprehensibility, and realizes a multi-dimensional and in-depth user experience.

INTRODUCTION

Products under the background of the digital era are increasingly homogene- ous in form, interaction, and perception, whereas users are setting higher demands for the emotional experience of products. Hence, it is essential to explore new innovative design methods to enrich the connotation levels and interactive experience of products. Synaesthetic translation, a design method mobilizes more users’ sensory modalities, is getting more attention and application in product design.

NOTIONS RELATED TO SYNAESTHESIA Synaesthesia, dependent on sensory perception and psychological perception experience, responses to memories and cognitive experience and triggers associations to another sense (Wang, 2017). Therefore, the development of synaesthesia goes through at least two stages, from the physiological perce- ption stage to the mental image stage. External objects stimulate senses to produce physiological perceptions. The receiving subject combines the envi- ronment and moods at the time, and through psychological processing, the “five senses” are imaged and transformed into subjective remembered expe- riences. It is of strong subjectivity in this stage. When cognitive “five senses” appear on external elements and are perceived by the subject, the association and the cognitive “five senses” form a mapping to trigger mood and envi- ronment memories to complete synaesthesia (see Figure 1). Synaesthesia in

© 2022. Published by AHFE Open Access. All rights reserved. 149

150 Gong and Ji

Figure 1: Synaesthetic translation model.
Original PDF Figure 1:

design extracts “five senses” images at the cognitive level to stimulate users’ physiological senses to trigger associations, leading the mood and environ- ment memories to the reflection level, and ultimately making users have emotional resonance with products. Sensation translation at the cognitive level enriches users’ sensory experience, and the intercommunication of ima- ges at reflection level satisfies users’ emotional appeal (Zhang, L.H. et al. 2018).

BASIS OF SYNAESTHESIA DEVELOPMENT Physiological Basis of Synaesthesia

Synaesthesia develops through the superposition, translation, and integration of vision, hearing, smell, touch, and taste, mainly acting on the cognitive level. Senses, including vision, hearing, taste, smell, and touch, are the fundamental way for humans to receive information from the outside world and the basis of creative thinking (Yu et al. 2018). Humans obtain external material infor- mation through their senses and establish connections between senses and objects through the brain. Studies have proofed that knowledge and memo- ries are stored in a neural network, and the connection mode of neurons in units constantly changes. Brains can seek and establish connections between objects. Therefore, synaesthesia mainly occurs in the process of cranial nerve stimulation that transforms objective perception into subjective impression. This process has strong subjectivity.

Psychological Basis of Synaesthesia The psychological basis of synaesthesia is that the individual physiological intuitive feelings are mapped to the psychological level under the drive of experience, memory, association, emotion, etc., and the information recei- ved by senses is integrated and transformed to perceptions (Zhang and Lin, 2018), mainly acting on the cognitive level. The cross-sensory phenomenon based on the psychological level has a certain metaphor and is constrained by subjects’ experience and imagination (Zhang, G. 2013). For example, que- nching thirst by thinking of plums and drawing cakes to allay hunger all use

The Application of Visual Translation in Synaesthesia to Product Design 151

Figure 2: Cognitive model of Synaesthetic translation between visual and tactile Sense.
Original PDF Figure 2:

associations to map physical intuitive feelings to the psychological level, link visual information with existing experience and memory, and then translate them into the stimuli to other sensory perceptions. If the subject’s experience is missing or beyond imagination, for instance, a subject who has no empiri- cal cognition of the acidity of plums can not be able to complete the mapping of plum’s visual image and taste, and synaesthesia can not be completed.

SYNAESTHETIC VISUAL TRANSLATION AND ITS MANIFESTATION IN PRODUCT DESIGN Since people perceive more than 80% of external information through visual sense, the most common approach of synaesthetic translation is through visual translation.

Synaesthetic Translation between Visual Sense and Tactile Sense After touching a material, the subject will have an overall subjective impres- sion of the shape, color, light perception and tactility of the material. When the visual image information of shape, color and light perception related to the material appears, it triggers the subject’s association to evoke the tactile memory, forming the mapping of vision and touch to complete synaesthetic translation (see Figure 2).

The visual manifestation of the texture features on product surface is mainly by extracting the visual texture features of related objects and the morphological characteristics of image schema to recall user’s tactile memo- ries, trigger perceptual cognition at the psychological level, and stimulate user’s behaviors to the reflection level, so that users can resonate with the emotions that products try to deliver.

Synaesthetic Translation Between Visual and Auditory Sense “bright” is often used to describe the purity and intensity of colors in vision, but it is also often used to describe the strength and pitch of a sound. This indicates there is a mapping relationship between visual patterns and the auditory sense. Objective things are perceived by visual sense and auditory sense, and stored in memories in the form of image schema through the subject’s mental processing. When the auditory related image schema at the cognitive level is visually perceived as the source of stimulus, the subject’s

152 Gong and Ji

Figure 3: Cognitive model of Synaesthetic translation between visual and auditory sense.
Original PDF Figure 3:
Figure 4: Synaesthetic translation model of visual sense and gustatory or olfactory senses.
Original PDF Figure 4:

association is triggered. The thinking operation and cognitive activity of asso- ciation map and replace sound related image schema and sound sources. Meanwhile, in the process of recall, the environment and moods when the first memory was stored are naturally triggered, and the “reproduction” of the moods and environment triggers emotional resonance (see Figure 3).

Synaesthetic Translation between Visual Sense and Gustatory or Olfactory Senses Quenching thirst by thinking of plums, drawing cakes to allay hunger, and a beauty to feast one’s eyes on all describe the synaesthetic translation between visual perception and taste or smell. When objective things are perceived by the subject, taste or smell information is often represented by more concrete and easy-to-capture visual image information, and is stored at cognitive level in the form of image schema. When the image schema is captured by the visual sense, the triggered association recalls specific taste or smell informa- tion it represents, in short, the synaesthetic translation between vision and taste or smell is completed (see Figure 4). The synaesthetic translation betw- een vision and smell or taste mainly relies on subject’s memories and cognitive experience. It develops into abstract subjective cognition based on speci- fic perceptual cognition of a certain object, and leads to specific cognition through association and memories accordingly (Wang, 2017).

The translation between vision and smell or taste is common in the packa- ging design of food and perfume. Designers use product images or visual elements with specific taste and smell, such as colors and appearances, to communicate the taste or smell information they represent to users and convey product features intuitively.

The Application of Visual Translation in Synaesthesia to Product Design 153

To sum up, the synaesthetic visual translation in product design usually uses the appearance or visual elements of the objects with specific five senses to arouse the memories of hearing, touch, taste, smell, and other attributes through visual manifestation, and then the characteristics of the objects they represent are used to convey the characteristics of products, including usage mode, functions, etc. The manifestations in product design can be roughly divided into: appearance, usage, and working principle (Zheng, 2017). No matter which way the synaesthesia is presented, it requires the help of sensory perception and psychological processing, and depends on people’s experience and imagination.

VISUAL TRANSLATION DESIGN PRACTICE OF PRODUCT MODELING DESIGN Visual Translation Process of Product Modeling Design

Through the argument above, it can be seen an essential factor for succes- sful synaesthetic visual translation is to evoke users’ relevant remembered experience. Therefore, when using synaesthetic thinking in product design, extracting the image schema at the cognitive level as the basis of the mode- ling element is the key to triggering the user’s associative memory and then achieving emotional resonance.

In the selection of image schema, the key is to establish the connection between the image schema of the product appearance and the usage or functi- onal characteristics of products. The screening steps include: first, analyze the target product from the three aspects of appearance, usage, and functional characteristics, and determine the design entry point; secondly, define which type of sense to be translated from vision and identify related image vocabu- laries, and based on the translation model of vision and other senses, with the Likert scale to verify the correlation between the user’s touch, smell, taste, or hearing and the synaesthetic vocabularies, and obtain the most appropriate sensory experience factor; based on the translation model, the fuzzy eva- luation method is used to extract design elements; finally, the abstract and simplified methods are used to simplify the figurative modeling factors into modeling language symbols with the characteristics of factors that are easy to identify and produce, and generate a design scheme (see Figure 5).

Extraction of Visual Image Taking the modeling design of vertical vibration equipment as an example, the approach and process of visual translation and synaesthetic translation model are used to conduct product modeling design practice. Vertical vibra- tion equipment generates an indirect and continuous upward thrust against the gravitational force through mechanical force. In other words, it stimula- tes the movement of rope skipping, so users can take full-body exercise from bottom to top, outside to inside.

Firstly, based on the functional characteristics of vertical vibration equi- pment that vibrates when working, identify the visual images more related to it. The typical visual images of vertical vibration equipment voted by the

154 Gong and Ji

Figure 5: Design flow chart.
Original PDF Figure 5:
Table 1. User evaluation form.

Table available in the original PDF paper.

Grades Image vocabularies u1 u2 u3 u4 u5

v1 4 5 4 6 6 v2 9 4 5 4 3 v3 3 8 6 4 4 v4 3 5 8 6 3 v5 3 3 6 8 5

design practice team are: gradient blur, density change, and rhythm swing. Based on 3 visual images, 5 image words were selected, and the final image words were determined according to the mathematical model of fuzzy eva- luation. 25 users were invited to evaluate the 5 image words selected. The evaluation indicators and weights used were: recognition degree 0.3, satisfa- ction degree 0.2, guidance degree 0.1, reflection degree 0.2, aesthetics 0.2. So get the weight vector W= (0.3, 0.2, 0.1, 0.2, 0.2), set the number of judging grades as 5, V= (v1, v2, v3, v4, v5) , v1 as the best, v2 as good, v3 as medium, v4 as bad, v5 as extremely bad; the judgment value is 5, 4, 3, 2, 1. Image voca- bulary set is: U= (u1, u2, u3, u4, u5), u1 for rhythm, u2 for diffusion, u3 for agitation, u4 for swaying, u5 for phantom. The image words were evaluated (see Table 1).

Convert Table 1 to a 5x5 matrix.

[Equation or mathematical symbol available in the original paper PDF]

Equations and mathematical formulas are available in the original paper PDF.

4 5 4 6 6 9 4 5 4 3 3 8 6 4 4 3 5 8 6 3 3 3 6 8 5

Convert the 5×5 matrix into a single-factor fuzzy evaluation matrix R.

The Application of Visual Translation in Synaesthesia to Product Design 155

Table 2. The scale of correlation between Synaesthesia imagery and original sense.

Table available in the original PDF paper.

(a) (b) (c) (d) frequency frequency frequency frequency Can trigger Strongly agree 7 12 9 7 tactile agree 6 9 7 10 (vibration) moderately agree 9 6 6 5 association disagree 5 2 6 7 strongly disagree 4 1 2 1

[Equation or mathematical symbol available in the original paper PDF]

[Equation or mathematical symbol available in the original paper PDF]

. (2)

Equations and mathematical formulas are available in the original paper PDF.

0.16 0.2 0.16 0.24 0.24 0.36 0.16 0.2 0.16 0.12 0.12 0.32 0.24 0.16 0.16 0.12 0.2 0.32 0.24 0.12 0.12 0.12 0.24 0.32 0.2

Calculate the composite result of W and R with the composite calculation of a fuzzy matrix.

[Equation or mathematical symbol available in the original paper PDF]

[Equation or mathematical symbol available in the original paper PDF] (0.3, 0.2, 0.1, 0.2, 0.2)

Equations and mathematical formulas are available in the original paper PDF.

0.16 0.2 0.16 0.24 0.24 0.36 0.16 0.2 0.16 0.12 0.12 0.32 0.24 0.16 0.16 0.12 0.2 0.32 0.24 0.12 0.12 0.12 0.24 0.32 0.2

= (0.264, 0.2, 0.116, 0.224, 0.168) . (3) Through data analysis, it can be seen that the image vocabulary rhythm has the highest score, so the association with the tactile senses is constructed by taking “rhythm” as the reference of the visual image modeling element.

Correlation Construction of Visual Images and Tactile Sense Based on the selected synaesthesia image word “rhythm”, four visual ima- ges were presented, and the correlation between synaesthesia image and the original sense was verified. Selected 30 users as subjects, and used the Likert scale to score the four images that can trigger vibratory tactile associative memory. Strongly agree is 5, agree is 4, moderately agree is 3, disagree is 2, and strongly disagree is 1. A total of 30 valid questionnaires were obtained (see Table 2).

Through the questionnaire analysis, image (b) has the highest scoring rate. When subjects receive visual stimulation from image (b), the sensory memory of vibration is most likely to be triggered. So, (b) is selected as the image modeling factor.

156 Gong and Ji

Figure 6: Transformation of modeling design language.
Original PDF Figure 6:
Figure 7: Product rendering.
Original PDF Figure 7:

Transformation of Visual Modeling Design Factors and Modeling Design Language Based on the image (b), the thinking is expanded. The image (b) is a water droplet falling into the still water surface and causing layers of ripples. It has the beauty of rhythm, and the ripples are orderly. Visually, it has a dyna- mic feeling, which has a close mapping relationship with the tactile feeling of vibration. There are orderly lines that spread and alternate, which can be regarded as the main feature of the product surface texture; visual color and touch temperature have a corresponding mapping relationship. Since the visual image is water, giving a clear and smooth texture, so in the color sele- ction, more consideration should be given to cool colors such as blue and white. In terms of material selection, more consideration can be given to materials with smooth texture and lightweight (see Figure 6).

According to the derived product modeling design language, in the mode- ling design of vertical vibration equipment, the surface texture of products should adopt simplified and abstract ripple patterns. Since vertical vibration equipment is in a state of vibration during work, the rubber material with large roughness is used on the pedal to improve the safety; the body shell is made of acrylic material with a glossy and smooth texture, and the organic shapes are superimposed at the corners, giving users a visual stimulation of

The Application of Visual Translation in Synaesthesia to Product Design 157

ripple diffusion, which in turn triggers the user’s tactile experience and achie- ves the translation between vision and touch, and brings a multi-dimensional and deep user experience; in terms of color, the color white is chosen, which echoes the pure and transparent texture of water droplets, and brings light and concise feeling to the user at the same time (see Figure 7).

CONCLUSION

Synaesthetic translation in product design is realized based on the user’s cognition, associative ability, and remembered experience. Exploring core laws and design methods of synaesthetic translation in product design not only enriches the product manifestation, but also broadens and improves the perception channel and cognitive efficiency of the product, bringing a unique experience to users and providing new perspectives for the product design.

ACKNOWLEDGMENT

We are grateful for the support of the “Research on Evaluation of Age- friendliness and Livability Index in Beijing” the Key Project of Beijing Social Science Foundation (No. 18YTA001).

REFERENCES

Wang, Y. (2017), “Research on Application of Synaesthesia Sense Transfer in Product Design”, MA thesis, Jiangsu University, Zhenjiang Wang, Z.T. (2017), “Latent Semantic Similarity Based Interpretation of Synaesthetic

Metaphors”, MA thesis, Xiamen University, Xiamen Yu, S.L., Mao, Y.M. (2018). Synaesthesia Elements and Synaesthesia Design Based on the Design of Naoto Fukasawa, PACKAGING ENGINEERING. Volume 39. pp. 24–28 Zhang, G., Pan, Z.P., and Wang, J.P. (2013). Research on Synaesthesia Rhetoric

Technique in Product Design. PACKAGING ENGINEERING. Volume 18. pp. 25 Zhang, K., Lin, H. (2018). Application of Sensory Transfer of Synaesthesia in Product Design. PACKAGING ENGINEERING. Volume 39. pp. 20–23 Zhang, L.H., Lou, M. and He, J. (2018). Art Synaesthesia Translation in the Sen-

sory Experience of Product Symbol, PACKAGING ENGINEERING. Volume 39. pp. 14–19 Zheng, Q.R. (2010), “Research on Application of Synaesthesia in Design of Touch Screen Interface”, MA thesis, Jiang Nan University, Wuxi.

Cover of the proceedings volume Interdisciplinary Practice in Industrial Design

Explore the complete digital volume

Interdisciplinary Practice in Industrial Design

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-24-7

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →