Muzheng Zhu, Xin Xie (2026). An Embodied Interaction System for Five-Tone Music Therapy: A Guqin-Inspired Multimodal Design. In: Affective and Pleasurable Design. AHFE Open Access, vol. 204, pp. 1–12. https://doi.org/10.54941/ahfe1007326
In the fast-paced and high-pressure conditions of contemporary life, chronic stress can aggravate anxiety and depression, increasing the need for gentle, everyday, non-pharmacological ways to support mind–body regulation. Grounded in Traditional Chinese Medicine, this study translates Five-Tone Therapy linking the pentatonic scale ( gong, shang, jue, zhi, yu) with five internal organs into an operable digital, music-based healing experience. Using the guqin’s aesthetic form as the central medium, we develop an interactive therapeutic system from an embodied cognition perspective to facilitate emotional regulation.The study adopts embodied cognition theory and a design research approach using user journey mapping. Key nodes of the therapeutic process, affective rhythms, and system response logic are identified and organized into five sequential stages: entering the application, completing a mind–body assessment, receiving system-recommended therapeutic strategies, conducting self-directed therapy, and reviewing therapy records and longitudinal trends. The paper focuses on personas, namely university students, working professionals, and retired older adults, and designs based on the common pain points shared by all these groups.Based on these findings, we prototype a multimodal system comprising a guqin-inspired therapeutic speaker, a companion mobile app, and tangible props representing the five organs. After a baseline assessment, users trigger corresponding Five-Tone playback by touching a prop with a built-in photosensitive sensor, synchronizing speaker lighting with app visuals to form a multisensory feedback loop. The outcomes offer practical insights for embedding cultural elements in therapeutic product design and for leveraging embodied interaction to enhance user experience and therapeutic effectiveness.
In today’s fast-paced and high-pressure society, chronic stress can readily exacerbate mental health problems such as anxiety and depression. The World Health Organization notes that anxiety disorders are among the most common mental disorders, with an estimated 4.4% of the global population currently experiencing anxiety. Accordingly, identifying gentler mind–body regulation approaches that can be seamlessly integrated into daily life has become a shared concern across contemporary health and design fields.
In recent years, music therapy approaches have attracted broad attention due to their relatively low barriers to use and strong adaptability across Received February 5, 2026; Revised April 4, 2026; Accepted April 21, 2026; Available online July 20, 2026
2 Zhu and Xie everyday contexts. In a study of postoperative patients with lumbar disc herniation, Liu et al. (2024) reported that Five-Tone music therapy combined with acupoint essential-oil massage significantly reduced anxiety and depression scale scores and improved sleep-quality ratings.
Meanwhile, products that leverage the five tones for emotion regulation are also evolving rapidly. Chen et al. (2024) propose an immersive digital Five-Tone healing experience that integrates mappings with physiological data analysis and VR scenarios to enable individualized health-state identification and music–scene matching. Yang and Wang (2025) targeting negative emotions, visualized five-tone frequencies and combined them with multisensory design to create an art installation that enhances emotional awareness and self-regulation. Moreover, current researches lack a pathway to translate Five-Tone healing into embodied interaction, especially lacks a framework that integrates assessment, recommendation, healing, and feedback into a coherent experiential journey.
Therefore, this paper adopts embodied cognition and embodied interaction, using the guqin as the core carrier and the user journey as design support. We develop a multimodal system—comprising a guqin-inspired healing speaker, a mobile app, and tangible props representing the five organs—where action-based interaction triggers synchronized sound–light–visual feedback to convey Five-Tone meanings with low cognitive load, enhancing cultural accessibility and engagement.
RELATED WORK
Research on Music Therapy Music therapy, as a non-pharmacological intervention, has been widely studied and applied in the mental health domain; when combined with other art forms, it can engage multiple sensory modalities and facilitate deeper emotional release. A controlled trial by Rahmani et al. (2016) focusing on depressed adolescents reported that an art-integrated, uplifting music-therapy approach significantly reduced depression scale scores. Eum and Yim (2015) further examined the effects of combining music and painting therapies on alleviating depression and anxiety in older stroke patients, confirmed the adjunctive therapeutic benefits of integrated music–art therapy. Liu et al. (2022) noted that research hotspots in arts therapy span topics such as depression and trauma, and that computer-based systems play a vital role in evaluating therapeutic efficacy.
Research on music-based healing has accumulated positive evidence across both clinical and non-clinical populations. However, many existing digital healing products remain screen-centric and interaction-light, lacking systematic designs that support bodily engagement, which makes it difficult for users to achieve stable relaxation under low cognitive load. Accordingly, the subsequent sections of this paper propose a design approach grounded in embodied interaction and multimodal coupling: leveraging personalized recommendations to enhance user trust and adherence, using multimodal interaction to reduce interpretation costs and strengthen immersion.
An Embodied Interaction System for Five-Tone Music Therapy: A Guqin-Inspired 3 Theoretical Foundation of Guqin Five-Tone Therapy
The guqin, as a traditional Chinese instrument, is closely associated with the five tones, with its core premise emphasizing the harmonizing effects of music on the organs, blood dynamics, and emotional states. Zhao et al. (2025) found that acoustically screened Five-Element music combined with Taijiquan produced beneficial outcomes, supporting the positive role of synergizing sensory stimulation for cognitive and emotional health in older adults. Zhang et al. (2025) further argue that traditional instruments such as the guqin and pipa are deeply rooted in Five-Tone Therapy, linking emotion regulation with body balance. And they also highlight a broader trend in contemporary music therapy toward integrating traditional with modern approaches.
Digital technologies have further accelerated the modernization of Five- Tone Therapy. Chen et al. (2024) proposed an immersive digital Five-Tone healing experience that reconstructs therapeutic processes through virtual reality and multimodal interaction, thereby enhancing user immersion. In parallel, Zhou et al. (2024) developed the Five-Element Harmony System, representing an early effort to operationalize Five-Element theory in practice via an automated pipeline integrating computer vision, music generation, and audio classification, thus bridging Eastern Five-Tone principles with Western therapeutic techniques.
In summary, guqin-based Five-Tone Therapy, grounded in Five-Element theory within traditional Chinese medicine, has demonstrated therapeutic potential across both clinical and digital domains. However, existing research and applications still face limitations, including insufficient embodied- interaction design and inadequate personalization. To address these gaps, this paper integrates the cultural semantics of guqin Five-Tone theory with digital technologies and seeks by strengthening active participation and real-time feedback to deepen therapeutic impact and improve real-world feasibility.
Application of Embodied Multimodal Interaction in Therapy In recent years, embodied interaction and multimodal design have been widely applied in health and therapeutic contexts. Building on Dourish’s embodied-interaction theory, Robert et al. (2023) developed a VR-based embodied multimodal experience framework that helps reduce the cognitive gap between systems and users, and validated its applicability and explanatory power for behavioural research by simulating low-vision navigation scenarios.
In mental health support settings, Vankit et al. (2025) experimentally showed that incorporating multimodal reinforcements, such as subtitles and visualizations, can significantly improve memory retention and promote behavioural change, without increasing cognitive load. In multisensory rehabilitation, Cavdir (2024) collaborated with Deaf participants to co-design a movement-driven system and an audio–tactile compositional vocabulary, demonstrating that on-skin vibrations can enhance participants’ experiences.
Overall, existing studies substantiate the value of embodied, multimodal interaction in therapeutic contexts, yet notable gaps remain. First, there is a lack of cross-context, generalizable design frameworks that unify body, space, and multisensory-based interaction into a coherent therapeutic design
4 Zhu and Xie methodology. Second, pathways remain underdeveloped for translating space and physical objects into sustainable, long-term therapeutic experiences. In response, this paper proposes an embodied, multimodal interaction design scheme for typical therapeutic scenarios, leveraging a guqin-inspired speaker and tangible organ props to create a daily-use therapeutic device that extends the boundaries of therapeutic space and helps address gaps in systematic design approaches and long-term effectiveness validation.
METHOD
User Journey Map During the user-journey mapping stage of this study, we adopted a desk- research approach to ensure the specificity and effectiveness of the design solution. Desk research uses secondary data that can be gathered without conducting fieldwork. Evidence was extracted from authoritative statistical sources, industry reports, and feedback from online communities (Charles, 2024). Overall, we employed desk research to conduct exploratory qualitative analysis and comparatively examine three typical user personas. Then synthesized the findings into design tools such as user personas and user- journey maps to inform subsequent functional definition and interaction design of the system.
By reviewing mental-health surveys from the WHO, we extracted prevalence estimates of anxiety and depression across populations. A BMC Psychiatry study of 688 freshmen reported 49% depressive and 75% anxiety symptoms; the general adult population has a 6% major depressive disorder and 4%-7% generalized anxiety disorder prevalence. WHO data show 14% of adults aged over 60 have mental disorders, contributing 10.6% of disability burden in this group. These data reflect the widespread psychological pressure on working professionals and demand for supportive workplace environments. We further developed a detailed user-journey map (see Figure 1), mapping user behaviours, emotional fluctuations and potential opportunities across each stage.
Figure 1: User journey map.
An Embodied Interaction System for Five-Tone Music Therapy: A Guqin-Inspired 5 Design Framework
The theoretical foundations of embodied interaction are rooted in phenomenological traditions. In Where the Action Is, Paul Dourish (2001) introduced the concept of embodied interaction, emphasizing that meaning is created through action situated in the physical and social world. In other words, embodied interaction is not merely an operational procedure but a holistic experience encompassing cognition, affect, and social engagement. From an embodied-interaction perspective, people are no longer passive recipients of information.They actively participate in situated interaction and co-construct meaning with their surrounding environment through bodily action.
Accordingly, therapeutic system should not be confined to a screen-based pipeline from selection to play, but should translate key understandings into practices that are perceivable and actionable. Framing embodied interaction as a three-layer framework, converts embodied-cognition principles into implementable interaction strategies, supporting a multimodal therapeutic system for user participation and deep therapeutic resonance, as shown in Figure 2.
QINYUN FIVE-TONE HEALING SYSTEM Research on Music Therapy
Building on the user journey and embodied-interaction framework described above, the Qinyun Five-Tone Healing System does not aim for a one-off intervention. Instead, it situates its application scenario as an everyday music-therapy experience. Prior research indicates that the effectiveness of digital mental health interventions is often associated with user engagement and adherence, yet real-world use commonly involves attrition and low- frequency utilization. Therefore, sustained use should be treated as a core objective (Forbes et al., 2023). Moreover, interaction design for health contexts emphasizes reducing learning costs and cognitive load to enhance usability and long-term usability (Langote et al., 2024).
To realize a sustainable therapeutic experience system, the Qinyun Five-Tone Healing System built on the user journey and overall design framework, adopts a multimodal setup including a mobile app, guqin-inspired healing speaker, Five-Element devices, and a sound-visualization layout. The subsequent section details the functional design of each component.
Mobile Application Design The mobile application acts as the central hub of the experiential system, enabling users to learn about and interpret their states while guiding them through the entire therapeutic journey, thereby lowering product adoption barriers. Guided by clarity, calmness and low cognitive burden, the app’s visual language adopts a unified blue–green palette to reduce excess ornamentation, ensuring key touchpoints deliver sustainable, engaging and comprehensible user experiences.
Architecturally, the application begins with the Evaluation entry point and connects five core modules: Home, Therapy, Community, Market, and Mine (see Figure 3). Home supports everyday use scenarios, while Therapy operationalizes the primary therapeutic functions and mapping mechanisms, helping users translate abstract states into actionable therapeutic pathways. Community enables social connection and communication, provides a dedicated area for user-experience feedback. Market primarily hosts product offerings and follow-up services. Mine consolidates personal information, messages, and saved items, supporting individualized settings and record management. This tightly integrated structure enables users across different personas to operate the product efficiently through a unified entry point.
Figure 3: Mobile application architecture design.
In terms of interaction flow, the app uses progressive question-and-answer interaction to assess users’ body states, transforming emotions and bodily sensations into structured response for initial screening. In the recommendation stage, the system generates suggestions via five-tone, five-organs, and five- element mapping, presenting playlists through intuitive visualizations and offering user-controlled adjustments to balance personal preferences. During therapeutic execution, the interface switches to a low-interference playback mode to minimize on-screen information and prevent disruptions to the therapeutic process. In the review stage, the app automatically produces
An Embodied Interaction System for Five-Tone Music Therapy: A Guqin-Inspired 7 a brief report for each session and integrates longitudinal data into trend visualizations, such as mood curves and sleep-quality calendars (see Figure 4). By supporting reflection on trajectories of mind–body change and fostering deeper meaning-making, this strategy constitutes a key design mechanism for sustaining long-term engagement.
Figure 4: Application interface and interaction process design.
Guqin-Inspired Healing Speaker The guqin-inspired speaker serves as the material carrier and the interaction hub of the therapeutic system, merging music playback with culturally rooted aesthetic healing. Its design draws from the Tang-dynasty guqin Jiuxiao Huanpei and reinterprets magnolia imagery for a contemporary expression. The main body blends high-transparency resin with wood veneer, using warm materials to echo the guqin’s serene, refined character; its hollowed structure with vine-like growth lines makes it an elegant piece embodying natural aesthetics and cultural symbolism even at rest (see Figure 5).
As the multisensory feedback medium of the therapeutic system, the speaker provides visible cues through rhythm-driven light-and-shadow dynamics, helping users disengage from text-based interpretation of therapeutic progress. The first is a seven-string spotlight array that generates dynamic light flows synchronized with musical rhythm during playback, visually rendering string vibration and sonic energy. The second is a magnolia- shaped soft night light that can deliver either steady ambient illumination or interaction-triggered atmospheric lighting, thereby establishing an immersive
8 Zhu and Xie and tranquil luminous environment. Accordingly, audio–light coupling enables the guqin-inspired speaker to move beyond a utilitarian therapy device, extending the therapeutic experience into physical space and aligning it with everyday living contexts.
Figure 5: Five-tone healing speaker design.
Five-Tone Interactive Sensing Device To better embed the therapeutic process into everyday routines, five props symbolizing the Five elements are deployed as a haptic interface bridging digital recommendations and embodied experience (see Figure 6). After the app completes mind–body assessment and outputs Five-Elements- based suggestions, users avoid repetitive interface navigation, selecting and activating via direct touch of the relevant prop. This advances therapy from cognitive interpretation to bodily participation and lowers operational burden. The five props are respectively designed to correspond to the Wood, Fire, Earth, Metal, and Water. They incorporate each element’s core imagery together with artistic illustrations of its associated organs.
Figure 6: Five-tone healing sensing props.
An Embodied Interaction System for Five-Tone Music Therapy: A Guqin-Inspired 9 In terms of interaction mechanism, each prop embeds a photoresistor sensor to detect user-triggered actions. When a user touches the prop for the recommended result, the system plays matched Five-Tone therapeutic music, synchronizing ambient light and audio-visual feedback to make the therapy perceptually explicit. Stable rhythmic dynamics and consistent feedback amplify immersion via immediate multisensory responses, reframing healing as a repeatable, contextualized ritual beyond passive listening (see Figure 7).
At the visual level, the system implements real-time audiovisual feedback and interactive mechanisms built with Touch Designer, assigning distinct visual styles to music associated with different attributes. For example, Wood-attributed music may drive a particle system in which branching vines grow across the screen, whereas Water-attributed music corresponds to fluid ripple flows (see Figure 8). The system also supports user interaction with the audiovisual output, enabling participation in the therapeutic rhythm at lower operational cost and thereby further strengthening embodied engagement.
Figure 8: Touch designer’s real-time audio-video feedback and interaction.
Throughout the overall therapeutic experience, users’ touch behaviors function not only as activation signals but also as keys that open a multimodal immersive space. This aligns with a central claim of embodied interaction— namely, that meaning is not confined to symbolic interpretation but is directly constructed and perceived through ongoing interactions between the body and the world.
CONCLUSION
This study addresses the need for lightweight emotion regulation in everyday contexts by using the Qinyun Five-Tone Healing System as a medium to effectively translate traditional cultural wisdom into a digital health intervention. By adopting an embodied-cognition perspective and integrating tangible props with multimodal, real-time feedback, the design transcends the conventional paradigm of screen tapping and passive listening in digital healing. Through low–cognitive-load bodily actions, users can directly activate a multisensory feedback environment that encompasses auditory and visual modalities. In addition, the system leverages a mobile application to establish interpretability and retrospective reviewability of the healing process. Visual explanations of assessment-based recommendations enhance user trust. Long-term logging and reflection further help users integrate fragmented sensory experiences into a narrative understanding of their personal state changes, thus facilitating effective somatic healing.
Although this study proposes a relatively complete design framework and implements a prototype, several limitations remain.
While the study advances from framework development to prototype implementation and workflow validation. It confirms the plausibility of the intervention through user-journey mapping and an embodied-interaction framework. However, evidence of real-world therapeutic efficacy and long- term effects remains limited, due to the lack of controlled experiments, longitudinal follow-up, and multidimensional quantitative evaluations for improvement. Meanwhile, the current system largely relies on one-off assessments to drive Five-Element recommendations and shows insufficient capability for dynamically adapting to fluctuations in user state. Future work could incorporate richer data dimensions and lightweight re-assessment mechanisms to form a more continuous, adaptive healing pathway.
In summary, grounded theoretically in embodied interaction and semantically in the cultural foundations of the Five Tones and the Five Elements. It proposes and operationalizes a design pathway that translates traditional mind–body healing into a digitally embodied experiential form. In response to stress and emotional distress in modern life, a gentle and sustainable form of daily healing provides a practical interactional route. This route lays the groundwork for subsequent validation and iteration across broader populations and scenarios, collectively advancing human- centred, culturally integrative digital health interventions.
ACKNOWLEDGMENT
This research was supported by Stable Support Program for Higher Education Institutions (General Program), Shenzhen Science and Technology Innovation Commission (GXWD20220817131619001).
REFERENCES
Cavdir, D. (2024). Development of embodied listening studies with multimodal and wearable haptic interfaces for hearing accessibility in music. Frontiers in Computer Science, 5, 1162758. Chen, Y., Guo, R., Pan, Y., Li, Y., Yu, L., & Yang, P. (2024, November). Immersive
Digital Pentatonic Healing Interactive Experience. In Proceedings of the Twelfth International Symposium of Chinese CHI (pp. 688-693). Chen, Y., Guo, R., Pan, Y., Li, Y., Yu, L., & Yang, P. (2024, November). Immersive Digital Pentatonic Healing Interactive Experience. In Proceedings of the Twelfth International Symposium of Chinese CHI (pp. 688-693). Dourish, P. (2001). Where the action is: the foundations of embodied interaction.
MIT press. Eum, Y., & Yim, J. (2015). Literature and art therapy in post-stroke psychological disorders. The Tohoku journal of experimental medicine, 235(1), 17-23. Forbes, A., Keleher, M. R., Venditto, M., & DiBiasi, F. (2023). Assessing patient
adherence to and engagement with digital interventions for depression in clinical trials: systematic literature review. Journal of Medical Internet Research, 25, e43727. Kwon, C. Y., Kim, H., & Kim, S. H. (2024, February). The modernization of oriental music therapy: five-element music therapy combined with Artificial Intelligence. In Healthcare (Vol. 12, No. 3, p. 411). MDPI. Langote, M., Saratkar, S., Kumar, P., Verma, P., Puri, C., Gundewar, S., & Gourshettiwar,
P. (2024). Human-computer interaction in healthcare: Comprehensive review. AIMS Bioengineering, 11(3). Liu, Q., Chen, J., Zhou, J., Xie, R., & Wu, J. (2024). The Impact of Five-tone Music Therapy Combined with Compound Essential Oil Acupoint Massage on Postoperative Negative Emotions and Sleep Quality in Patients with Lumbar Disc Herniation. Liu, Z., Yang, Z., Xiao, C., Zhang, K., & Osmani, M. (2021). An investigation into
art therapy aided health and well-being research: a 75-year bibliometric analysis. International Journal of Environmental Research and Public Health, 19(1), 232. Okonkwo, C. (2024). Assessment of user experience (UX) design trends in mobile applications. Journal of Technology and Systems, 6(5), 29-41. Rahmani, M., Saeed, B. B., & Aghili, M. (2016). Integrating effect of art and
music therapy on depression in adolescents. Journal of Educational Sciences & Psychology, 6(2). Robert, F., Wu, H. Y., Sassatelli, L., Ramanoel, S., Gros, A., & Winckler, M. (2023, June). An integrated framework for understanding multimodal embodied experiences in interactive virtual reality. In Proceedings of the 2023 ACM International Conference on Interactive Media Experiences (pp. 14-26). Vankit, S. A., Sabnam Shama, N., Diaz, D., & Volonte, M. (2025, September).
Multimodal Reinforcement in Screen-Based ECA Health Interactions: Effects on Memory Retention, Dietary Attitudes, and Cognitive Load. In Proceedings of the 25th ACM International Conference on Intelligent Virtual Agents (pp. 1-10). Yang, M., & Wang, Y. (2025, August). Design of Emotional Experience Art Installation Based on Five-Tone Therapy. In Proceedings of the 2025 International Conference on Generative AI and Digital Media Arts (pp. 215-220). Zhang, S., Charanyananda, A., Natchanawakul, N., Tayrattanachai, N. O., &
Buranaprapuk, A. (2025). The Literacy of Traditional Chinese Music and Modern Practices in Contemporary Music Therapy in China. International Journal of Education and Literacy Studies, 13(3), 126-135.
Music Combined with Traditional Chinese Mind–Body Exercises on Emotion Regulation, Working Memory, and Functional Brain Connectivity in Older Adults: A Randomized Repeated-Measures Study. Behavioral Sciences, 15(5), 699. Zhou, Y., Bian, W., Zhang, K., & Gu, X. (2024, November). Advancing Music Therapy: Integrating Eastern Five-Element Music Theory and Western Techniques with AI in the Novel Five-Element Harmony System. In 2024 IEEE 14th International Symposium on Chinese Spoken Language Processing (ISCSLP) (pp. 234-238). IEEE.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Affective and Pleasurable Design
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-80-9
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional