Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Internet of Senses (IoS) and Internet of Sensory Health (IoSH): A New Technology Epiphany

1 IADE, Universidade Europeia, Av. D. Carlos I, 4, 1200-649 Lisboa, Portugal; UNIDCOM/IADE - Unidade de Investigação em Design e Comunicação, Av. D. Carlos I 4, 1200-649 Lisboa, Portugal ↗
2 Politecnico di Milano, Department of Design, School of Design, via Durando 10, 20158 Milano, Italy ↗
Publication year
Digital Object Identifier10.54941/ahfe1001403

Abstract

In the field of healthcare design, a great revolution is taking place. The use of Internet of Things (IoT) technologies, and in particular the great success of smartwatches, fit bands, and specific wearable medical devices allowed people to self-monitor their health parameters. At the same time, physicians were allowed to track, assist, make diagnoses, and prescribe treatments, remotely. Everything is done with the collection of and analysis of high-quality data, assisted by Artificial Intelligence (AI) (Forgan; 2021; Chiapponi & Ciotti, 2015; Islam et. al, 2015). At the same time, the easier accessibility to sensory technology, the high-speed internet connection, and rate of adoption of Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR), which are altogether known as eXtended Reality (XR), is accelerating innovations, with the digital experiences expected to become even more immersive.According to a recent report of Sony Ericsson® the biggest trend for future technology developments until 2030, will be the evolution from IoT (Internet of things) to IoS (Internet of Senses). Nowadays, we can use XR solutions to support patients’ recovery, promote mental health, or treat chronic pain conditions, but, digital communications are still audio-visual. In the next decade, it is expected that devices, sensors, and actuators, and software would enable these experiences to become even richer, through the concomitant use of all our senses, and merging the digital and the physical reality. This type of experience is based on the Internet of Senses.This revolution will drive designers to create more immersive environments. Future experiences with the diffuse use of haptic feedback will be enriched with digital flavors/aromas, more sophisticated haptic stimulations, and immersive interfaces. Future experiences are going to involve multiple sensory modalities, opening interesting possibilities for multi-sensory and cross-sensory interactions. (Sony Ericsson, 2020)These observations are particularly interesting for the field of Synesthetic Design, («synaesthesia» from the Greek syn," together", and aisthēsis, "sensation", literarily “perceiving together”), the study of sensory perception is used to design sensory stimuli with the specific purpose of “contaminating” other different sensory modalities (senses) changing the nature of stimuli. All the sensations can be coordinated based on the systematic connections between different modalities”. (Haverkamp, 2014). How this revolution is going to affect the world of healthcare?The Internet of Senses revolution will open important horizons for designers, responsible for the sensory characterization of everyday experiences. In this paper, we are going to introduce what are the opportunities of implementing Internet of Senses technologies for healthcare. To do so, we are going to present and discuss a Case-Study (a between-subjects experience involving 42 participants) in which a synesthetic design approach has been used to reduce the sensation of pain in people (using cold-induced pain CPT). The study has been realized creating an immersive experience based on cross-sensory interactions in a sensory-controlled environment. Particular attention will be given to the methodological aspects of the study.

INTRODUCTION

The concept of Internet of Things, also known with the synonyms ‘web of things’, ‘internet of objects’, ‘connected devices’, has been defined as: “An open and comprehensive network of intelligent objects that have the capacity to auto-organize, share information, data and resources, reacting and acting in face of situations and changes in the environment” (Madakam et al., 2015).

From the beginning, the concept of Internet of thing have been very focused on technology, rather than on people’s needs. Historically, the first connected object was a coke vending machine at Carnegie Melon University in 1982. As soon as the Internet was created the intention of connecting more ‘things’ together was manifested: in 1990, a year before the launch of the World Wide Web, John Romkey (Founder of FTP Software) & Simon Hackett connected a bread toaster to the Internet.

Despite these pioneering experiences, the term ‘Internet of Things’ (IoT) was used for the first time in 1999 by Kevin Ashton (at that time the Executive

© 2022. Published by AHFE Open Access. All rights reserved. 313

Director of Auto-ID Labs in MIT) in a Presentation about RFID, a bre- akthrough technology for that time (Ashton, 2010). From that moment, the development of sensors and micro-electric devices, the diffusion of smartphones (iPhone, 2008), and the improvement of the communication infrastructures (3G, 4G, and then 5G), allowed the creation of new digi- tally identifiable and mutually connected objects, able to “interact with each other and with the external world by exchanging data over the existing Inter- net infrastructure” (McEwen & Cassimally, 2013) (Vitali et al., 2016). In 2008 the number of connected objects surpassed the number of people on Heart.

From Situations and Changes in the Environment of Health Things: A Technology Epiphany With the re-elaboration of the Internet of Things known as Internet of Every- thing (IoE) (Cisco®, 2013) the network is no longer composed only by objects and data, but people were for the first time considered the nodes of the network.

The success of these devices in field as smart-home, smart-cities, and in smart-health, is the result of the identification of new meanings applied to breakthrough technologies. According with Roberto Verganti: “when a new technology emerges, it can simply substitute the obsolete one, by keeping the existing meaning […] or can “hide” a more powerful quiescent meaning - related to new possible uses/applications - that is waiting to be uncovered”. (Verganti, 2009) This manifestation of hidden meanings has been defined by Roberto Verganti (Professor of Leadership and Innovation at the Stockholm School of Economics and Harvard Business School) with the term ‘technolo- gical epiphany’ and it helps companies to discover the technology full value and to create products or services that are considered more meaningful than the current ones.

For example, the terms Internet of Health Things (IoHT) or Internet of Medical Things (IoMT) describes all the applications of the IoT technologies for medical and health related purposes. The development of IoHT devi- ces (e.g. pacemakers, smartwatches, hearing devices, sphygmomanometers, etc.) allowed the possibility of interconnecting medical devices for monito- ring remotely general health parameters for heart-related diseases, diabetes, etc. (e.g. blood pressure, heart rate, glycemia), making remote diagnose (i.e telemedicine), and generating of emergency notifications (e.g. fall detection for elderlies). The above-mentioned innovations allowed the local health Institutions to reduce the hospital-related costs.

From an economical perspective, if the Internet of Things (IoT) Mar- ket size was estimated around $310 billion in 2020 (and is anticipated to reach around USD 1842 billion by 2028), the market of Internet of Health Things market was estimated around $30.79 billion in 2021 (the 10% of the IoT market) and is projected to grow from to 187.60 billion in 2028. (Fortune Business Insights, n.d.). If IoT technology represents a technological advance, with the definition of Internet of Everything and the subsequent Internet of Health Things, a new use of these technology

Internet of Senses (IoS) and Internet of Sensory Health (IoSH) 315

Figure 1: Verganti’s framework for innovation strategy applied to IoT, IoE and IoHT.
Original PDF Figure 1:

was found, and a new meaning was unveiled. We can describe this pro- cess using Roberto Verganti framework for innovation strategy (fig.1) (Verganti, 2009): Internet of Senses, a New Breaktrough Technology is Emerging

“Imagine an immersive experience of a beach where you can feel the wind blowing on your face, the heat of the sun on your back, the humidity of the ocean air on your skin, and experience the fresh smell of the ocean breeze - right into your living room.” (Ericsson, 2020).

Current audio-visual and haptic devices are already allowing people to have digital experiences very similar to the physical ones. According to a recent report (Sony Ericsson, 2020), the biggest trend for 2030 will be the creation of an Internet of Senses: a new technology paradigm in which digi- tal experiences will become more multisensory and therefore fully immersive. In this scenario devices, sensors, and networks, supported by a large use of Extended Reality (XR), will merge the digital and the physical reality, invo- lving all our senses. These experiences will be possible thanks to the new generation of connected devices and sensors, that will be able to:

  • improve the quality of XR experiences (supported by holographic repre- sentation)
  • introduce new sensory experiences as digital flavors and digitals aroma.
  • create new interactions will be created using brain-computer Interfaces.

At this point a question naturally arises: how can we apply IoS technology to human-factors?

From a design perspective, the development of IoS technologies opens inte- resting scenarios for multi-sensory and cross-sensory design approaches in Human Centered Design. Particularly interesting is the possible contribution of these technologies for Synesthetic Design.

Synesthetic Design is a field of research that studies the systematic conne- ctions between the different sensory modalities. How? When we interact with a stimulus (let’s imagine a smell - or a digital smell – of a pizza) we recognize that it doesn’t trigger only the sensory system which has been directly stimu- lated (in this case the olfactory system), but also other senses that, even if they are not directly stimulated (i.e gustatory system), contribute to provide a more complete information about the stimulus (a perception of the taste of the pizza). Synesthetic Designers study and design these correlations/con- taminations between different modalities. Indeed, “any designed stimulus is able to solicit, refer to and interact with the whole concomitant information modifying the intrinsic nature of stimuli” (Anceschi & Riccò, 2000).

The effects of stimuli are evaluated within the specific sensory moda- lity (the main sense triggered by the solution) or in correlation with other modalities. The main objective of Synesthetic-Design is to achieve the opti- mal configurations, based upon the systematic connections between the modalities (Haverkamp, 2013), Synesthetic Design approaches have been successfully used in various fields of design: from inclusive design (cre- ating accessible design for blind people) to communication design, from Automotive Design to Design for Health and Well-being.

Internet of Sensory Health (IoSH): A New Technology Epiphany?

In the UX/UI Lab of UNIDCOM/IADE, Universidade Europeia of Lisbon, we imagined a new meaning for IoS, technology (fig. 2), analogously to what happened with Internet of Health Things (IoHT) and Internet of Things (IoT): if health has been a successful strategy for generating new meanings for Internet of Everything (IoE), can health be a successful strategy for generating new meanings for the Internet of Senses?

The Sensuous Project is a current study in which we are exploring the possibilities of using a synesthetic design approach to create an immersive multi-sensory experience (fig.3), with the use of current IoT devices, with the aim of evaluating the impact of concomitant environmental sensory stimuli on pain perception.

The study consists of a 4x2 within-subject design study based on a Cold Pressor Test, involving 42 participants.

The effects of sensory stimuli on pain perception have been demonstrated with studies involving concomitant sensory stimulation in different moda- lities, proposing mono-medial interventions to distract, motivate or create counter-irritations that might reduce pain, involving concomitant sensory stimulation in different modalities:

  • Tactile (Boensch, 2011; Chen & Johnson, 2010; Riley & Levine, 1988)
  • Acoustic (Basínski et al., 2018; BURT & KORN, 1964; Garcia & Hand, 2016; Gardner et al., 1960; Gardner & Licklider, 1959; Howitt & Stricker, 1966; Mitchell et al., 2006; Morosko & Simmons, 1966)
  • Visual (Rahimi et al., 2013)(Yousuf Azeemi & Raza, 2005) Considering that every stimulus in the environment is likely to be designed (Rognoli et al., 2016) and considering that the design of each stimulus pro- duces effects in other sensory registers (Riccò, 1999, 2008) it was decided to

Internet of Senses (IoS) and Internet of Sensory Health (IoSH) 317

Figure 2: Internet of Sensory Health as a technology epiphany.
Original PDF Figure 2:
Figure 3: A picture of the experimental set-up of the sensuous project.
Original PDF Figure 3:

use a Synesthetic design approach to create a multi-sensory immersive expe- rience. The objective of the study is to identify which sensory combination between stimuli of different modalities could have a stronger impact in terms of analgesic effects.

DISCUSSION

Internet of Senses represents a breakthrough technology for the next years. The diffusion of IoS device may lead to new innovations of meaning regarding new possible uses/application of these technologies.

In this scenario, the diffusion of IoS technologies for immersive multi- sensory experience opens interesting scenarios for the development of sensory-based strategies unveiling new quiescent meanings regarding the pro- motion of peoples’ health. We could define this technology epiphany with the term ‘Internet of Sensory Health’ (IoSH).

Some possible IoSH scenarios are:

  • IoSH technologies for the remote treatment of chronic pain conditions (e.g. fibromyalgia; phantom-limb pain; etc);
  • IoSH technologies for pain management in specific hospital divisions (e.g. post-surgery care; labour rooms, dental surgery, etc);
  • IoSH technologies for promoting relaxation in people with autistic spectrum; In general, the adoption of a Internet of Sensory Health technology could be a strategy to humanize healthcare settings, generally considered stressful, chaotic and unpleasant places, (Ulrich, 1991, 2000, 2005; Ulrich et al., 2008) adopting a Synesthetic Design approach.

ACKNOWLEDGMENT

This work was supported by the grant SFRH/BD/129136/2017 to Davide Antonio Gambera, given by the FCT, Fundação para a Ciência e a Tecnolo- gia (Portuguese Science Foundation). This study was also conducted at the UNIDCOM, UX.Lab and supported by the Fundação para a Ciência e Tecno- logia (FCT), under Grant No. UIDB/00711/20120 attributed to UNIDCOM – Unidade de Investigação em Design e Comunicação, Lisbon, Portugal (unit 711).

REFERENCES

Anceschi, G., & Riccò, D. (2000). Research of Communication Design: a synesthetic approach. In S. Pizzocaro, A. Arruda, & D. De Moraes (Eds.), Design plus Rese- arch, Proceedings of the Politecnico di Milano conference, Politecnico di Milano, may 18-20 (pp. 1–7). https://doi.org/10.13140/2.1.2121.4089 CITATIONS 0 2 Ashton, K. (2010). That Internet of Things’ thing. That “Internet of Things” Thing-

RFID Journal. http://www.rfidjournal.com/article/print/4986 Basínski, K., Zdun-RyZewsk, A., & Majkowicz, M. (2018). The Role of Musical Attributes in Music-Induced Analgesia: The Role of Musical Attributes in Music- Induced Analgesia: A Preliminary Brief Report. https://doi.org/10.3389/fpsyg. 2018.01761 Boensch, S. (2011). Stimulation-produced analgesia: TENS, acupuncture and alter-

native techniques. Anaesthesia & Intensive Care Medicine, 12(1), 28–30. https: //doi.org/10.1016/J.MPAIC.2010.10.003 BURT, R. K., & KORN, G. W. (1964). Audioanalgesia in Obstetrics. “White Sound” Analgesia During Labor. American Journal of Obstetrics and Gynecology, 88(3), 361–366. https://doi.org/10.1016/0002-9378(64)90436-3 Chen, C. C., & Johnson, M. I. (2010). A comparison of transcutaneous electrical

nerve stimulation (TENS) at 3 and 80 pulses per second on cold-pressor pain in healthy human participants. Clinical Physiology and Functional Imaging, 30(4), 260–268. https://doi.org/10.1111/j.~1475--097X.2010.00936.x

Internet of Senses (IoS) and Internet of Sensory Health (IoSH) 319

Cisco®. (2013). The Internet of Everything Cisco IoE Value Index Study. Ericsson. (2020). 10 Hot Consumer Trends 2030 – The internet of senses - Ericsson. Report. https://www.ericsson.com/en/reports-and-papers/consumerlab/repor ts/10-hot-consumer-trends-2030 Fortune Business Insights. (n.d.). Internet of Medical Things [IoMT] Market Size and

Growth, 2028. Retrieved April 6, 2022, from https://www.fortunebusinessinsight s.com/industry-reports/internet-of-medical-things-iomt-market-101844 Garcia, R. L., & Hand, C. J. (2016). Analgesic effects of self-chosen music type on cold pressor-induced pain: Motivating vs. relaxing music. Psychology of Music, 44(5), 967–983. https://doi.org/10.1177/0305735615602144 Gardner, W. J., & Licklider, J. C. R. (1959). Auditory analgesia in dental operations.

The Journal of the American Dental Association, 59(6), 1144–1149. https://doi. org/10.14219/JADA.ARCHIVE.1959.0251 Gardner, W. J., Licklider, J. C., & Weisz, A. Z. (1960). Suppression of pain by sound.

Science, 132, 32–33. https://doi.org/10.1126/science.132.3418.32 Giambattista, A. (2017). Designing Care. How Design can improve medical products for a therapeutic wellbeing. The Design Journal, 20(sup1), S2158–S2167. https: //doi.org/10.1080/14606925.2017.1352732 Haverkamp, M. (2013). Synesthetic design : handbook for a multisensory approach

(W. de Gruyter (ed.)). Birkhäuser. https://books.google.pt/books?id=NfLTAAA AQBAJ&dq=michael\protect$\relax+$haverkamp\protect$\relax+$synesthetic\p rotect$\relax+$design&hl=it&source=gbs_navlinks_s Howitt, J. W., & Stricker, G. (1966). Objective evaluation of audio analgesia effects.

Journal of the American Dental Association (1939), 73(4), 874–877. https://doi. org/10.14219/jada.archive.1966.0292 Madakam, S., Ramaswamy, R., Tripathi, S., Madakam, S., Ramaswamy, R., & Tripathi, S. (2015). Internet of Things (IoT): A Literature Review. Journal of Computer and Communications, 3(5), 164–173. https://doi.org/10.4236/JCC.2015.35021 McEwen, A., & Cassimally, H. (2013). Designing the Internet of Things. Wiley.

https://books.google.pt/books?id=iYkKAgAAQBAJ Mitchell, L. A., MacDonald, R. A. R., & Brodie, E. E. (2006). A comparison of the effects of preferred music, arithmetic and humour on cold pressor pain. European Journal of Pain, 10(4), 343–351. https://doi.org/10.1016/j.ejpain.2005.03.005 Morosko, T. E., & Simmons, F. F. (1966). The Effect of Audio-Analgesia on Pain

Threshold and Pain Tolerance. Journal of Dental Research, 45(6), 1608–1617. https://doi.org/10.1177/00220345660450060701 Rahimi, M., Makarem, J., & Rooyan, P. (2013). Effects of a flash of light in different colors on venous cannulation pain: A randomized, controlled trial. Journal of Clinical Anesthesia, 25(1), 42–46. https://doi.org/10.1016/j.jclinane.2012.06.006 Riccò, D. (1999). Sinestesie per il design. Le interazioni sensoriali nell’epoca dei

multimedia. Etas. Riccò, D. (2008). Sentire il design : sinestesie nel progetto di comunicazione.

Carocci. https://books.google.pt/books/about/Sentire_il_design.html?id=A0G7N wAACAAJ&redir_esc=y Riley, J. F., & Levine, F. M. (1988). Counterstimulation and pain perception: effects of electrocutaneous vs. auditory stimulation upon cold pressor pain. Pain, 35(3), 259–264. https://doi.org/10.1016/0304-3959(88)90135-2 Rognoli, V., Ferrara, M., & Arquilla, V. (2016). ICS_Materials: materiali interattivi,

connessi e smart. MD Journal, 2(SINAPSI. DESIGN E CONNETTIVITÀ), 44–57. http://materialdesign.it/media/formato2/allegati_6021.pdf

Ulrich, R. S. (1991). Effects of Interior Design on Wellness. In Journal of health care interior design. Ulrich, R. S. (2000). Effects of Healthcare Environmental Design on Medical

Outcomes. http://www.capch.org/wp-content/uploads/2012/10/Roger-Ulrich-W CDH2000.pdf Ulrich, R. S. (2005). The environment’s impact on stress. In Improving Healthcare with Better Building Design. Ulrich, R. S., Zimring, C., Zhu, X., DuBose, J., Seo, H.-B., Choi, Y.-S., Quan, X., &

Joseph, A. (2008). A review of the research literature on evidence-based healthcare design. HERD, 1(3), 61–125. Verganti, R. (2009). Design-driven innovation : changing the rules of competition by radically innovating what things mean. Harvard Business Press. https://books.go ogle.com/books/about/Design_driven_Innovation.html?hl=it&id=rpaj0vLzPRkC Vitali, I., Rognoli, V., & Arquilla, V. (2016). Mapping the IoT: Co-design, test

and refine a design framework for IoT products. ACM International Conference Proceeding Series, 23-27-Octo. https://doi.org/10.1145/2971485.2987681 Yousuf Azeemi, S. T., & Raza, S. M. (2005). A critical analysis of chromotherapy and its scientific evolution. Evidence-Based Complementary and Alternative Medicine, 2(4), 481–488. https://doi.org/10.1093/ecam/neh137

Cover of the proceedings volume Human Dynamics and Design for the Development of Contemporary Societies

Explore the complete digital volume

Human Dynamics and Design for the Development of Contemporary Societies

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-01-8

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →