Maria Laura Nappi, Sonia Capece, Camelia Chivăran, Elena Laudante, Mario Buono (2023). Neuro Digital Experiences for Adaptive Museums. In: Human Factors in Accessibility and Assistive Technology. AHFE Open Access, vol. 87, pp. 41–50. https://doi.org/10.54941/ahfe1003652
The interaction between humans, space and advanced technologies evolves into a relationship of “confluence”, a “symbiosis” between the users’ diversified needs and the potential of technologies to return tailored experiences. Being able to analyze the users’ behavior and their cognitive and perceptual-sensory processes enables the structuring of adaptive experiences that place the user at the center of the process of learning and understanding cultural content. The paper provides a critical-analytical recognition of the current models of fruition and interaction used in museum environments with reference to “digital neuro-experiences”. In this regard, “information interchange” contributes to enhance the visitor experience, enabling the activation of a “silent dialogue” between visitors and artefacts and outlining adaptive trajectories with new opportunities for the valorization of cultural heritage.
With the advancement of technologies applied to the cultural field, museums have introduced innovative visiting paths in order to increase the level of emotional involvement of the audience.
However, the introduction of advanced fruition systems has not always coincided with the design of adaptive paths for users, following the Universal Design approaches (NC State University, 1997) and the main regulatory refe- rences and International Guidelines (United Nations, 2015; Come-In, 2019), since making the museum accessible means making it available to everyone, with spaces and content usable by a wide audience without any distinction.
In recent years, leading international museum organizations have high- lighted the need to integrate inclusive strategies that take into account the diversity of users, promoting adaptive design approaches to extend and diver- sify audiences, and improving the museum-visitor connection by applying the Design for All methodology (European Institute for Design and Disability, 2004). In this direction stands the recent update of the museum definition by ICOM - International Council of Museums, which has embraced the concepts of inclusion and accessibility: ≪open to the public, accessible and inclusive, museums foster diversity and sustainability≫(ICOM, 2022).
If well configured, technologies can be the infrastructure able to multiply the opportunities for interaction and participation within the physical and virtual dimensions of museums, fostering accessibility to information even by users with different physical, cognitive and sensory abilities from an inclusive perspective (Marzin, 2018). Alternative and accessible ways of sharing know- ledge can be implemented through multimodal technologies, intuitive visiting systems, and communication channels that amplify the content conveyed by museums and facilitate the learning process in terms of understanding and retention of information by an extended range of users.
In the outlined scenario, it is interesting to analyze the behavior, cognitive and perceptual-sensory processes of user-visitors in order to design innovative experiences that place the user at the center of the process of learning and understanding cultural content.
NEW PARADIGMS OF USER EXPERIENCE DESIGN TO DEVELOP ADAPTIVE MUSEUMS Recent studies have drawn attention to the role of museums in improving people’s personal, cognitive, social, and physical well-being. In particular, they have focused on new design approaches related to the museum experie- nce that the recent changes in society’s technological and cultural paradigms are requiring (Falk, 2021).
Structuring adaptive access to information is essential to enable peo- ple with different backgrounds, physical abilities, knowledge, and interests to access museum content without special physical and cognitive strain (Partarakis et al. 2016; Solima, 2022).
Access, understanding, and use of technologies relate to how people gene- rally interact with a product or service (UNI EN 17161:2019), from access (approach and perception) to understanding (knowledge and decision) and to use (action or reaction). By taking into account greater user diversity, solu- tions and services are defined that are more accessible, understandable, and usable by the widest range of people.
As verifiable from the (UNI EN 17161:2019) guidelines, design should be aimed at as many people as possible without the need for adaptations, so that systems and devices can be used easily, smoothly, and effectively by individu- als with different characteristics, abilities, preferences, or needs. Furthermore, in human-system interaction (UNI EN ISO 9241-20:2022), design tasks can be developed according to a nonlinear but iterative process, within which every task has a role in each of the development phases and feeds the other tasks for reviewing the results based on feedback received from the system and users:
1. “Identity the intended users and the context of use. 2. Analyze, understand and describe the needs, characteristics, capabilities and preferences of the users so that they can be incorporated into the requirements specification. 3. Produce solutions to meet the user requirements. 4. Evaluate solutions against the user’s needs characteristics, capabilities and preferences against the user requirements” (UNI EN ISO 9241- 20:2022).
Neuro Digital Experiences for Adaptive Museums 43
In this process, it is not enough to identify an “average” user (Triberti and Brivio, 2017) but, rather, to design for a plurality of users with diffe- rent characteristics and needs. They can be understood through survey and research methodologies to be transferred to the three different moments of the museum experience: before (to anticipate), during (to track) and after (to analyze) the visit.
Starting from the main strategies in the field of Experience Design, Pietroni (2019) identifies some aspects that can guide designers in the development of effective cultural transmission from the earliest stage of conceiving a new fruition project by posing useful questions to develop an inclusive museum experience: analysis of the target audience (Public and Accessibility); identifi- cation of the objectives and configuration of the visit route (Issues the Project Aims at) and the ways of accessing multimedia content (Condition of Frui- tion) between real and virtual (Relation between Real and Virtual Contents); definition of the contents and communication modes (Communicative Style and Interaction) to be transferred through sustainable technological solutions (Sustainability).
In this direction, the museum evolves from a space of preservation to a place of experience, within which processes and modalities of User Experie- nce Design radically reinterpret the concept of enjoyment of the cultural asset, where the cognitive level is flanked and overlaid by the emotional and bodily dimension of the fruition process. The users’ emotional response, which is difficult to detect but fundamental to defining the different levels of adaptive interaction in the museum space and in contact with the artifacts, should therefore be taken into account.
Our knowledge of visitors’ emotional reactions to the design of museum spaces is strongly influenced by the complexity of the human experience. In fact, during the museum experience, visitors have different emotions ranging from happiness, wonder, empathy, anxiety, and confusion (Rappolt- Schlichtmann, 2020).
Emotional experience is based on complex interactions directly influenced by environmental and individual stimuli, that are highly variable with respect to the different characteristics of the users. In situations of emotional invo- lvement, this latter is embodied in particular objects or experiences, which make us reflect, remember other events, people, spaces and moments rela- ted to the experience. Most meaningfully: intellectual pleasure mixes with that of memory and human relationships, and it creates a unique feeling of involvement through a mental image (Viola and Cassone, 2017).
According to the analysis of the Peabody Essex Museum - one of the first museum institutions to include a team working on neuroscience applied to exhibitions - reported by Banzi (2023), bottom-up factors can be represen- ted by the characteristics of an object or scene and the stimuli with which users come into contact with it through the senses. Top-down factors include instead the background knowledge associated with the interpretation of an artwork or the memories elicited by a particular experience.
In addition to these two aspects of emotional experience, defined as Core affect and Subjective feeling, Rappolt-Schlichtmann et al. (2020) recognize as essential the consideration of a third aspect, Emotional intelligence, intended
as ≪your capacity to govern and leverage your emotion≫that substantially varies in individual emotional experiences in museum spaces.
In this context, new technologies push us to reconsider emotions. Voice humanizes interactions, social robots accompany people with difficulties, vir- tual reality invites us to think deeply about the quality of sensory information and physical experiences (Pavliscak, 2019).
All of this illustrates the potential that the analysis and understanding of emotions have for design, and at the same time it demonstrates the need to not neglect these aspects in designing museum experiences, providing proper consideration to the stimulation of human emotions and perceptions in the fruition paths.
Starting from emotions, it is indeed possible to design for accessibility: ≪the design of supportive emotional conditions promotes the approach and interaction of diverse people with physically accessible exhibits, shaping the emergence of engaged visit experiences≫(Rappolt-Schlichtmann, 2020, p. 142), through an approach that considers the “whole visitor” in their social, cognitive, bodily, and emotional characteristics to create equitable and meaningful museum experiences for visitors.
TRACKING AND “VISUALIZING” DIGITAL NEURO-EXPERIENCES Making museums really adaptive is possible through the design of solutions of fruition based on knowledge on different target audiences and continuous monitoring of visitor experiences. In fact, due to data sensing and tracking technologies, it is possible to “visualize” the “symbiotic” human-machine interaction (Triberti and Brivio, 2017), initiating iterative processes of deve- lopment and verification of cultural products and processes in order to adapt them to users with different physical and perceptual-sensory characteristics, within the same museum space (NC State University, 1997).
Sensors within museum rooms provide the tools to build visitor profi- les, understand needs and provide appropriate responses (Del Bimbo, 2021). Analysis of biometric data, detection of facial microexpressions, posture and body movements are used to recognize emotional states.
By processing data tracked during user experiences, it is possible to predict the needs and expectations of specific targets and structure adaptive ways of transferring cultural information as knowledge on visitors feeds from data about specific types of users.
In this direction, the principles of cognitive psychology - which, according to the approach of Eysenck and Keane (2020), includes neuropsychology, computational cognitive science, and cognitive neuroscience - can support the design of adaptive museum experiences focused on the analysis of cognitive processes and emotions (Banzi, 2023, p. 1).
Within the exhibition “L’ultimo Caravaggio. Eredi e nuovi maestri” set up in 2018 at the Gallerie d’Italia in Piazza Scala in Milan (see Figure 1), surveys were conducted on a sample of thirty visitors, analyzing the psychophysiolo- gical responses - eye movements and central and peripheral nervous system activation - generated by viewing four artworks (Maida, 2018).
Neuro Digital Experiences for Adaptive Museums 45
Figure 1: Exhibition “L’ultimo Caravaggio. Eredi e nuovi maestri” (from Caldato, 2018 and Maida 2018).
Through the use of eye-tracking glasses for visualizing eye movements and points of major focus, a brain electrical activity meter for identifying emo- tional responses, and a bracelet for detecting peripheral micro-suddenness, the different emotional impacts that the four works of art had on the target- visitors, whose experience of artistic enjoyment was studied and evaluated, were recorded (Caldato, 2018). The “Martirio of Sant’Orsola” by Caravag- gio and the “Ultima cena” by Procaccini were found to be the two works of greatest impact and emotional intensity experienced by the sample of visitors.
It is evident that there is a growing interest in recent discoveries in neuro- science that considerably influence design culture and design methodologies such as automatic interpretation of facial expressions that is used to mea- sure and analyze user engagement and emotional reaction; brainwave sensing technologies such as fNIRS, EEG; affective computing software that reco- gnize human behaviors and emotions. In this regard, a study conducted at The Museum of Fine Arts, Boston (2019-2020), uses EEG technology (Muse 2) to analyze user experience in the perception of works through technolo- gies such as Virtual Reality, Augmented Reality, digital copies on iPads, and physical works.
Analyzing the data collected through the peak activation and through the frequency bands for each sensor, it is interesting to observe that the digi- tal representation of the artworks seems not to compromise the aesthetic experience (Cuseum, 2020). In fact, in relation to digital reproductions in Augmented Reality (AR), there was an increase in brain activity compared to the enjoyment of the original works. In reference to FFT analysis, users reported clearer recollection of the painting in AR one week after visiting the museum (Ciecko and Sinha, 2021).
Data collected through these tracking systems can be the knowledge basis for designing new itineraries with attention to the characteristics of diverse users, starting from the identification of points of interest, preferential visitor paths, and involvement in relation to a specific artwork or a defined museum space, whether physical or digital.
Data sensing technologies help to prefigure “digital neuro-experiences” by activating systematic actions useful for designing adaptive experiences that enable:
1. to profoundly alter personal experience; 2. to promote empowerment and psychological well-being; 3. to develop new learning/training procedures; 4. to identify behavioral profiles; 5. to define new modes of interaction; 6. to build integrated experimental paradigms for studying cognitive and behavioral processes in simulated environments (Triberti and Brivio, 2017).
HUMAN-TECHNOLOGY CONFLUENCE AND NEW INTELLIGENT SCENARIOS IN MUSEUMS In a society where people are continuously subjected to the mechanical and fast interactivity of devices, there is a need to develop pathways that can stimulate the imaginary by revealing unprecedented expressive and cogni- tive potential of the relationship with the machine, making every gesture and movement become meaningful, thus moving out of induced behavior (Balzola and Rosa, 2019).
In the context of museum fruition, interaction is smoothly configured, and technology becomes an “invisible” tool within the space and in mediating with artifacts (Triberti and Brivio, 2017).
ArtLens Exhibition, an experiential gallery at the Cleveland Museum of Art (Museums and the Web, 2018) proposes a series of connections between visitors and artworks by encouraging user engagement on a personal and emotional level.
A constantly updated collection of 20 artworks can be enjoyed in barrier- free digital interactive experiences that use gesture-sensing projections that respond perfectly to the body movement. Integrated eye-tracking and facial recognition technology devices record visitors’ emotional reactions with the artworks to reshape them into new personalized experiences. The gal- lery opens up temporal spaces and unprecedented connections between the artworks to the visitors enabling interaction through body language (see Figure 2).
Technology is increasingly integrated into the environment and less inva- sive by disappearing from the user’s perception; it works “automatically”, and it anticipates users’ needs by “learning”, as in the case of Artificial Intel- ligence, to respond automatically to actions, in what Gaggioli et al. (2016) define Human-Computer Confluence: humans and technology converge in advanced scenarios in which to act and react without the need of user input. In such interactive structures such as smart environments, tools, and wearable devices, the interface disappears to make way for spontaneous user actions that act naturally and to which technology responds accordingly.
It is possible to design fruition experiences activated by gestures and acti- ons that the user performs by being guided by the experience to sensorially
Neuro Digital Experiences for Adaptive Museums 47
Figure 2: ArtLens Exhibition, Cleveland Museum of Art (from MW18: Museums and the Web 2018).
explore the space (Buono et al. 2023) and being surprised by what his actions entail (Balzola and Rosa, 2019).
Technology becomes an integral part of the exhibit, and users do not reco- gnize the space of fruition, as it happens in a hyper-mediated dimension (Bolter and Grusin, 2002), but they build personal paths that make the exhibit more engaging. The involvement is often a special relationship, an interaction with an object or situation in which barriers break down, and we become one (Viola and Cassone, 2017).
These technological spaces are designed following the characteristics defi- ned by Triberti and Brivio (2017):
context-aware (it makes use of information detected in the here-and- now);
personal (it is tailored to the needs of the individual user);
predictive (it develops the ability to anticipate users’ needs);
adaptive (it is able to change its functions and behaviors based on users’ habits);
“ubiquitous” (it is embedded and distributed in the interested environ- ment);
transparent or invisible (it is able to operate without direct action, perception, or knowledge of its activeness on the user’s part).
The perfect user-artwork integration mediated through “invisible” tech- nologies in the fruition space makes the experience emotionally satisfying, in an “information interchange” that contributes to enhancing the visitor experience, enabling the activation of a “silent dialogue” between visitors and artworks (Solima, 2022) and outlining adaptive trajectories with new opportunities for cultural heritage valorization.
The symbiosis between the diversified needs of users, tracked in real time, and the potential of technologies to return customized experiences enable the creation of intelligent adaptive spaces and devices, replacing traditio- nal fruition models with new possibilities for the transmission of cultural information.
CONCLUSION
Within museums, the experimentations with advanced intelligent techno- logies introduce new opportunities for creating accessible and inclusive learning experiences to meet the needs and preferences of diverse audiences (Rappolt-Schlichtmann, 2020).
The paper investigated some adaptive fruition approaches within museums and exhibitions to guide designers toward creating adaptive emotional experiences, introducing future symbiotic scenarios in the user-machine relationship within the museum space.
Technology provides effective tools to collect, sort and express data in novel ways; it manifests itself in the multimedia language that brings the narrative to life in a kinetic, fluid and engaging way, but it is above all the lan- guages, the codes used to communicate today, located inside our imaginary and translated into new behaviors (Balzola and Rosa, 2019).
REFERENCES
Balzola, A., Rosa, P. (2019). L’arte fuori di sé: un manifesto per l’età post-tecnologica.
Milano: Feltrinelli Editore. Banzi, A. (2023). “The Brain-Friendly Museum: How Psychology and Neuroscience Can Help Museums to be Brain-Friendly and Promote Wellbeing”, in: The Brain- Friendly Museum, Banzi, A (ed.), pp 1–5. Routledge. Bolter, J. D., Grusin, R. (2002): Remediation: competizione e integrazione tra media
vecchi e nuovi. Milano: Guerini e associati. Buono, M., Capece, S., Chiv˘aran, C., Gerbino, S. Giugliano, G., Greco, A., Laudante, E., Nappi, M. L., Scognamiglio, C. (2023). “Multisensory Fruition Between Cultural Heritage and Digital Transformation”, in: Perspectives on Design and Digital Communication III, Martins, N., Brandão, D., Paiva, F. (eds), pp. 329–355. Springer Series in Design and Innovation, vol 24. Cham: Springer. https://doi.org/10.1007/978-3-031-06809-6_21
Neuro Digital Experiences for Adaptive Museums 49
Caldato (2018). TSW e Gallerie d’Italia Intesa Sanpaolo per la mostra “L’ultimo Caravaggio”. Journal delle Esperienze. TSW Website: https://www.tsw.it/journal/progetti/tsw-e-gallerie-ditalia-intesa-sanpaolo/ Ciecko, B., Sinha, P. (2021). Neurological Perceptions of Art through Augmented and Virtual Reality. Cuseum Report. Cuseum (2020). Neurological Perceptions of Art through Augmented & Virtual
Reality. Cuseum Website: https://cuseum.com/neuroscience. Del Bimbo, A. (2021). Emotions in digital, in: EMOTIONS and LEARNING in MUSEUMS. A NEMO Report by LEM – The Learning Museum Working Group, Mazzanti, P. (ed.), pp. 24–30. Berlin: The Network of European Museum EN 17161:2019 - Design for All - Accessibility following a Design for All approach
in products, goods and services - Extending the range of users European Institute for Design and Disability (2004 [1993]). The EIDD Stockholm Declaration 2004. EIDD-Design for All Europe Website: https://dfaeurope.eu/what-is-dfa/dfa-documents/the-eidd-stockholm-declaration- 2004/ Eysenck, M. W., Keane, M. T. (2020). Cognitive Psychology. A Student’s Handbook.
Oxon-New York: Routledge. Falk, J. H. (2021). The value of museums: Enhancing societal well-being. Lanham, MD: Rowman & Littlefield. Gaggioli, A., Ferscha, A., Riva, G., Dunne, S., Viaud-Delmon, I. (2016). Human
Computer Confluence. Berlin/Boston: De Gruyter Open. ICOM. (2022) Museum Definition. International Council of Museums. ICOM Website: https://icom.museum/en/resources/standards-guidelines/museum-definition/ Interreg Central Europe. (2019) COME-IN!: COME-IN! Linee Guida. Interreg
Central Europe. UNI EN ISO 9241-20: 2022 - Ergonomics of human-system interaction - Part 20: An ergonomic approach to accessibility within the ISO 9241 series. Maida, D. (2018). Musei e Neuroscienza. Alle Gallerie d’Italia di Milano studiano le reazioni emotive del pubblico. Artribune Website: https://www.artribune.com/progettazione/new-media/2018/04/musei- neuroscienza-gallerie-ditalia-milano-studio-reazioni-emotive-pubblico/ Marzin, C. (2018) Plug and pray? A disability perspective on artificial intelligence, automated decision-making and emerging technologies. European Disability Forum (EDF) . EDF Website: https://www.edf- feph.org/content/uploads/2020/12/edf-emerging-tech-report-accessible.pdf Museum and the Web, (2018). Museum and the Web Website: https://mw18.mwconf.org/glami/artlens-gallery/ NC State University. (1997). The principles of universal design. Center for Universal
Design Website: https://design.ncsu.edu/research/center-for-universal-design/ Partarakis, N., Klironomos, I., Antona, M., Margetis, G., Grammenos, D.; Stephanidis, C. (2016) “Accessibility of cultural heritage exhibits”, proceedings of the International Conference on Universal Access in Human-Computer Interaction, Toronto, ON, Canada, pp. 444–455. Pavliscak, P. (2019). Emotionally Intelligent Design. Sebastopol, CA, USA: O’Reilly
Media. Pietroni, E. (2019). Experience Design, Virtual Reality and Media Hybridization for the Digital Communication Inside Museums, Applied System Innovation. 2(4):35. https://doi.org/10.3390/asi2040035
Rappolt-Schlichtmann, G., Todd, K., Daley, S. (2020) “Universal Design for – Emotion in – Learning: A Practice for the Creation of Emotionally Accessible Digital Learning Experiences”, in: Inclusive Digital Interactives: Best Practices and Rese- arch, Ziebarth, B., Majewski, J., Marquis, R. L., Proctor, N. (eds.), pp. 139–166. Access Smithsonian, Institute for Human Centered Design (2020). Solima, L. (2022). Le parole del museo Un percorso tra management, tecnologie
digitali e sostenibilità. Roma: Carocci Editore. Triberti, S., Brivio, E. (2019). User experience: psicologia degli oggetti, degli utenti e dei contesti d’uso. Santarcangelo di Romagna: Maggioli Editore. United Nations (2015). Transforming our World: The 2030 Agenda for Sustainable
Development. United Nations, New York (2015). Viola, F., Cassone, V. (2017). L’arte del coinvolgimento. Emozioni e stimoli per cambiare il mondo. Milano: Hoepli.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors in Accessibility and Assistive Technology
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-63-6
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional