Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Integration of Immersive Learning Simulation and Augmented Reality for Healthcare Training: Towards a Transmedia Solution to Address the Opioid Crisis

1 University of Central Florida College of Medicine, Orlando, FL 32826, USA ↗
2 Lumis Corp, Pittsburgh, PA 15213, USA ↗
Publication year
Digital Object Identifier10.54941/ahfe1002404

Abstract

Immersive Learning Simulation (ILS) and Augmented Reality (AR) technologies hold tremendous potential for transforming training and education for healthcare professionals and patients. While both ILS and AR have demonstrated an unprecedented level of success in a variety of healthcare training and education applications, particularly in the last few years, combining the two approaches remains a relatively uncharted territory. This is largely due to the relative lack of evidence-based methods, strategies, and practical guidance for the design and implementation of mixed reality and transmedia learning environments. This paper presentation will discuss new integration strategies for bringing together the ILS and AR technologies to provide a mixed transmedia learning experience where the virtual and physical components seamlessly co-exist to create a new level of sophistication for education, training, and human performance support. The ILS/AR integration strategies will be presented within the context of an ongoing federally-sponsored research effort that aims to design, develop, and evaluate an innovative learning solution to help address the national opioid crisis with an ultimate goal of improving outcomes for people at risk for opioid misuse and overdose. Focusing on the training needs of healthcare professionals, first responders, and ordinary people (family members, caregivers, etc.) who are on the frontlines of the ongoing opioid crisis, the presented integrated ILS/AR solution illustrates how this transmedia approach can be used for training diverse learner audiences. Based on this research effort, the presenters will share practical insights, lessons learned, and best practices for the design of comprehensive integrated ILS/AR-based solutions that can be easily applied to create effective and engaging learning experiences for a broad spectrum of training, education, and human performance support needs.

INTRODUCTION

The paper describes an ongoing study by an interdisciplinary team of rese- archers to create an innovative simulation-based training solution to help

© 2022. Published by AHFE Open Access. All rights reserved. 221

222 Andrews and Nelson

address the national opioid crisis with an ultimate goal of improving outco- mes for people at risk for opioid misuse and overdose. This effort seeks to advance innovation in the area of opioid use disorder treatment and harm reduction training, with an emphasis on overdose education and naloxone distribution (OEND). By harnessing the power of Augmented Reality (AR) and Immersive Learning Simulation (ILS) technologies, the study aims to design, develop, and evaluate a new learning solution that will be tailored to the training needs of healthcare professionals, first responders, and lay people (family members, caregivers, etc.) who are on the frontlines of the ongoing opioid crisis.

INTEGRATION OF IMMERSIVE LEARNING SIMULATION AND AUGMENTED REALITY The rationale for the integration of ILS and AR technologies during the course of the study is based on the experimental evidence suggesting that mixed reality simulations offer an opportunity to maximize both cognitive and motivational benefits of multiple simulation components by embedding the learner within a dynamic learner-centric system involving both computer- based and embodied interaction (Lindgren et al., 2016; Logeswaran et al., 2021). By leveraging the power of both ILS and AR technologies, the com- prehensive integrated solution will help maximize training efficiency and effectiveness for diverse learner groups, reduce stigma associated with opioid use disorder and overdose, and ultimately address the opioid crisis, which has been significantly exacerbated during the ongoing COVID-19 pandemic.

The new mixed reality solution involves an integration of two comple- mentary instructional systems developed by the project collaborators: an Immersive Learning Simulation for Training Opioid Risk Management in an Interprofessional Environment (STORM-IPE) developed at the University of Central Florida (UCF) College of Medicine (see Figure 1) and the InSight Platform’s Augmented Reality Training for Overdose (ART-OD) Treatment developed by Lumis Corp (see Figure 2), a healthcare training technology company. The STORM-IPE is a virtual patient simulation system designed to foster effective communication and shared decision making with patients and members of an interprofessional team (e.g., physicians, nurses, counse- lors, pharmacists, and others). The STORM-IPE system immerses the learners into contextually rich patient cases to promote knowledge and skill develo- pment across three distinct opioid risk management phases: pain assessment, opioid risk assessment, and treatment/monitoring. Designed to promote inte- grated knowledge acquisition and skill development, this immersive learning simulation enables learner engagement through high fidelity and contextual authenticity of training scenarios.

The Lumis InSight Platform is an advanced projection-based Augmented Reality (AR) system that presents immersive views of human anatomy and physiology coupled with automated instruction through customizable lear- ning modules. ART-OD encompasses learning modules related to recognizing

Integration of Immersive Learning Simulation and Augmented Reality 223

Figure 1: Immersive learning simulation for opioid risk management (STORM-IPE).
Original PDF Figure 1:
Figure 2: Augmented reality training for overdose (ART-OD) treatment.
Original PDF Figure 2:

and responding to an opioid overdose, concluding with scenarios featu- ring patients in various stages of consciousness/physiologic deterioration that require life-saving skills developed during the training.

ILS/AR Integration Strategies: Analysis of Alternatives During the course of the study, the project team have explored several ILS/AR integration strategies for the envisioned solution as summarized in Table 1 below.

Based on this analysis of alternatives, the study team made a decision to pursue the narrative-driven transmedia integration strategy. This decision was motivated by the fundamental goal of creating a learner-centric solution that will maximize learning and engagement.

Transmedia Storytelling Approach to Promote Learning and Engagement Transmedia storytelling, also known as transmedia narrative, is an approach of presenting a single story or experience across multiple digital platforms. Originally coined by Henry Jenkins, the term “digital storytelling” represents a process where integral elements of a fiction get dispersed systematically across multiple delivery channels for the purpose of creating a unified and

224 Andrews and Nelson

Table 1. ILS/AR integration: analysis of alternatives.

Table available in the original PDF paper.

Strategy Pros Cons

  • Reduced contextual authenticity and realism

Seamless Integration

  • Component flexibility
  • Reduced deployment footprint Compartmentalized Integration
  • Independent component management
  • Reduced learner immersion and engagement Narrative-driven Transmedia Integration
  • Potentially increased deployment footprint
  • Learner-centric mix of computer-based and embodied interactions
  • Enhanced engagement and learning

coordinated entertainment experience (Jenkins, 2003, 2006). Ideally, each medium is leveraged to make its own contribution based on the unique set of capabilities to unfold the story. Transmedia storytelling takes advantage of the way the human brain processes information and constructs meaning: syn- thesizing, linking, and exploring multiple sources of information. (Rutledge, 2012). Transmedia storytelling has been extensively used in edutainment, serious games, and other immersive learning experiences and it lends itself particularly well to training and education applications focusing on high- complexity topics requiring higher-order skills, such as critical thinking and decision making. There are multiple transmedia storytelling strategies that have demonstrated success in education and training (Dudacek, 2015; Gron- stedt & Ramos, 2014). The following three strategies have been selected to guide the design of the overarching transmedia learning experience based on their particular relevance to the training and education goals of this study:

  • Situated Learning (SL) takes place through the interaction between peo- ple (represented by real people and/or virtual characters) and building competency through an authentic, contextually rich experience (Oliver & Harrington, 2000; O’Brien & Battista, 2020).
  • Experientiality (E) refers to the ways in which narrative-driven environ- ments tap into the participants’ existing knowledge and familiarity with experience through the activation of “natural” cognitive parameters, the understanding of intentional action, the perception of temporality, and the emotional evaluation of experience (Fewster-Thuente & Batteson, 2018; Carraciolo, 2014). In transmedia environments, experientiality supports both experiential and situated learning.
  • Cognitive Templates (CT), similarly to schemata in social sciences, repre- sent mental structures that learners can practice in a training environment to organize and advance knowledge and skills in the form of cognitive- /mental models, which can eventually be transferred into a real life to guide cognitive processes and behavior (Szulewski, et al., 2020; Azzam & Easteal, 2021).

Integration of Immersive Learning Simulation and Augmented Reality 225

CONCLUSION

Integration of ILS and AR technologies offers unique opportunities for healthcare training and education requiring a thorough consideration of a multitude of factors that have the potential to influence learner engage- ment, motivation, and outcomes. Placing the learner at the center of the mixed reality learning experience through a transmedia storytelling appro- ach and embodied interactions is a way to ensure the learners develop all the necessary building blocks of target competency: knowledge, skills, abilities, and attitudes. Creating learner-centric experiences that are both compelling and authentic is of paramount importance for broad-impact training and education topics, such as opioid risk management and overdose treatment.

ACKNOWLEDGMENT

The authors would like to acknowledge the National Institute on Drug Abuse (NIDA) as the project sponsor, award number 2R44DA050371-02, project collaborators Dr. Analia Castiglioni from the UCF College of Medicine, Mr. Daniel Franceschini from HCA Healthcare, the development team from the Mixed and Emerging Technologies Integration Lab (METIL) of the UCF Institute for Simulation and Training, Dr. John O’Donnell and Dr. Xiaoyan Zhang from Lumis, and the entire development team from Lumis.

REFERENCES

Azzam, M.B and Easteal, R.A. (2021). Pedagogical strategies for the enhancement of medical education. Medical Science Educator, 31, 2041-2048. Carraciolo, M. (2014). The Experientiality of Narrative. DeGuiter. Dudacek, O. (2015). Transmedia storytelling in education. Procedia – Social and

Behavioral Sciences, 197, 694–696. Fewster-Thuente, L. and Batteson, T.J. (2018). Kolb’s Experiential Learning Theory as a Theoretical Underpinning for Interprofessional Education. Journal of Allied Health, 47(1), 3–8. Gronstedt, A. and Ramos, M. (2014). Learning through transmedia storytelling.

Learning Technologies, 31 (1401), ASTD Learning Technologies Community. Jenkins, H. (2003). Transmedia storytelling. MIT Technology Review. https://www.

technologyreview.com/2003/01/15/234540/transmedia-storytelling/ Jenkins, H. (2006). Convergence Culture: Where Old and New Media Collide. New York: New York University Press. Lindgren, R., Tscholl, M., Wang, S., & Johnson, E. (2016). Enhancing learning

and engagement through embodied interaction within a mixed reality simulation. Computers & Education, 95, 174–187. Logeswaran, A., Munsch, C., Chong, Y.J., Ralph, N., McCrossnan, J. (2021).

The role of extended reality in healthcare education: Towards a learner-centered approach. Future Healthcare Journal, 8(1), 79–84. O’Brien, B.C. and Battista, A. (2020). Situated learning theory in health professions education research: a scoping review. Advances in Health Sciences Education, 25, 483–509. Oliver, R. and Herrington, J. (2000). Using Situated Learning as a Design Strategy for

Web-Based Learning. In Instructional Design and Cognitive Impacts of Web-Based Education, pp. 178–191. Idea Group.

226 Andrews and Nelson

Rutledge, P. (2012). What is Transmedia Storytelling? Dr. Pam Rutledge Media Psychology, https://www.pamelarutledge.com/speaking-consulting-2/transmedia -storytelling-is-the-future-of-communication/transmedia-storytelling/what-is-tra nsmedia-storytelling/ Stein, D. (1998). Situated learning in adult education. https://www.ericdigests.org/

1998-3/adult-education.html Szulewski, A., Howes, D., van Merrienboer, J.J.G., & Sweller, J. (2020). From theory to practice: the application of cognitive load theory to the practice of medicine. Academic Medicine, 96(1), 24–30.

Cover of the proceedings volume Training, Education, and Learning Sciences

Explore the complete digital volume

Training, Education, and Learning Sciences

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-35-3

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →