Elisabeth Broneder, Christoph Weiß, Jaison Puthenkalam, Youssef Ibrahim, Markus Karlseder, Daniela Weismeier-sammer, Nathan Coyle, Monika Psenner, Astrid Holzinger (2025). A scalable MR-training system to prepare military and civil experts for crisis areas. In: Human Interaction and Emerging Technologies (IHIET 2025). AHFE Open Access, vol. 197, pp. 240–250. https://doi.org/10.54941/ahfe1006717
The goal of the EU’s Common Security and Defence Policy (CSDP) is strengthening the civilian and military crisis management. Optimally prepared personnel are essential to efficiently meet security challenges and actively work towards conflict prevention and peace keeping. Several EU initiatives and training institutions across EU Member States help prepare personnel for these challenges. Involved stakeholders are civilian, military, and police personnel as well as representatives of government institutions and non-governmental organisations (NGOs). In crisis situations, effective collaboration among stakeholders is vital. Deployed personnel require extensive skills including monitoring, mediation, situational awareness, intercultural competence and first aid skills. Personnel deployed in crisis zones are required to make decisions under extreme stress. Currently, training programs typically combine theory with practical components such as role plays and simulation exercises. However, real-world simulations often fall short of realistically replicating crisis scenarios due to organizational and financial constraints. Additionally, especially dangerous situations cannot be safely recreated, further limiting the realism and effectiveness of such training. Virtual and mixed reality technologies offer promising new possibilities for simulating crisis areas and for training personnel in a realistic and immersive way. Such technologies allow personnel to engage in training with minimal logistical burden. They provide scalable, cost-effective, repeatable and immersive training, reducing dependency on large-scale physical simulations. SkillDrill provides an innovative MR framework, providing learning modules to develop and train essential skills such as mission planning, situational awareness, map reading and advanced first aid for self-aid and buddy care, in an immersive way. Depending on the trained skill, the system offers different layers of immersion (digital, VR, MR). Building upon the results of the end-user requirements and methodologies described in (Broneder, 2024), this paper will describe the current state of the SkillDrill system as well as the first results of the end user tests.
A Scalable MR-Training System to Prepare Military and Civil Experts for Crisis Areas 241
INTRODUCTION
The EU’s Common Security and Defense Policy (CSDP) has the goal to strengthen civilian and military crisis management. It is crucial to consider the different dimensions (humanitarian, civilian, military, diplomatic) of the mission in the planning, preparation and execution phases where the cooperation of many actors is essential. The success of the mission is dependent on the quality of planning and preparedness of the deployed personnel which requires a large variety of skills – from mentoring, mediation and monitoring to patient care and self-emergency treatment. Often, these skills must be performed in extremely stressful and potentially dangerous situations. When operating in such environments, deployed personnel require a high-level of situational awareness and a solid understanding of appropriate behaviour to minimize risks to personal safety and security. Current trainings cannot optimally prepare the deployed personnel for these stressful situations as strategic planning and mission preparation come with high costs, high logistical efforts and often lack digital support. Simulation exercises for e.g. safety and security trainings typically require the involvement of multiple actors. They are held as in-person events and, given the high costs, can only be conducted once. This means that repetitions are not possible, and trainees have no opportunity to train independently.
Virtual and Mixed reality trainings offer a higher immersion into the simulation and thus provide more convincing training settings, allowing the training of stressful situations that are usually costly or hard to train because they cannot be convincingly recreated in real life (Skryabina, 2017). VR and MR applications in the training domain specifically target emergency first responders like EMR (emergency medical responders) (Mossel, 2021) or firefighters (Broneder, 2024). Various training scenarios (e.g., CBRN, building fire) can be successfully simulated in MR and cooperation between different first responder organizations efficiently trained. VR and MR training of personnel for CSDP missions has received less attention. Regarding peace keeping and humanitarian aid, the focus is on understanding cultural sensitivity and on security tasks, especially for military personnel, police, and NGOs (Ragazzoni, 2015) (Trzci´nska, 2019).
EMR training requires hands-on experience especially for skill-training, therefore the haptic feedback during training is essential and VR training is often not sufficient. This haptic feedback can be achieved by overlaying training manikins with injured virtual avatars. Various approaches for tracking the avatar can be used. Some use green screen technology (Vourvachis, 2022), others use marker tracking and inertial sensors (Broneder, 2024). The latter approach allows the manikin to be moved within the real environment and does not restrict the training to a pre-configured green room. The SkillDrill system builds upon this approach and enhances the manikin tracking via new algorithms and sensor fusion technology.
This paper describes a modular training framework, along with the developed tools, to train the most important skills needed for the planning of and preparation for CSDP missions. Depending on the immersion level aspired, different technologies are applied. For planning CSDP missions, a
digital module is implemented. Training of situational awareness and map reading in crisis areas is realized via VR modules, and the training of self and buddy first aid is trained via MR to allow for real patient treatment with a haptic experience using real equipment.
SKILLDRILL TRAINING FRAMEWORK During the user requirements phase of the project, various training modules were defined which are implemented at different levels of immersion: (a) strategic planning of missions, (b) preparation for missions via situational awareness training and map reading, (c) self-aid and buddy care in crisis areas. In the following the scenarios for the three modules are described.
Digital Tool for Strategic Planning The quality of strategic planning is of uttermost importance to the success and effectiveness of CSDP missions. Since most strategic planners have a military background, the character of civilian missions must first be learned so that the planners can ask appropriate questions that help to define objectives, mandate, duration, exit strategy, risks and challenges more precisely. Current planning courses take place once a year without any digital support. Trainees are provided with a scenario description and a fictitious paper map and are tasked with assessing the situation. Once they have conducted a conflict analysis, they are asked to develop options for potential missions. Since the training of strategic planning is based on fixed scenarios using physical maps and information, it is complex to grasp the scenario and analyse the political situation within the limited course time. Therefore, we offer an interactive web-based digital map that increases the level of detail of the presentation of the political situation in the operation area and thus provides a better overview of the situation. Trainers can thereby choose a certain area of the world as a training map, making the tool highly adaptable. Trainees can view locations on the map and gain a better overview of the situation. Digital content can be added quickly and easily by trainers and trainees. This makes it possible to better understand the physical environment and socio-political dynamics in the area of operation. An interactive map prepared by the training staff is provided, in which data on demographics, natural resources, administration, police, justice, customs, etc. are visualized. Depending on the scenario, the map shows, among other things, conflict hotspots and refugee camps. Important points in the crisis zone can feature additional information material (both texts and images) to get a more precise idea of the situation on the ground. The interactive map can be used to display various scenarios. Planning skills can be trained remotely since trainers can invite trainees to a certain training group which share the same map. Trainees can add additional information to the map.
VR Module for Situational Awareness Training (SAT) When going into the mission, experts expose themselves to danger in the unknown territory and possible unexpected situations that can be
A Scalable MR-Training System to Prepare Military and Civil Experts for Crisis Areas 243
life-threatening. Therefore, situational awareness training is crucial to evaluate various situations accordingly. Currently, real-life simulation trainings are used for situational awareness training, but due to the high effort (personnel, costs), the full variety of potential threats cannot be recreated accordingly. SkillDrill offers a VR-based module for realistically simulating the complexity and unpredictability of crisis zones. When working in such environments, mission personnel require a high level of alertness, the ability to quickly assess their surroundings, and an understanding of adequate behaviour to minimize security risks. In the VR-module trainees are placed in a highly detailed virtual representation of a rural village affected by violent conflict. They are tasked to assess their surroundings and identify immediate or potential security threats. The simulation includes approx. 40 randomised security threats - such as illegal checkpoints, minefields and gunfire - of which 5–7 are dynamically selected per scenario run. This ensures that each session presents a unique challenge, enhancing learning transfer and avoiding memorisation of static environments.
VR enables repeated exposure to a diverse range of threat scenarios, fostering deeper cognitive pattern recognition and quicker situational responses, regardless of physical location, weather conditions, or trainer availability. All features in the simulations are crafted based on expert input from experienced professionals in CSDP and peacekeeping missions, ensuring high accuracy and operational relevance. The training finishes with an interactive assessment interface (feedback board, see Figure 2), allowing users to review their performance. Identified and non-identified threats appear on the board as imminent or potential security threats. This immediate feedback loop together with repeated practice, whether with or without instructor supervision, facilitates self-improvement.
VR Module for Map Reading In order to navigate in an unknown crisis area, map reading skills are essential when technical equipment fails. SkillDrill provides a VR module for training essential map reading skills required for orientation and navigation in the field. The module covers basic map reading principles, such as how to read coordinate systems like the Universal Transverse Mercator (UTM) grid, how to read conventional map scales, and how to calculate real-word distances. Understanding these principles helps participants navigate and interpret maps more accurately. The module provides a progressive series of interactive exercises intended to enhance the user’s fluency in interpreting maps. Participants are instructed in the recognition and interpretation of symbols from a standard legend and are tasked with locating and identifying specific objects on the map, utilising a random assignment of Universal Transverse Mercator (UTM) grid coordinates. Each task is permitted three attempts before revealing the correct answer, incorporating a gamified element that promotes engagement and encourages trial-based learning. To facilitate comprehension of distances, a virtual measuring tool is provided, enabling users to practically apply map scales when estimating movement ranges or planning patrols. To maximise the potential of VR, the user can
transform the 2D map into a topographical 3D view (see Figure 3). This easy- to-use feature helps users understand changes in height and the formation of the land better. The immersive nature of this feature further improves spatial awareness and facilitates deeper cognitive integration of the learning material. The training environment contains a database of almost 200 map objects, of which several are randomly selected and placed on the maps, ensuring varied and non-repetitive learning experiences.
MR Module for Training Self-Aid and Buddy Care Life-saving skills for self-help and buddy aid are essential in missions, especially for civilian experts and military that do not have any medical education. Currently basic first aid skills are taught via training manikins and by trainers explaining what the status of the patient is (e.g., injuries, vital parameters). The training is constantly interrupted by trainer feedback and thus not immersive. Apart from that, certain stress factors when working in the field in a crisis area cannot be simulated in classrooms. Real-life simulations with actors are used for training patient care, but variations of training procedures are difficult and time-consuming because injuries have to be recreated using makeup. Therefore, the simulation of virtual scenarios is a suitable tool for supplementary training. MR was chosen over VR, since real equipment and training manikins have to be included within the virtual scenario to maintain the haptic feedback. In doing so, SkillDrill offers an MR module for training critical bleeding control and performing cardiopulmonary resuscitation (CPR), since these are the two most essential skills that were identified within the requirements phase of the project. The integration of equipment (e.g. tourniquet) and training manikins create a realistic and convincing training simulation. As injuries, gunshot wounds and deep cuts are integrated since these are the injuries that have to be commonly treated in missions. To train users with different qualifications and skill levels, the difficulty can be adapted by the trainer by adding different stressors and varying the injury patterns.
TECHNICAL OUTCOMES Digital Tool for Strategic Planning
The strategic planning tool is a web service built upon the web platform Django, its geospatial expansions and the map tool Leaflet. It provides a lightweight e-learning platform, giving trainers the possibility to upload course materials on several topics, to assign trainees to courses, to form groups of trainees and to create simulation exercises. Trainees can upload result documents for their groups.
The main part is formed by the map view for simulation exercises (see Figure 1). This provides methods for selecting regions for the simulation exercise, drawing borders and towns on the map, placing map labels as well as creating types of points of interest (such as transport routes, water treatment facilities, ceasefire zones) and placing such points. Trainees can view the given scenario and enhance it with their own information and
A Scalable MR-Training System to Prepare Military and Civil Experts for Crisis Areas 245
points of interest which are only visible by their group. Trainers can view the progress of groups on the map through a view that shows each group’s entered information.
Figure 1: Strategic planning tool showing a point of interest, borders and map labels (left) and the placement of a map label (right).
VR Modules for Situational Awareness and Map Reading The VR modules developed for Situational Awareness and Map Reading are developed in Unity3D with OpenXR for the Meta Quest 3 headset to deliver high-impact, scalable and mobile training experiences for CSDP mission preparation. The Situational Awareness module uses room-scale VR and teleportation to put users into a fictional village that has been affected by violent conflict. The environment automatically adds random threats, so the same scenario can be used again without having to change anything. Trainees can interact with objects in the environment to train scanning and threat assessment in a way that is not possible with traditional simulations (see Figure 2).
Figure 2: Situational awareness scene – feedback board to check results (left), unexploded ordinance objects spawned in village (right).
The Map Reading module uses VR to allow users to scale maps up to wall- sized formats for detailed analysis. Another useful feature is that 2D maps can be turned into 3D models of the land, which facilitates understanding
contour lines. A virtual measurement tool can be used to estimate distances more accurately. Figure 3 shows a screenshot of the 3D and 2D map.
Figure 3: Map reading: 3D view of a 2D map (left), selected objects on wall map with right and wrong selections (right).
Both modules include built-in tutorials, allowing for fully unguided use, relieving the trainer. Session data, like how well the user performed (which task, how long it took, etc), is automatically logged. Trainers could use this data to adapt their teaching to the needs of each learner, improving training efficiency and adaptability.
MR Module for Training Patient Care The MR training module – developed with Unity3D – for training self-aid and buddy care uses the HTC Vive XR Elite as MR device. For training patient care – as mentioned above – it is crucial to maintain the haptic feedback to train different movements and medical procedures. The focus within the training lies on cardiopulmonary resuscitation (CPR) and management of bleedings. To provide an immersive training and keep the haptic feedback, training manikins were included within the system. Since patient care should be trained under stressful conditions within a crisis scenario, the manikin and further equipment is projected into a virtual scene. To bring the manikin into the virtual world it has to be tracked. A camera-based sensor is used to track its 3D position and pose to provide accurate and robust tracking even when the manikin is not in the user’s field of view. Figure 4 shows the virtual scenario and the manikin with the sensors.
Motion tracking is achieved via small inertial sensors mounted on the manikin’s limbs and head. For management of bleedings a physical tourniquet has to be applied. As this device is not easily trackable, whenever the user reaches for the virtual tourniquet (which is lined up with its physical counterpart) a see-through window around their hands is activated to allow grabbing the tourniquet and applying it on the manikin. After application, the window is disabled by the trainer via a trainer interface. Trainees use the tourniquet to stop the bleeding of the virtual patient while receiving haptic feedback. The scenario is controlled using a trainer interface, where trainers can add stressors (e.g. barking dogs, gunshots, explosions) to the scenario,
A Scalable MR-Training System to Prepare Military and Civil Experts for Crisis Areas 247
customize the appearance of the patients and their virtual injury patterns. This allows maximal adaptability of the scenario as well as the recreation of high-pressure operational environments. The trainer interface is developed for Android tablets and communicates with the MR app via RabbitMQ. Apart from the MR app, a Unity3D server is provided to track the state of simulated elements (patients, stressors) and provide the current state to all connected devices. For exchanging the data between the Unity3D server and the MR app, the Photon SDK is used. This also allows for scaling the training to include multiple users within the same scenario. Figure 5 shows screenshots of the trainer interface.
Figure 4: Left: manikin with the sensors. Right: view through the glasses.Figure 5: Trainer interface: Left: configuration of the patient; Right: stressors.
RESULTS
The feasibility of the developed training modules was assessed in a series of scenario workshops and user tests. A total of 26 participants participated across all workshops, including military, civilian, and government staff. In the latest workshop that took part in the beginning of April 2025, 13 end-users from five end-user organisations (Austrian Ministry of Interior, Austrian Ministry for European and International Affairs, Austrian Ministry of Defence, Austrian Centre for Peace, Johanniter) participated. The participants included 4 women and 9 men with civilian, military, paramedic and government backgrounds. 3 of the participants were trainers (all male, between 45 and 55 years) and 3 were alumni of the ACP’s International
Civilian Peacebuilding Training (IPT) Programme (1 male, 2 females, all between 35 and 45 years). They contributed their expertise from working in different crisis areas.
The goal of the workshop was to assess the current state of the prototypes as described in the Technical Outcomes, with regards to their usability, user experience and suitability for the intended training purposes. The prototypes were individually tested by the end-users and afterwards discussed in small groups together with the technical developers.
After assessing the current state of the prototype, the users were requested to fill out a survey, consisting of an assessment of the technology using the TAM2 model, as well as open-ended questions on improvements, usability, data protection, and GDPR. Finally, the key findings based on the discussion and observations during the study, were reflected upon in a larger group discussion, to outline the next steps in development. The following sections will summarize the key findings of the most recent user test for each of the respective prototypes. Figure 6 shows the results of the TAM2 survey filled out by the participants for the individual modules.
Figure 6: TAM2 survey results of the second workshop (5-point Likert scale) for 3 modules.
Situational Awareness The VR Module for situational awareness was positively evaluated in terms of perceived usefulness (Mean = 3,69; SD = 0,83) and ease of use (Mean = 4,02; SD = 0,83). In the category ease of use, the learnability of the system was especially well-rated (Q: ‘Learning to operate this MR/VR hardware is easy’, with a mean of 4,31 and a SD of 0,75). The biggest challenge for the situational awareness system in the current version seems to be related to the realism of the scenario (Mean = 3,37; SD = 0,71). Participants discussed various changes to the assets and environment to improve the realism, and a desire for a higher granularity in objects that could be marked as potential risks. Still, the overall perception of the tool was very positive. Participants stated that it allowed for a far more realistic training of situational awareness than current tools, as indicated by receiving the highest rating among the tested tools in terms of behavioural intention to use (Mean = 4,21; SD = 0,77). This could indicate a high willingness to integrate such VR solutions into potential future training setups.
A Scalable MR-Training System to Prepare Military and Civil Experts for Crisis Areas 249
Map Reading The map reading module received the highest ratings in terms of perceived usefulness (Mean = 4,02; SD = 0,94) and perceived ease of use (Mean = 4,13; SD = 0,91). Participants noted that the module helped improve their spatial reasoning and map literacy. They appreciated the simple interface, but many suggested adding scenario-linked elements to provide more context and enhance the learning experience. Furthermore, they expressed an interest in having a more comprehensive list of icons and potential map elements in the legend provided, as well as support for multiple languages (e.g. both German and English terms).
In the concluding discussion of the workshop, multiple participants mentioned that the map reading module felt the most ‘complete’ from the various tools tested and could be used for basic training with some minor additions. They also expressed interest in a potential expansion of the tool to also include advanced types of interaction with maps, such as route planning.
Patient Care The patient care module received similarly high ratings in terms of usefulness (Mean = 3,73; SD = 1,14) and behavioural intention to use (Mean = 4,08; SD = 1,06) as the situational awareness module. Compared to the other modules that were tested, the results for patient care were generally more variable in both the surveys and the verbal feedback. Participants highlighted the module’s potential for integrating medical, military, and civil sector training in resource-constrained environments, as well as potential for expansions through additional scenario-specific environments and assets. They also commented on the effectiveness of the simulated stressors to further enhance the experience. While the score in terms of perceived ease of use were positive (Mean = 3,73; SD = 0,84) with regards to the interaction with the system itself, concerns were raised during the workshop regarding the comparatively complex technical setup.
Strategic Planning As there were only minor changes between the current prototype and the previously tested ones and due to time constraints, the strategic planning module was only assessed qualitatively through interviews and group discussions in the April workshop. Participants rated it highly in terms of usefulness and ease of use, highlighting its ability to facilitate structured decision-making in difficult scenarios. They valued the abilities to customise the map and add custom areas and elements to it and made various suggestions to further improve the available selection and the structure of the map itself (e.g. by adding multiple layers). They also requested the ability to specify changes over time, e.g. by adding a timeline feature that allows to mark changes related to certain events.
SUMMARY & OUTLOOK This work highlights the current state of the modular training system and the results of the second user workshop of the SkillDrill project. As outlined
in Section 4 the results showed that the system was generally well received with positive intention to perceived usefulness and ease of use, which may indicate a high willingness to integrate the developed solutions into future training setups. The map reading module was perceived as the most complete one, whereas the situational awareness module needs some improvements regarding realism. However, the overall perception of the two VR modules was very positive and a high added value was claimed. The patient care module received high ratings in the behavioural intention to use and end- users claimed that it has potential for integrating it in the medical, military and civil sector training. However, a simpler technical setup is crucial to reduce the complexity. For the strategic planning module, a timeline feature was suggested to mark and visualize certain events in time. In general, end- users value the module’s potential to facilitate structured decision-making in difficult scenarios. The feedback from the latest user workshop will be incorporated into a final prototype, which will be tested by various end-user organizations in September 2025.
ACKNOWLEDGMENT
The project SkillDrill is funded by the Austrian security research program KIRAS of the Federal Ministry of Finance.
REFERENCES
Broneder, E, Weismeier-Sammer, D, Psenner, M, Holzinger, A, Egger-Lampl, S, Karlseder, M, Puthenkalam, J & Ibrahim, Y. (2024), “SkillDrill – A Modular Training System for Preparing Civilian Experts for Peace Missions”, in Human Systems Engineering and Design (IHSED 2024): Future Trends and Applications. Bd. 158, 6th International Conference on Human Systems Engineering and Design, Split, Kroatien, 24/09/24. https://doi.org/10.54941/ahfe1005549 Mossel, A., Schoenauer, C., Froeschl, M., Peer, A., Goellner, J., & Kaufmann, H.
(2021). “Immersive training of first responder squad leaders in untethered virtual reality”. Virtual Reality, 25(3), 745–759. Ragazzoni, L., Ingrassia, P. L., Echeverri, L., Maccapani, F., Berryman, L., Burkle Jr., F. M., & Della Corte, F. (2015). “Virtual reality simulation training for Ebola deployment”. Disaster medicine and public health preparedness, 9(5), 543–546. Skryabina, E., Reedy, G., Amlôt, R., Jaye, P., & Riley, P. (2017), “What is the value of
health emergency preparedness exercises?” A scoping review study. International journal of disaster risk reduction, 21, 274–283. Trzci´nska, W. (2019). “Preparation of Polish police officers for peace missions.
Security and Defence Quarterly”, 23(1), 126–143. Vourvachis, I., Giordanis, A., Nguyen, Q., Gyllencreutz, L., & Schrom-Feiertag, H.
G. (2022). “MED1stMR: An integrated Training Using a Mixed Reality Approach Featuring Haptic Feedback for Enhanced Realism-EUO’s Requirements”. In 9th International Conference on Civil Protection & New Technologies-Safe, Thessaloniki 2022 (pp. 301–303).
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Interaction and Emerging Technologies (IHIET 2025)
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-73-1
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional