Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Human-Centric Approach for Developing XR Applications in the Space Domain

VTT Technical Research Centre of Finland Ltd, Tampere/Espoo, Finland ↗
Publication year
Digital Object Identifier10.54941/ahfe1006017

Abstract

This paper introduces the human-centric approach for developing the extended reality (XR) applications in the space domain. XR applications for supporting high-knowledge, high-value work have been developed in five projects in collaboration with the European Space Agency (ESA). These projects include (1) EdcAR, which focuses on augmented reality for assembly, integration, testing and validation (AIT/AIV), and orbit operations; (2) mobiPV4Hololens, which brings the international space station (ISS) procedure viewing to HoloLens; (3) AROGAN, which is based on augmented reality for ISS and ground applications; (4) VirWAIT, which creates a virtual workplace for AIT & product assurance (PA) training and operations support; and (5) DPIAR, which digitalizes procedures and introduces augmented reality. Development began in 2016, and the system is currently being implemented in the ESA Test Centre’s. The consecutive projects formed the four phases of development process: The first phase focused on demonstrating proof-of-concept, the second phase on establishing connections to space systems, and subsequent phases on developing features based on user needs and integrating connections to the ESA Test Centre ’s sensor systems. This phased approach ensured that the system evolved in a structured manner, addressing both technical and user-centric requirements at each stage. All these projects employed a human-centric approach in their development, incorporating the most suitable parts of the standard ISO 9241-210 (2019) on ergonomics of human-system interaction. This standard emphasizes the importance of designing systems that are both effective and satisfying for users. Based on four development cycles, the XR environment combined with human-centric evaluation and design has proven to be a powerful method from early-stage proof-of-concept to actual implementation. Throughout the process, potential users have been able to provide valuable feedback, enhancing the novel tool's ability to support high-knowledge, high-value work. This iterative feedback loop has been crucial in refining the applications to better meet the needs of the users, ensuring that the final product is both functional and user-friendly.

INTRODUCTION

Applications in maintenance and assembly are potential solutions for XR guidance and training, also known as AR instructions (Re & Bordegoni, 2014), AR-based job aid (Anastassova et al., 2005), AR-assisted maintenance system, and AR-based assembly guidance (Ong et al., 2008). AR guidance provides instructions in textual and/or visual format, augmented on target objects. Studies noted benefits of AR guidance in assembly and maintenance: tasks were easier, performed more effectively with fewer mistakes, and skill

© 2025. Published by AHFE Open Access. All rights reserved. 95

transfer was enhanced (Ong et al., 2008). MR and/or AR has been tested in space domain projects for training and manual work support (Tedone et al., 2016) and supporting robotics operation in the ISS (Maida et al., 2007). MR and/or AR usability has reached an acceptable level in space-related training and maintenance support (Helin et al., 2018). User studies in AR contexts examined human perception, task performance, collaboration, and system usability (Dünser et al., 2011). Users’ experiences were measured using preference, ease of use, perceived performance, and intuitiveness. Methods included questionnaires and/or performance measures (Bai & Blackwell, 2012), with qualitative measures from observations, video analysis, and interviews (Dünser & Billinghurst, 2011). Many studies highlighted AR guidance’s potential in work support and suggested improvements in user experience and design (e.g., Aaltonen et al., 2016; Helin et al., 2015; Kuula et al., 2012). User-centric development should be implemented in technology and content creation to develop effective AR solutions.

HUMAN-CENTRIC DESIGN METHOD The design process has been iterative and cyclical, adhering closely to human- centred design best practices outlined in the ISO standard. The basic design cycle includes: understanding the context, specifying user requirements, creating design solutions, and evaluating the design (ISO 9241-210). The development team conducted four main design cycles, known as phases. The phases were executed within five consecutive research and development projects. Each phase encompassed sub-design cycles throughout the development process (see Figure 1). Throughout the development process, users were consulted, and their feedback significantly influenced the final design of the prototypes. As Brooks (1995) suggests in “The Mythical Man-Month: Essays on Software Engineering,” it is advisable to plan a quick- and-dirty first iteration of the project rather than investing resources to get it right on the first try, as design flaws will inevitably arise from the first version of a product.

Figure 1: Four main design cycles, each encompassing sub-design cycles, were conducted throughout the development process.
Original PDF Figure 1:

Human-Centric Approach for Developing XR Applications in the Space Domain 97

Data Collection Methods During all four phases, communication with end-users and their representatives was active. User needs and feedback were collected and received in various development meetings and workshops. Furthermore, user tests were executed in each phase. The methods for collecting data from the test users included:

  • Observation: The development team observed users’ interaction with the system and took notes. This was the most common method for data collection.
  • Semi-structured qualitative interview: Thematic interviews focused on usability, usefulness, applicability, and improvement suggestions.
  • Think-aloud method: The users was asked to speak aloud and describe their experience during the test execution. Method was utilised especially in remote user test setting during the COVID pandemic, when observation on-site was not possible.
  • System Usability Scale (SUS) questionnaire: The questionnaire was used to provide numeric evaluation of the system usability. It enabled the straightforward way to follow the system improvement between development phases. The SUS scores calculated from individual questionnaires represent the system’s usability (Brooke, 1996). According to validation studies (Bangor et al., 2009; Brooke, 2013), a SUS score of 68–70 indicates acceptable system usability. The suggested acceptability ranges are: 0–50 (not acceptable), 50–70 (marginal), and 70+ (acceptable).

The goal of user testing was to gain general understanding on feasibility of the system for the specific work tasks and to provide improvement suggestions for further development. The users were commonly asked to describe their general experience with the system, to evaluate the suitability of system features for their work tasks, and to provide ideas for improving the features and user interface. Specific test tasks based on users’ real work were designed for user tests to simulate the authentic working situation. The user tests resulted in lists of improvement suggestions for the developers to work with.

PHASE1: EDCAR Description of the Developed XR System

The final version of the EdcAR system supports Microsoft HoloLens v1 and adheres to the ARLEM (IEEE Std 1589-2020) standard (see Figure 2 left). It is configured with Activity and Workplace JSON files. The Workplace JSON, parsed by the Workplace manager, contains workplace information like points of interest and sensors. The Activity JSON, parsed by the Activity manager, details action steps and content for each step, with data transferred to the AR layer via local storage. The final version can annotate:

1) UI in 3D space 2) Warning symbols 3) Location based warnings

4) Symbols 5) 3D models and animations 6) Video and audio annotations.

Evaluation and Main Outcomes The EdcAR system was evaluated at the ISS-Columbus training mock-up in ESA’s European Astronaut Centre (EAC) in Cologne, Germany. The test group, consisting of 14 subjects, included an astronaut, EAC trainers, other EAC personnel, and students. All participants tested the Microsoft HoloLens- based system. The Figure 2 (right) shows the testing scenario. Based on the feedback of the evaluation, the XR system is usable for basic daily use as its usability has reached a reasonable level (Helin et al., 2017), but it should be connect to the space systems.

Figure 2: Left: ARLEM based user interface located in 3D space including general warning symbols and 3D models with animation. Right EAC personnel are testing the updated version.
Original PDF Figure 2:

PHASE2: MOBIPV4HOLOLENS Description of the Developed XR System

During second phase, the XR system was connected to the space system called mobiPV, which displays Operations Data File Standards (ODF) content (NASA, 2010). The operational mobiPV4Hololens MR-system components are shown in Figure 3. The system includes: (1) Microsoft HoloLens with the mobiPV4Hololens app as the main user interface, (2) a mobile phone with a web player for mobiPV for non-supported content such as reference documentation, (3) mobiPV – server (flight) for all ODF content and collaboration, and (4) mobiPV – server (flight) for all ODF content and collaboration for the ISS.

The operational mobiPV4Hololens MR-system allows the astronaut to utilize the main mobiPV features, which have been implemented in the system. The main features implemented include: (1) Links to mobiPV server with standard mobiPV WebSocket communications. (2) Working hands free with voice commands e.g. “Betsy pin image”. “Betsy”added to all voice commands to minimize side-talk issues. There were in total 37 separate voice commands.

Human-Centric Approach for Developing XR Applications in the Space Domain 99

(3) Images and video notes clips captured with the built-in HoloLens camera. (4) Text-to-speech functions which allows the following of an ODF procedure without reading. Selectable by the astronaut as an optional feature. (5) Pin information within 3D space e.g. text, note, video or image next to working area. (6) Collaboration mode between the ISS and ground support.

Figure 3: mobiPV4Hololens system set-up.
Original PDF Figure 3:

Evaluation and Main Outcomes The user evaluation of the mobiPV4HoloLens system was conducted at the European Space Research and Technology Centre (ESTEC) in Noordwijk, the Netherlands. The test group consisted of five subjects: four males and one female, all employees of ESA with no previous experience in ODF or mobiPV. Most participants had prior experience with mixed reality and/or Microsoft HoloLens devices. The mobiPV4HoloLens system functioned properly as a proof of concept, with no significant issues related to voice commands. However, there is a learning curve associated with using the system, particularly concerning voice commands and information location within the 3D space. The primary usability concern pertains to the fit and comfort of the HoloLens device, which does not accommodate all head shapes well, along with its limited field of view. Additionally, some users were not familiar with the ODF/IPV procedure format. While most users found the system beneficial for supporting procedure execution, they indicated that it would not be feasible to wear the device for an entire working day. Based on the specifications of the new MS HoloLens 2, most of these usability issues should be addressed. Furthermore, there is a need for an authoring environment for augmented reality content (Helin et al., 2019).

PHASE3: AROGAN AND VIRWAIT The AROGAN and VirWAIT project were executed parallel. AROGAN was focusing more XR features development and VirWAIT were testing it features in actual use and test cases like the JUpiter ICy moons Explorer’s (Juice)

Network Data Interface Unit (NDIU) in preparation to the Thermal Vacuum test campaign.

Description of the Developed XR System The operational VirWAIT MR-systems workflow has been described in Figure 4. The workflow includes three main steps (1) Content authoring, (2) Mixed reality based procedure execution and (3) Automatic reporting and as-build 3D model generation.

Figure 4: The AROGAN/VirWAIT system’s workflow.
Original PDF Figure 4:

Content Authoring can be done in two modes: Desktop Mode, which is off-site, and Mixed Reality Mode, which is on-site. Mixed reality-based procedure execution involves voice and gesture-based control, including voice commands such as “Betsy Next.” The graphical user interface is displayed in 3D space, with augmented reality annotations and 3D models that can be animated alongside 2D symbols. This system guides users to information in 3D space with pictures and videos, offering step-by-step procedure guidance from the mobiPV server and marking procedure steps as done. Users can create notes in various formats, including video, audio, text, and image. It also features text-to-speech capabilities and locates additional information within the 3D space. The system allows users to select installed sensors and cables from a list and add the final location of the installed sensors. And finally automatic reporting and as-build 3D model generation, which could be used final reporting of integration (Helin et al., 2021).

Evaluation and Main Outcomes The system were tested in two use cases at the ESA Test Centre premises (see Figure 5); (1) MR supported installation of thermocouples on an Heat Plate used as GSE for the Solar wind Magnetosphere Ionosphere Link Explorer – SMILE, and (2) MR supported AIT, phase 2 sensor installation on TEDY (TEst DummY) for a vibration test campaign on the Hydra facility

The AROGAN/VirWAIT MR-system shows potential to enhance work tasks performed during tests. Currently, sensors are placed using printed CAD models and guides; the MR-system can greatly accelerate this process. It is expected to boost productivity and efficiency for complex tasks, while

Human-Centric Approach for Developing XR Applications in the Space Domain 101

simpler cases can continue with the current method. Users described the MR-system as positively novel, akin to “something out of a sci-fi movie,” and their overall response was favorable. SUS scores were 77, indicating acceptable usability. Final user reviews showed improved usability compared to preliminary tests, particularly as actual operators used the system for their real tasks. Users were also highlighting that it is necessary to display sensor values in real-time for users during operations (Helin et al., 2021).

Figure 5: The AROGAN/VirWAIT system’s test cases: left SMILE and right: TEDY.
Original PDF Figure 5:

PHASE4: DPIAR Description of the Developed XR System

Building upon the success of AROGAN/VirWAIT MR, the final phase, the project called DPIAR-V1, represents a continuation of the ESA’s MR use cases and development. The project outcome was a complete end to end solution for the authoring (off- and on-site), deployment and usage of MR to support execution of manual procedures and operations within the Test Centre of the European Space Agency. The operational DPIAR-v1 MR-systems and authoring components can be found in Figure 7 (Helin et al., 2024). The system includes the Microsoft HoloLens 2 with DPIAR-v1 MR-player app, mobiPV server for all ODF content, MR annotation and 3D models with animation, mobiPV’s web interface that allows user interaction with systems, STAMP sensor data server, authoring environment including both on-site and off-site authoring, remote observation via web access, and reporting.

Evaluation and Main Outcomes To evaluate the main project objectives, two most relevant use cases were selected. Them were use cases at the ESA Test Centre premises.

(1) Large Space Simulator – LSS Basement procedure (see Figure 7). (2) Vacuum Test Chamber - VTC1.5 Operating Procedure and Pre- operation.

Both use cases were created using off-site and on-site authoring. Semi- automatic off-site templates were used and refined, while on-site authoring

added Point of Interests without 3D models. The process involved partners and ESA personnel, and the MR system for remote observation was tested, especially in the LSS Basement procedure (Helin et al., 2024).

Figure 6: DPIAR-v1 system set-up.
Original PDF Figure 6:
Figure 7: Execution of LSS basement procedure (left), and remote observation (right).
Original PDF Figure 7:

CONCLUSION

The final DPIAR-v1 system demonstrates a successful human-centric approach for developing XR applications in the space domain, particularly in implementing novel technologies within complex and high-knowledge systems. Through four development cycles, the XR environment combined with human-centric evaluation and design has been an effective method from early-stage proof-of-concept to actual implementation. During the process, potential users have provided feedback, improving the tool’s ability to support high-knowledge, high-value work. This iterative feedback loop has been important in refining the applications to better meet the needs

Human-Centric Approach for Developing XR Applications in the Space Domain 103

of the users, ensuring that the final product is functional and user-friendly. Various methods for data collection in user tests have been utilised to ensure that user feedback is accurate and extensive. This comprehensive end-to- end solution effectively addresses the challenges of authoring, deploying, and utilizing MR within the European Space Agency’s Test Centre. The MR procedure execution application, deployable on HoloLens2 devices, offers a user-friendly interface for accessing and executing procedures. It enhances procedural efficiency and provides an immersive user experience through features like annotation display, Point of Interest highlighting, multimedia support, and real-time integration of STAMP sensor information. Additionally, the system supports detailed logging and multimedia streaming during procedure execution, facilitating live remote viewing and session reviews, which enhance collaboration, knowledge sharing, and continuous improvement. One of the significant advantages of the DPIAR-v1 system is its optimized operational workflow, which reduces MR procedure authoring time through semi-automated off-site procedure templates and intuitive on- site authoring methods via HoloLens 2. Overall, the DPIAR-v1 system showcases the potential of MR technology to enhance manual procedures and operations within the European Space Agency’s Test Centre.

ACKNOWLEDGMENT

These studies have been funded by the European Space Agency under contracts: 4000113373/15/NL/MH “Augmented Reality for AIT, AIV and Operations”, 4000125238/18/NL/AF/as “Prototype a Media Helmet for mobiPV Implemented Using MS HoloLens”, 4000127710/19/NL/GLC “Procedure viewer and authoring tool for ground AIV/AIT applications”, 4000129549/19/NL/BJ “Virtual Workplace for AIT & PA Training and Operations Support”, and 4000140015/22/NL/MG “Digitalisation of Procedure and Introduction of Augmented Reality (Step 1)”.

We would like to express my sincere gratitude to Oliveira David Martinez, Mikael Wolf, and Gianluca Casarosa from ESA for their invaluable support and excellent cooperation throughout these projects.

REFERENCES

Aaltonen, I., Kuula, T., Helin, K. & Karjalainen, J. (2016). Maintenance Past or Through the Tablet? Examining Tablet Use with AR Guidance System. Proceedings of European Association for Virtual Reality and Augmented Reality Conference, EuroVR-2016, Athens, Greece, 22–24 November, 2016. Anastassova M., Burkhardt J. M., Mégard C., Ehanno P.: Results from a user-centred

critical incidents study for guiding future implementation of augmented reality in automotive maintenance. International Journal of Industrial Ergonomics 35, 1 (2005), 67–77. Bai Z., Blackwell A. F. (2012). Analytic review of usability evaluation in ISMAR.

Interacting with Computers 24, 6 (2012), 450–460. Bangor, A., Kortum, P. & Miller, J. (2009). Determining What Individual SUS Scores mean: Adding an Adjective Rating Scale. Journal of Usability Studies. Vol. 4, Issue 3, May 2009. pp. 114–123.

Brooke, J. (1996). “SUS: a ‘quick and dirty’ usability scale,” in Usability evaluation in industry. P. W. Jordan, B. Thomas, B. A. Weerdmeester and I. L. McLelland (Eds.) Taylor and Francis Ltd. London, UK. Brooke, J. (2013). SUS: A Retrospective. Journal of Usability Studies. Vol. 8, Issue 2,

February 2013. pp. 29–40. Brooks, Frederick P., Jr. (1995). The mythical man-month (anniversary ed.). Addison- Wesley Longman Publishing Co., Inc., USA. Dünser A., Billinghurst M. (2011). Evaluating Augmented Reality Systems. In:

Furht, B. (Ed.): Handbook of Augmented Reality: Springer Science+Business Media, 2011, 289–307. Helin, K., Karjalainen, J., Bolierakis, S., Frangakis, N., Kemppi, P., Tedone, D., & Oliveira, D. M. (2017). Augmented Reality System for Space Station Maintenance Support. Paper presented at European Association for Virtual Reality and Augmented Reality Conference, EuroVR-2017, Laval, France. Helin, K., Karjalainen, J., Kiernan, P., & Casarosa, G. (2024). The Mixed Reality

System for Operation, AIT/AIV Support and Reporting in the Space Domain. In EuroXR 2024: Proceedings of the 21st EuroXR International Conference (Vol. 432, pp. 31–34). VTT Technical Research Centre of Finland. https://doi.org/ 10.32040/2242-122X.2024. T432 Helin, K., Karjalainen, J., Kiernan, P., Oliveira, D. M., & Wolff, M. (2019). Mixed Reality User Interface for Astronauts Procedure Viewer. In 2019 International Conference on Cyberworlds (CW) (pp. 17–20). Article 8919243 IEEE Institute of Electrical and Electronic Engineers. https://doi.org/10.1109/CW.2019.00011 Helin, K., Karjalainen, J., Kuula, T., Kiernan, P., Casarosa, G., & Oliveira, D. M.

(2021). The Remote User Test of Mixed Reality System for AIT/AIV in the Space Domain. In K. Helin, J. Perret, & S. Arlati (Eds.), EuroXR 2021: Proceedings of the Virtual EuroXR Conference (pp. 11–13). VTT Technical Research Centre of Finland. https://publications.vtt.fi/pdf/technology/2021/T395.pdf Helin, K., Kuula, T., Karjalainen, J., Aaltonen, I., Wild, F., Viitaniemi, J. & Väätänen, A. (2015). Usability of the ARgh! Augmented Reality System for on- the-job Learning. Proceedings of the 19th Triennial Congress of the International Ergonomics Association (IEA), Melbourne, Australia, 9–14 August, 2015. Helin, K., Kuula, T., Vizzi, C., Karjalainen, J., & Vovk, A. (2018). “User Experience

of Augmented Reality System for Astronaut’s Manual Work Support”. Frontiers in Robotics and AI, 5(SEP), [106]. https://doi.org/10.3389/frobt.2018.00106 IEEE Standard for Augmented Reality Learning Experience Model,” in IEEE Std 1589-2020, vol., no., pp. 1–48, 17 April 2020, doi: 10.1109/ IEEESTD.2020.9069498. ISO 9241–210. (2019). Ergonomics of human-system interaction. Part 210: Human-

centred design for interactive systems. European Committee for Standardization. Kuula, T., Piira, K., Seisto, A., Hakkarainen, M. & Woodward, C. (2012).

User Requirements for Mobile AR and BIM Utilization in Building Life Cycle Management. In proceedings of 12th International Conference on Construction Applications of Virtual Reality (CONVR), Taipei, Taiwan, 1–2 November, 2012. pp. 203–211. Maida, J. C., Bowen, C. K., & Pace, J. (2007). Improving Robotic Operator Performance Using Augmented Reality. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 51(27), 1635–1639. https://doi.org/ 10.1177/154193120705102703

Human-Centric Approach for Developing XR Applications in the Space Domain 105

National Aeronautics and Space Administration (2010) Operations Data File Standards, Report SSP50253, Revision R, Houston, Texas, NASA Johnson Space Centre. Ong S. K., Yuan M. L., Nee A. Y. C. (2008). Augmented reality applications in

manufacturing: a survey. International Journal of Production Research 46, 10 (2008), 2707–2742. Re G. M., Bordegoni M. (2014). An augmented reality framework for supporting and monitoring operators during maintenance tasks. In: Shumaker, R.., Lackey, S. (Eds.): VAMR 2014, Part II, Lecture Notes in Computer Science 8526: Springer International Publishing Switzerland, 2014, pp. 443–454. Tedone, D., Marello, M., Musso, G., Frangakis, N., Martinez Oliveira, D.,

Agostinho, S., & Helin, K. (2016). “Augmented Reality for the Support of Space Exploration Missions and On-Ground/On-Orbit Operations”. In Advances in Human Factors and System Interactions (pp. 141–151). Springer. https://doi.org/ 10.1007/978-3-319-41956-5_14

Cover of the proceedings volume Accessibility, Assistive Technology and Digital Environments

Explore the complete digital volume

Accessibility, Assistive Technology and Digital Environments

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-964867-38-0

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →