Kathrin Konkol, Hannah Reusing (2023). Validation of virtual reality tools for unique aircraft interiors. In: Human Systems Engineering and Design (IHSED 2023): Future Trends and Applications. AHFE Open Access, vol. 112, pp. 446–456. https://doi.org/10.54941/ahfe1004157
A Virtual Reality (VR) application has been developed which supports engineers during product creation processes of unique aerospace interior. Ten trained engineers participated in a user study, which explored the differences in efficiency and user experience of two use cases within VR and conventional working methods. The study represented real day-to-day challenges and included two tasks: detecting the visibility of components for design reviews and cable routing for assembly processes. Five experts per task had to complete their assignment in both VR and their conventional working tools. In the visibility task participants had to decide whether the component of interest is visible or not in three different scenarios. For the cable task the experts were asked to route a specific cable for components in order to plan the amount of material that was needed in the final assembly. They had to estimate the cable length for four different cable routings and the outcome was compared to the optimal cable length for each given task. In both tasks the time until a decision has been measured. The results show that engineering processes can be supported by VR applications, which can help saving time in visibility testing and cable routing, as well as potentially saving resources by improving the accuracy of calculation for an ideal cable length. There are further potential benefits for users, as VR strains the workload less than conventional working methods. Furthermore, the study has shown that participants who were less experienced with VR technologies did significantly better with the VR application compared to conventional working methods during the visibility check.
The recent advances in product development processes have brought a series of new challenges for companies and work forces alike, such as high demand on product variety or expert knowledge needed for manufacturing. Mistakes or down times that occur through the absence of workers with a high level of experience are expensive. So is the time it takes to train an expert and build the knowledge base they have been collecting over years’ worth of time.
There has been a steady development of technology, which enables workers to view and interact with virtual environments and improves engineering pro- cesses, such as Augmented Reality (AR), Virtual Reality (VR) and Extended Reality (XR). With increasing computation power at a lower price, high performance devices become more affordable. Nowadays these technologies
Validation of Virtual Reality Tools for Unique Aircraft Interiors 447
have been established in a wide field of scenarios, e.g. product design, manu- facturing (Berg, 2017) and assembly methods (Seth, 2011). Existing solutions are being compared in order to benchmark the impact of such tools via Key Performance Indicators (Jetter, 2018). User-centered design approaches can increase product efficiency and effectiveness (Abras, 2004) and is imple- mented through means of close collaboration between developer and user, low fidelity prototypes and users testing (Silva, 2011). (Alves, 2021) com- pared multiple devices for assembly use cases, and found that the efficiency of AR Devices depends on the specific use case. The authors mention that in order to assess the validity of a system for a specific engineering use case, the tasks given should be complex and realistic.
In order to assess the viability of a VR demonstrator, we conducted a user study together with a manufacturer of VIP plane interior. We used a realistic task for the study and compared the VR system with the conventional way the individual users would solve the task.
DEVELOPING THE VIRTUAL REALITY ENVIRONMENT The VR system that was used for this study has been developed with agile development methods. This means that over the years it has consecutively been tested and evaluated by engineers who used it with their real-life use cases (Konkol 2021, Konkol 2022). Multiple use cases, such as design review and virtual cabin assembly, have been developed and evaluated. For this user study we took the use cases cable routing (Konkol, 2020) and visibility check as examples, in order to compare efficiency and perceived stress level of each working method. The visibility check tool was designed in order to support scenarios in which visibility of components needs to be monitored such as visibility of cables or warning signs. It offers possibilities to adjust the height of a user and therefore the individual perspective according to standardized viewing perspectives for 3 chosen body sizes (see Figure 1). By changing the viewing height and therefore the camera position in virtual space, it cre- ates the illusion of a different body height to what the user would normally have.
Figure 1: Screenshot of visibility check tool with instructions on the left.
448 Konkol and Reusing
Planning cable routes in aircraft interior design is a crucial and complex task, which requires planning, expertise and documentation. The purchased cable length for construction is based on the estimation of the experienced worker or previously measured length. Cables that are too long or too short can lead to further costs. With a cable measurement that is close to the ideal cable length, costs and resource wastage can be reduced. The cable routing tool was designed for the user to be able to route cables in virtual space. The user can place cable points in virtual space and on geometries. The individual points are automatically connected by a spline by using a specified maximum bending radius for each cable. The tool consists of direct manipulation meth- ods on the left controller that enables the user to place, edit and delete such cable points (see Figure 2).
Figure 2: Screenshot of cable routing tool.
At the right controller there’s a graphical user interface attached, which visualizes information regarding the create cable, such as name, length, max- imal bend radius and creation date. The user also has the possibility to delete the cable. Thereby the user is able to monitor individual cables and also export the work as a geometry file, which creates the possibility to import the cable into other systems.
USER STUDY DESIGN In this chapter we describe the user study as such as the tasks given, the demo- graphics of the participants and the chosen questionnaires. The tasks were abstracted for this paper to comply with their release rules, as we used engi- neering and customer data to conduct the user study. A total of ten experts participated in the study. Five of each for the two tasks, according to exper- tise and work activities and field of work, who have the necessary knowledge to fulfil the task. The participants are between 20 and >50 years old and have been working in their profession for between a quarter of a year and 21 years (M = 8.11 and SD = 6.16). Figure 3 shows the division of VR knowledge and expertise in the two tasks. This shows that six out of ten participants are familiar with the VR application. They rated their expertise at an average of
Validation of Virtual Reality Tools for Unique Aircraft Interiors 449
0.33 points. Compared to that all ten participants rated their expertise in the conventional method at an average of 0.95.
Figure 3: Participants subjective expertise in the VR application and the conventional method (for the VR application only participant with experience in the application are included). Expertise is divided into the two tasks.
Task 1 - Components Visibility Check In the component visibility task, the participants have been presented a vir- tual scene and asked whether or not a chosen object was visible to them (see Figure 4). The hidden construction elements used in the interior design, such as the observation lights, should not be visible to future passengers. This requirement must be checked during the planning stage. This vali- dation of non-visibility is implemented here as an experimental task. For this purpose, three typical components were selected and a fixed position from which the participant had to decide, whether the component is visible or not.
Figure 4: Simplified overview of visibility check task.
In the visual inspection use case, a distinction was made between three different visual inspection points. These test points were varied so that they were visible or invisible for small or tall body sizes.
P1: Sign that’s mounted to the cabin wall and be clearly visible
not visible for both body sizes
P2: Attachments of LED stripe that should be hidden behind a panel
visible when small, non-visible when tall
P3: Holes and dirt-corners in a cabin that should not be visible
visible for both body sizes
450 Konkol and Reusing
Two extreme body heights were chosen: small (1.65 m) and tall (1.95 m). Each of these three visual inspection points was processed both with a conventional working method SolidWorks and with the provided VR tool. The two independent factors (Inspection Point with different visibilities and Method) are inner-subject factors, which means that each of the five experts processed all variations. The dependent variables, i.e. the measurement vari- ables, were the correctness of the statement made and the time needed to make it.
Task 2 - Cable Routing To check if the VR application is helpful in the process of planning the cable routing, four different cable routing scenarios were selected (see Figure 5). It was measured how long it takes to finish all four routings in the two methods, VR and conventional approach (within-subject variable). Since the scenarios were taken from a real customer use case, the authors could estimate the ideal cable length by comparing it with the length that was chosen for the final product. The participants chose their preferred tool as a conventional working method, which included Solidworks, Sketches and Paper. As in the visibility task, the dependent variables are the measured time until a decision was made and the measured cable lengths (objective variables).
Figure 5: Simplified overview of cable routing task.
Two standardized questionnaires were completed after each task to assess the subjective state of mind during task performance. Firstly, the NASA- TLX (Hart & Staveland, 1988) was used to record the perceived stress and assessment of how well the task was performed with ratings from 0 (no stress/performance of the task) to 100 (high stress/performance of the task).
Validation of Virtual Reality Tools for Unique Aircraft Interiors 451
On the other hand, an extract from DIN 9242–110 and the corresponding questionnaire on task adequacy (Prümper, 1992) was used. The five questions were rated from -3 to 3 on the Likert scale.
RESULTS
All handwritten data was digitalized and analysed with R (version 4.2.2). First, the results of the component visibility tasks are being presented, fol- lowed by the cable routing task. As shown in Table 1 the needed time for a decision varied significantly between inspection points as well as between the two methods with less time needed in VR (M = 2 min, SD = 1.15) than in the classical approach (M = 4.59 min, SD = 2.68). There was also a significant interaction between method and inspection points (see Figure 6).
Table 1. Results of two-way repeated measures ANOVA using time as the criterion.
Table available in the original PDF paper.
Effect df F p Inspection points (P1, P2, P3) 2 15.07 .005 Method (classical, VR) 1 15.76 .029 Inspection points x method 2 13.24 .003
Figure 6: Time needed for a decision (in min) about the visibility of the component in both VR and the classical method, separated by inspection points (P1-P3).
To determine whether the decision was correct or not, another two-way repeated-measures ANOVA was performed, with correctness serving as the outcome variable. There was a main effect of inspection points but no main effect of method and no significant interaction (see Table 2 and Figure 7, left side). A Wilcoxon signed rank test was performed to analyse the efficiency
452 Konkol and Reusing
of the decision, measured in time in relation to correctness. There was a sig- nificant effect, V = 38, p = .037 (see Figure 7, right side), with participants using VR needing less time for a correct decision (M = 1.77 min, SD = 1.11) than when using the classical method (M = 3.6 min, SD = 1.79).
Table 2. Results of two-way repeated measures ANOVA using correctness of decision as the criterion.
Table available in the original PDF paper.
Effect df F p Inspection points (P1, P2, P3) 2 31.71 <.000 Method (classical, VR) 1 2.25 .104 Inspection points x method 2 545 .30
Figure 7: Left: graphical representation of the frequency of correct answers in every inspection point for each height in the visibility task in both VR and the classical method. Right: graphical representation of the time needed until a correct decision about the visibility of the component was made in both VR and the classical method. The horizontal line represents the median.
In order to compare the methods in terms of the time needed to estimate the cable lengths, an exact Wilcoxon test (signed rank test) was carried out, which compared the method (VR, classical approach) as independent variable and time in minutes as outcome variable. The method did not show a significant effect (see Figure 8), with V = 14, p = .063, the effect size is r = 0.784, but there was the tendency that participants needed less time when using the VR application (M = 7.31 min, SD = 1.54) compared to the classical approach (M = 9.59 min, SD = 1.95).
The cable measurements have been analysed via Wilcoxon signed rank exact test including method as independent and deviation from the ideal length as dependent variables. The absolute value was taken instead of neg- ative deviation. The difference in deviation is significant, V = 15, p = 0.031 (see Figure 9), with measurements taken in VR being closer to the ideal length (M = 97.01 mm, SD = 41.65) than in the conventional method (M = 235.36 mm, SD = 202.6).
Validation of Virtual Reality Tools for Unique Aircraft Interiors 453
Figure 8: Time needed until a decision about a cable length was made in both VR and the classical method. The horizontal line represents the median.Figure 9: Deviation of the measured to the ideal cable length in both VR and the clas- sical method, averaged over the four individual tasks. The horizontal line represents the median.
All subscales of the NASA-TLX questionnaire, except one, measure neg- ative impacts on perceived stress. The subscale performance captures how well participants think they performed in a task and was therefore sepa- rately analysed from the other subscales. Without this subscale, only the negative impact of each method on the perceived workload is compared. A paired t-test was performed. There was a non-significant tendency that VR was rated less stressful and frustrating, t(9) = 1.648, p = .067 (See Figure 10). Perceived performance in VR was significantly better than in the
454 Konkol and Reusing
classical method, t(9) = −3.329, p = .004. There was no significant differ- ence regarding method for task adequacy (See Figure 11), t(9) = −1.266, p = .119.
Figure 10: The NASA-TXL-scores, perceived stress in the given task, separated by method. The horizontal line represents the median.Figure 11: ISO-scores, task adequacy, separated by method. The horizontal line represents the median.
DISCUSSION AND CONCLUSION This study aimed to validate the developed VR application and the results show that the participants were able to solve the tasks better in VR than in the conventional method. Participants were able to decide more quickly if the component is visible or not when they used the VR application compared to their conventional method. One advantage could be that it was easier for par- ticipants to estimate their sitting height by bringing their body into a sitting position on a chair instead of calibrating a sitting height for a virtual cam- era. Referencing the results for the correctness, there is a tendency that the answers were more often correct and closer to an ideal solution. Furthermore, the results show, that using the VR application made the user feel more secure about their decisions and the perceived stress level was lower compared to working with their conventional method. The authors want to point out that the participants of this study do not use VR in their day to day work as of
Validation of Virtual Reality Tools for Unique Aircraft Interiors 455
the time of this user study, and yet they can produce adequate work results in similar time, quality and with similar stress perceived. The authors assume that this VR environment can adequately support engineers with conducting such tasks, independent on their individual experience with the respective expert tools. In order to gain more insight on the comparison, the study can be repeated with more experts on the field and different use cases. It needs to be considered that there is not one standardized procedure for the chosen use cases in the working environment of the experts, which can influence the time needed for the conventional working methods. These results show that engineering processes can be supported by VR applications, which can help saving time in visibility testing and cable routing, as well as saving money by calculating a more ideal cable length. There is also a benefit for workers because VR does not strain the workload as much. Another important fact to mention is that most of the participants indicated that they were profi- cient in the conventional approach and were less likely to have either used the VR application or were more likely to have not mastered it yet. But still they did significantly better with the VR application. Furthermore, the study has shown that participants who were less experienced with VR technologies did significantly better with the VR application compared to conventional working methods.
ACKNOWLEDGEMENT
This user study has been developed and conducted in cooperation with Lufthansa Technik AG as part of the research project Virtual Modifiction of Aircraft Cabins (VMOD), which was funded by the German Federal Ministryfor Economic Affairs and Climate Action (Bundesministerium für Wirtschaft und Klimaschutz) and German Aerospace Center (Deutsches Zentrum für Luft- und Raumfahrt) as part of the program LuFo.
REFERENCES
Abras, C.; Maloney-Krichmar, D. & Preece, J. (2004), ‘User-centered design’, Bainbridge, W. Encyclopedia of Human-Computer Interaction. Thousand Oaks: Sage Publications 37 (4), 445–456. Alves, J., Marques, B., Ferreira, C., Dias, P., Santos, B. (2021), Comparing Aug-
mented reality visualization methods for assembly procedures, Virtual Reality, Elsevier, https://doi.org/10.1007/s10055-021-00557-8 Berg, L. P., Vance, J. M. (2017) Industry use of virtual reality in product design and manufacturing: a survey. Virtual Reality 21, 1–17. https://doi.org/10.1007/ s10055-016-0293-9 Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (Task Load
Index): Results of Empirical and Theoretical Research. In P. A. Hancock, & N. Meshkati (Eds.), Human Mental Workload. Advances in Psychology (Vol. 52, pp. 139–183). North Holland. https://doi.org/10.1016/S0166-4115(08)62386-9 Konkol, K., Geiger, A., & Stark, R. (2020, April). Verlegung von Kabeln und Rohren in Virtual Reality im Engineering. WiGeP, (1/2020), pp. 17–18. Konkol, K., Geiger, A., (2021) Haptikgerät für zweihändige Montagesimulation in
VR, WiGep, (2/2021), p. 27.
456 Konkol and Reusing
Konkol, K., Geiger, A., Riedemann S., Janzen Mr. (2022), Virtual Reality als Werkzeug für die Anfertigung von Flugzeugkabinen bei Lufthansa Technik, in VR Best Practice Navigator, VR Business Club, pp. 121–132, https://vrbusiness.club/ best-practice-navigator Prümper, J., & Anft, M. (1992). Entwicklung eines Software-Beurteilungsverfahrens
auf Grundlage des Entwurfs zu ISO 9241/10. Konzept und erste Ergebnisse. Manuskript. Data Train, Berlin. Jetter, J., Eimecke, J., Rese, A. (2018), Augmented reality tools for industrial applications: What are potential key performance indicators and who benefits? Computers in Human Behavior, pp. 18–33, Elsevier, https://doi.org/10.1016/j.ch b.2018.04.054 Seth, A., Vance, J., Oliver, J., (2011) Virtual reality for assembly meth- ods prototyping: a review, Virtual Reality, Springer-Verlag, London Limited, DOI: 10.1007/s10055-009-0153-y. Silva, T., Martin, A., Maurer, F., Silveira, M., (2011), User-Centered Design and
Human Systems Engineering and Design (IHSED 2023): Future Trends and Applications
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-88-9
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional