Boris Englebert, Lodewijck Foorthuis, Guido Tillema (2024). Rest-Frame Cueing for Cybersickness Mitigation in Virtual Reality Helicopter Flight Simulation. In: Human Factors and Simulation. AHFE Open Access, vol. 139, pp. 16–25. https://doi.org/10.54941/ahfe1005020
The integration of Virtual Reality (VR) into flight simulation not only presents an opportunity to revolutionize flight crew training from an economical point of view, compared to conventional flight simulation methods. However, it also introduces challenges, such as an increased occurrence of cybersickness, potentially undermining the effectiveness of VR-based training. To optimize the benefits of VR, addressing the heightened likelihood of cybersickness is crucial. The most widely accepted theory for cybersickness attributes its emergence to a sensory conflict between visual inputs from the virtual environment and the motion sensed by the vestibular system The mitigation of this sensory conflict is considered an effective strategy for combating cybersickness. A potential method to dissolve this sensory conflict is to introduce a visual reference to the actual environment in the virtual environment, for which the resulting visual inputs correspond to the motion that is sensed in the actual environment. The presentation of such a visual reference in virtual environments, also referred to rest-frame cueing, is as such hypothesized to be an mitigation method for cybersickness, although the effectiveness of rest-frames on cybersickness mitigation is dependent on the precise manner in which the rest frame is cued to the operator. An example of how rest-frame cueing for cybersickness mitigation is through mixed reality (MR), in which the virtual environment is visually blended with the actual environment, which has shown promising results for the mitigation of cybersickness.However, it has also been found that while MR reduces cybersickness, it also leads to a decline in pilot performance in virtual environment flight training, attributed to reduced immersion and lower simulation fidelity. As an alternative, it is hypothesized that a visual grid in the virtual environment that is fixed to the orientation of the actual environment, also referred to as a Virtual Reference Grid (VRG), can be an effective alternative rest-frame cueing strategy that offers similar cybersickness reduction benefits as MR but may be more conducive to maintaining simulation fidelity. This study explores the effectiveness of a VRG as a form of rest-frame cueing on cybersickness mitigation, aiming to minimize cybersickness without compromising pilot performance. The effectiveness of a VRG in helicopter flight simulation for cybersickness mitigation and pilot performance was investigated by means of a human-in-the- loop experiment, featuring experienced helicopter pilots from the Royal Dutch Air Force. The experimental setup includes an AgustaWestland AW139 helicopter flight model, low-fidelity flight controls, and the Varjo XR-3 visual device for rendering the virtual environment. A bespoke scenario, featuring the challenging ADS-33E-PRF pirouette Mission Task Element (MTE), serves as the backdrop for evaluating cybersickness and pilot performance metrics. Participants navigate three conditions: No Visual Reference, VRG Reference, and MR Reference. Post-trial, participants report their cybersickness levels using the Misery Scale (MISC), with measures in place to prevent excessive build-up between conditions.The results highlighted subtle differences in pilot performance, with the VRG rest-frame negatively affecting distance and heading control in the execution of the ADS-33 pirouette Mission Task Element (MTE). However, it was also observed that the performance degradation for distance control in the VRG Reference condition was less severe than in the MR Reference condition, suggesting that VRG rest-frame cueing may be less detrimental for certain aspects of pilot performance.The contrasting outcomes between MR and VRG rest-frame cueing emphasize the need for a deeper exploration of the role of cognitive processing of rest-frame cues for effective cybersickness mitigation. The study suggests that participants’ lack of cognitive awareness regarding the fixed nature of the VRG with respect to the actual environment contributed to its limited impact on mitigating cybersickness. This insight underscores the critical importance of effective design in rest frame cues for cybersickness mitigation in VR. In light of these findings, it is evident that designing rest-frame cues aligned with participants’ cognitive processes and enhancing their awareness of the virtual environment is crucial for the purpose of cybersickness mitigation and promoting pilot performance in VR helicopter flight simulation.
The integration of Virtual Reality (VR) into flight simulation not only presents an opportunity to revolutionize flight crew training from an eco- nomical point of view, compared to conventional flight simulation methods. However, it also introduces challenges, such as an increased occurrence of cybersickness, potentially undermining the effectiveness of VR-based training (Chang et al., 2020). To optimize the benefits of VR, addressing the height- ened likelihood of cybersickness is crucial. The most widely accepted theory for cybersickness attributes its emergence to a sensory conflict between visual
Rest-Frame Cueing for Cybersickness Mitigation in Virtual Reality Helicopter 17
inputs from the virtual environment and the motion sensed by the vestibular system (Chang et al., 2020; Palmisano et al., 2020; Reason and Brand, 1977). The mitigation of this sensory conflict is considered an effective strategy for combating cybersickness.
A potential method to dissolve this sensory conflict is to introduce a visual reference to the actual environment in the virtual environment, for which the resulting visual inputs correspond to the motion that is sensed in the actual environment. The presentation of such a visual reference in virtual environments, also referred to rest-frame cueing, is as such hypothesized to be an mitigation method for cybersickness (NATO Science & Technology, 2021), although the effectiveness of rest-frames on cybersickness mitigation is dependent on the precise manner in which the rest frame is cued to the operator. An example of how rest-frame cueing for cybersickness mitigation is through mixed reality (MR), in which the virtual environment is visually blended with the actual environment, which has shown promising results for the mitigation of cybersickness (Englebert et al., 2023).
However, it has also been found that while MR reduces cybersickness, it also leads to a decline in pilot performance in virtual environment flight train- ing, attributed to reduced immersion and lower simulation fidelity (Englebert et al., 2023). As an alternative, it is hypothesized that a visual grid in the vir- tual environment that is fixed to the orientation of the actual environment, also referred to as a Virtual Reference Grid (VRG), can be an effective alter- native rest-frame cueing strategy that offers similar cybersickness reduction benefits as MR but may be more conducive to maintaining simulation fidelity. This study explores the effectiveness of a VRG as a form of rest-frame cue- ing on cybersickness mitigation, aiming to minimize cybersickness without compromising pilot performance.
The effectiveness of a VRG in helicopter flight simulation for cybersickness mitigation and pilot performance will be investigated by means of a human- in-the-loop experiment, incorporating multi-modal interaction with the real and virtual environments. It is expected that the participants will be able to cognitively process that the visual motion inputs from the VRG in the virtual environment correspond with the motion sensed in the actual environment, thereby diminishing the sensory conflict and alleviating feelings of cybersick- ness, while not breaking immersion to such an extent that it leads to pilot performance degradation.
METHOD
In order to test the hypothesis that a VRG is effective for the mitigation of cybersickness in virtual environments, a human-in-the-loop experiment is carried out. This human-in-the-loop experiment featured a number of partic- ipants performing a set of flight maneuvers on a predefined parcours, while intermittently reporting their self-perceived level of cybersickness. This exper- iment task was performed in three conditions: (1) a baseline condition in which no rest-frame cueing is presented to the participant, also referred to as the No Visual Reference condition, (2) a condition in which a VRG is used
as rest-frame cueing, known as the VRG Reference condition, and (3) a con- dition that features MR as a form rest-frame cueing. This is carried out in the same way as was done by Englebert et al. (2023), in order to compare this rest-frame cueing method to the VRG, called the MR Reference con- dition. This section describes the exact method that was employed for the human-in-the-loop experiment.
Simulation Setup For the human-in-the-loop experiment simulation setup, a simulated AgustaWestland AW139 helicopter flight model was utilized, coupled with low-fidelity helicopter flight controls and a Varjo XR-3 visual device to present the virtual environment. The Varjo XR-3, identified as an enhanced Mixed Reality device, boasts a high-resolution focus area (27
x 27
) and incorporates two RGB cameras for video input. The generation of the virtual environment was accomplished using Unity version 2020.3.4.
To ensure uniformity in each participant’s experimental flight profile, a scenario encompassing multiple Mission Task Elements (MTE) from the ADS-33E-PRF (Baskett, 2000) was devised. Despite executing various MTEs during the scenario, particular attention was directed towards the ADS-33E- PRF pirouette MTE during results analysis, as it was deemed both the most visually dynamic and the most flight-technically challenging maneuver. In this task, the participants were instructed to navigate a circular trajectory while maintaining a consistent heading aligned with the center pole at specified dis- tance and height. The scenario also incorporated a 15-knot wind in a fixed direction. An expectation is set that the dynamic, near-ground maneuvers in the pirouette MTE may heighten cybersickness more profoundly compared to stable tasks at higher altitudes. This anticipation is rooted in the likelihood of increased sensory conflict due to elevated vection levels (Lawson, 2014; Zelie and Qadeer, 2019). Figure 1 provides a visual representation of the pirouette MTE virtual parcours and flight task, and detailed specifications of the pirouette performance can be found in Table 1.
Adequate Performance Maintain distance to circle center (100 ft) ± 10 ft ± 15 ft Maintain radar altitude / height (15 ft) ± 3 ft ± 10 ft Complete full circle within 45 seconds (∼8 kts)
60 seconds (∼6 kts)
Cybersickness Measurements Given the acknowledged variability in motion sickness susceptibility across individuals, participants in the experiment will undergo the Motion Sickness Susceptibility Questionnaire (MSSQ), designed by Golding (2006), before their involvement. This questionnaire aims to assess the diverse degrees of
Rest-Frame Cueing for Cybersickness Mitigation in Virtual Reality Helicopter 19
motion sickness susceptibility within the participant pool. The outcomes of the MSSQ will serve as a means to elucidate any irregularities or notable outliers observed in the cybersickness results. Additionally, participants will engage with the Misery Scale (MISC) following each trial. Ranging from 0 (“No problems”) to 10 (“Vomiting”), the MISC provides a quantifiable measure of the discomfort experienced by participants during the experiment.
Participants Four helicopter pilots from the Royal Dutch Air Force were enlisted for the experiment, averaging 35.8 years in age (SD = 6.83 years) and accumulat- ing an average flight experience of 1634 hours (min = 400, max = 4000). Among them, three pilots specialized in operating the Chinook, while one pilot possessed proficiency in both the AW139 and the NH90 helicopters.
To gauge the participant’s susceptibility to cybersickness, the Motion Sickness Susceptibility Questionnaire (MSSQ) by Golding (2006) was admin- istered prior to the experiment. The combined MSSQ score for the four participants averaged 8.75 (SD = 7.89). This positions the mean score within the 40th to 50th percentile range for motion sickness susceptibility, indicat- ing a slightly lower susceptibility compared to the general population average (Golding, 2006). Consequently, it is deduced that they can partake in the experiment without significant concern for experiencing excessive sickness.
Figure 1: Combining the pirouette MTE procedures and performance limits outlined by Baskett (2000) with the virtual environment implementation of the pirouette MTE course.
RESEARCH DESIGN As discussed, three experiment conditions were devised to assess the influence of visual referencing utilizing MR and motion on cybersickness and flight per- formance: the No Visual Reference condition, the MR Reference condition, and the VRG Reference condition. A visual representation of these conditions in the virtual environment, showing the precise form of rest-frame cueing that was applied, is presented in Figure 2. From Figure 2a, it can clearly be seen that no visual artifacts related to the rest-frame are present in the No Visual Reference baseline condition, while Figure 2b shows the visual blending of the virtual and actual environments using an 80–20 transparency ratio that is used in the MR Reference condition, where the physical room in which the simulator is situated can be seen. Finally, Figure 2c shows the presence of a white visual grid as a VRG for rest-frame cueing, for which the orientation is fixed to the actual environment in which the simulator is situated, used in the VRG Reference condition.
Participants undergo all three experimental conditions within a single day, adhering to a predetermined counterbalanced order. Prior to commencement, participants receive instructions on utilizing the MISC (Bos and Patterson, 2006) and execute a familiarization run through the virtual environment in the No Visual Reference configuration. Following the familiarization run, participants rest until reporting a MISC score of 0, ensuring no carryover effects from the familiarization.
Figure 2: The participant’s virtual cockpit eye reference point view for the three exper- iment conditions: (a) no visual reference, (b) MR reference, and (c) VRG reference.
Rest-Frame Cueing for Cybersickness Mitigation in Virtual Reality Helicopter 21
Within each experimental condition, participants navigate the virtual course, encompassing all selected Mission Task Elements (MTEs), includ- ing the pirouette, performed twice to achieve a total scenario duration of approximately twenty minutes. This duration allows adequate time for the manifestation of cybersickness. Upon the completion of each MTE, the par- ticipants report their MISC score. In addition, the experiment condition was aborted when the participant reported a score of MISC 3, indicating mild cybersickness symptoms such as stomach awareness and dizziness, in order to prevent the excessive build-up of cybersickness that could prevent the partic- ipants from properly returning to a baseline level of cybersickness to execute the remaining experiment conditions on the experiment day. After complet- ing a condition, the participants rested to recover from any cybersickness symptoms before initiating the next condition, contingent upon reporting a MISC score of 0, in order to prevent the cybersickness build-up from one condition affecting the development of cybersickness in the next condition.
Dependent Measures and Data Analysis Given the limited sample size and the exploratory nature of the experiment in this paper, no formal statistical analyses are undertaken. Nevertheless, various parameters are evaluated. Descriptive statistics and visual representa- tions of cybersickness, specifically the MISC scores reported, are employed to scrutinize differences across conditions. Performance assessment involves the analysis of pertinent pirouette performance parameters, encompassing height deviation, distance deviation, and deviation from the heading corresponding to a specific position. Furthermore, flight trajectories for each condition are mapped and evaluated using the performance conditions outlined in Table 1.
RESULTS
As described in the previous chapter, the results analysis features a descriptive assessment of both cybersickness and pilot performance measurements. The cybersickness and pilot performance results are presented in the next two sections.
Cybersickness
Figure 3 visualizes the average MISC scores and standard errors that were reported after the completion of the pirouette MTEs (n = 8) for the three experiment conditions. Even though the average MISC scores and standard errors are comparable across the three conditions, it can be observed that the average MISC scores for the MR Reference condition appear to be slightly lower than for the baseline No Visual Reference condition, as was also reported by Englebert et al. (2023). However, from Figure 3 it is also appar- ent that this reduction in the average MISC score cannot be observed when comparing the VRG Reference and No Visual Reference conditions. In fact, the average of the MISC scores reported for the VRG Reference condition appears to be slightly higher than for the No Visual Reference condition.
Despite the fact the average MISC scores differ slightly across the experi- ment conditions, t-test paired samples statistical testing did not reveal any statistically significant differences for the MISC scores between the ‘No
Visual Reference - MR Reference’ (t7 = 1.53, p = 0.511), ‘No Visual Refer- ence - VRG Reference’ (t7 = −0.424, p = 1.00), and ‘MR Reference - VRG Reference’ (t7 = 1.43, p = 0.591) experiment condition pairs.
Figure 3: Average MISC scores and standard errors (SE) reported for the pirouette.
Pilot Performance In accordance with the performance metrics for the ADS-33 Pirouette MTE in Table 1, the pirouette MTE height, distance, and heading deviations for all available datapoints are presented in Figure 4 for the three experiment condi- tions. By comparing the height and distance deviations with the performance specifications laid out in Table 1, it can be noted that the pilots had difficulty with adhering to both the desired and adequate performance margins, for all three experiment conditions. The same can be said for the heading deviation, for which it is clear from Figure 3 that deviations in excess of five degrees are common, while a deviation of zero degrees is desired as the participants were instructed to keep the nose of the helicopter pointed to the center object in the pirouette MTE parcourse.
Figure 4: Pilot performance in terms of the height, distance, and heading deviation.
Rest-Frame Cueing for Cybersickness Mitigation in Virtual Reality Helicopter 23
For the pilot performance in terms of the height and deviations, it can be argued that the deviations are comparable between the three experiment conditions, where only a minor increase in the heading deviations appears to be present for the VRG Reference condition, compared to the other two conditions. When it comes to the distance control, it can be observed from Figure 4 that the distance deviations are relatively high for the MR Reference and VRG Reference conditions compared to the No Visual Reference condi- tion, where the MR Reference condition appears to more negatively impact the distance control pilot performance than the VRG Reference Condition.
The notion that the rest-frame conditions result in poorer pilot perfor- mance in terms of distance control compared to the No Visual Reference condition is also apparent when examining the flown pirouette MTE trajec- tories, which are visualized by means of a top-down view in Figure 4. To elaborate, a comparison between the flight trajectories in Figure 4 and the reference pirouette MTE trajectory shown in Figure 1 reveals that the flight trajectories for the No Visual Reference condition are not only more accurate with respect to the reference trajectory compared to the rest-frame conditions, but also that the variance of the distance to the pirouette center is higher for the rest-frame conditions than for the No Visual Reference condition. Between the two rest-frame conditions, it can be noted that the flight tra- jectories for the VRG Reference condition appear to be more accurate with respect to the reference trajectory than the MR Reference condition flight trajectories, highlighting that differences in distance perception and control do exist depending on the exact cueing of the rest frame.
Figure 5: Pirouette MTE pilot flight trajectories for the three experiment conditions: (a) no visual reference, (b) MR reference, and (c) VRG reference.
CONCLUSION
In order to assess the impact of a VRG as a form of rest-frame cueing on the development of cybersickness in VR helicopter flight simulation, a human-in- the-loop experiment was performed. While it had already been demonstrated that the use of MR rest-frame cueing can mitigate cybersickness (Englebert et al., 2023), the results from this human-in-the-loop experiment indicated that VRG rest-frame cueing does not have a mitigating effect on cybersick- ness, compared to the No Visual Reference baseline condition, in which no rest-frame cueing is presented. In terms of pilot performance, the results showed that the VRG rest-frame negatively affected distance and heading control in the pirouette MTE compared to the baseline condition. It was, however, also found the performance degradation for distance control in the VRG Reference condition was not as severe as in the MR Reference con- dition, indicating that a VRG rest-frame is potentially less detrimental for certain aspects of pilot performance than a MR rest-frame.
An explanation for these findings can be found in the remarks made by the participants after completion of the experiment. Three out of four partic- ipants reported that the VRG was interfering for the execution of the MTEs, that it did not provide any added value, and that attempts were made to ignore the VRG altogether. These remarks, in combination with the notion that the VRG did not have any beneficial effects for cybersickness mitigation, can be an indication that the participants did not understand that the VRG was fixed to the actual environment, and that the presence of the VRG only led to contrasting visual cues in the virtual environment. When it comes to pilot performance, the participants reported that the VRG rest-frame cue- ing did not impede on the visual perception of the virtual environment to the same extent as the MR rest-frame cueing, that effectively rendered the virtual environment less visible due to the blending with the actual environment. As such, they rated MR rest-frame cueing as more disruptive for task execu- tion than VRG rest-frame cueing, despite MR rest-frame cueing resulting in a reduction of cybersickness severity.
The contrasting outcomes between the MR and the VRG types of rest- frame cueing prompt a deeper exploration of the role of cognitive processing of rest-frame cueing in cybersickness mitigation. Participants’ lack of aware- ness regarding the fixed nature of the VRG as a rest frame to the actual environment raises questions about the effectiveness of this type of rest-frame cueing in providing a visual reference to the actual environment. The study suggests that the cognitive processing of the VRG was not sufficiently impact- ful in mitigating cybersickness. This conclusion highlights the importance of adequate rest-frame cueing design, to promote participants’ understand- ing and cognitive awareness of rest-frame cues for effective cybersickness reduction for future research.
The study’s findings underscore the critical role of effective design in rest- frame cues for cybersickness mitigation in VR. Designing cues that align with participants’ cognitive processes and enhance their awareness of the virtual environment appears to be crucial. This consideration becomes even more
Rest-Frame Cueing for Cybersickness Mitigation in Virtual Reality Helicopter 25
pertinent as the VRG rest-frame, while not directly contributing to cyber- sickness mitigation, exhibits positive impacts on cognition and interaction. Researchers and designers should strive for a delicate and careful balance between cognitive benefits and cybersickness mitigation when crafting virtual environments.
ACKNOWLEDGMENT
This research is part of an internally funded research program by the Royal Netherlands Aerospace Center. The results presented in this paper are part of a series of publications. We thank the Royal Dutch Air Force Pilots who participated in the experiments.
REFERENCES
Baskett B. “ADS-33E-PRF Aeronautical Design Standard Performance Specification – Handling qualities requirements for military rotorcraft”. In: Defense Technical Information Center (2000). Bos J. E., Mackinnon S., and Patterson A. “Motion sickness symptoms in a ship
simulator: Effects of inside, outside, and no view”. In: Aviation, Space, and Environmental Medicine 76 (2006), pp. 1111–1118. Chang E., Kim H. T., and Yoo B. “Virtual Reality sickness: A review of causes and measurements”. In: International Journal of Human–Computer Interaction 36 (2020), pp. 1658–1682. doi: 10.1080/10447318.2020.1778351. Englebert B., Marsman L., and Crijnen J. “Evaluating the Effectiveness of Mixed
Reality as a Cybersickness Mitigation Strategy in Helicopter Flight Simulation”. In: Human Factors and Simulation. AHFE (2023) International Conference 83 (2023). doi: 10.54941/ahfe1003570. Golding J. F. “Predicting individual differences in motion sickness susceptibility by questionnaire”. In: Personality and Individual Differences 41 (2006), pp. 237–238. doi: 10.1016/j.paid.2006.01.012. Lawson B. “Motion sickness symptomatology and origins”. In: Handbook of Virtual
Environment: Design, Implementation, and Applications (2014), pp. 531–600. NATO Science & Technology. “Guidelines for mitigating cybersickness in Virtual Reality systems. Peer-Reviewed Final Report of the Human Factors and Medicine Panel/Modeling & Simulations Group Activity”. In: Number 323 (NATO STO- TR-HFM-MSG-323) (2021). Palmisano S., Allison S. R., and Kim J. “Cybersickness in head-mounted displays is
caused by differences in the user’s virtual and physical head pose”. In: Frontiers in Virtual Reality 587698 (2020). doi: 10.3389/frvir.2020.587698. Reason J. T. and Brand J. J. “Motion sickness”. In: The American Journal of Psychology 90 (1977), pp. 1822–188. Zelie B. and Qadeer A. “Vestibular and mulit-sensory influences upon selfmotion
perception and the consequences for human behavior”. In: Frontiers in Neurology 10 (2019), pp. 1664–2295. doi: 10.3389/fneur.2019.00063.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors and Simulation
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-15-1
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional