Yujia Du, Wang Haining (2024). Evaluation of Wearing Comfort in Virtual Reality Devices Using Analytic Hierarchy Process. In: Ergonomics In Design. AHFE Open Access, vol. 129, pp. 182–191. https://doi.org/10.54941/ahfe1004823
This paper aims to evaluate the comfort of wearing virtual reality (VR) devices using a popular multi-criteria decision-making technique, the Analytic Hierarchy Process (AHP),which systematically organizes and simplifies complex decision-making issues by breaking them down into more manageable components. A hierarchy model was developed through factor analysis and the Delphi method, comprising a primary goal (comfort of VR devices), six main criteria (skin friendliness, breathability, weightiness, stability, tenderness and fit ), and twenty sub-criteria. The paper details the methodology employed, including the construction of a pairwise comparison matrix and the assessment of consistency. The robustness of this approach was verified via a case study. This research introduces a novel methodology for assessing VR device comfort and prioritizing the elements related to this aspect. The findings offer significant insights for manufacturers by identifying specific aspects for product design enhancement.
Today people increasingly expect more from the immersive interaction of Virtual Reality (VR), the wearing comfort of head mounted devices (HMDs) has emerged as an important topic. Especially, the wide application of VR devices in various fields have proposed huge challenges to the wear experi- ence in prolonged use, which becomes a decisive factor in the development of XR industry. The psychological concept of comfort can be consisted of material, aesthetic, socialization and conformity comfort (Dumur, Barnard et al., 2004). From physical point of view, comfort involves the physiological state, the bio-mechanical state and health problems (Dumur, Barnard et al., 2004). Vink and Hallbeck (2012) claimed that the definition of comfort is not only a mental state of a human being but also a feeling of the human body in reaction to its physical environment. This complex construct defies simplistic measurement and is intricately tied to product usage (Kokosis, Gould et al., 2022). Consequently, the subjective nature of comfort evaluation presents significant obstacles in conducting swift and systematic assessments within the design and testing phases.
Evaluation of Wearing Comfort in Virtual Reality Devices 183
In the field of wearables, scholars have promoted various perspective to evaluate products’ wearing comfort. Knight and Baber (2005) and Knight, Deen-Williams et al. (2006) decomposed the comfort of wearables with vir- tual reality (VR) or augmented reality (AR) technology into visual impact and overall comfort, which included emotion, attachment, harm, perceived change, movement and anxiety. Song, Shin et al. (2020) divided the wear- ing comfort of wireless earphones into comfort, pain, pressure and fixation. Kouchi and Mochimaru (2004) pointed that the preference of glasses’ frame size can be deduced by overall fit, and slip sensation and pressure sensation. Pearson (2009) found that most studies evaluated comfort or discomfort as a single indicator, small of them divided comfort by body regions, but the impact of symptoms, environmental variables and emotion factors were gen- erally not considered. Lin, Ze-yuan et al. (2021) proposed that the ergonomic assessment should integrate quantitative methods such as geometric adap- tation, physical measurement, and interactive characteristics, as well as qualitative methods such as user experience and expert evaluation.
It is obvious that decomposing the wearing comfort into sub-criteria is an effective method to make the final decision. During the last two decades, the analytic hierarchy process (AHP) (Saaty 1990) has become one of the most widely used methods for the practical solution of multi-criteria decision- making (MCDM) problems and also in ergonomic evaluation. Liu et al. (Liu, Lee et al., 2011) investigated the relationship between the pillow shape design and subjective comfort level by AHP method. Marciano et al. (Marciano, Rossi et al., 2018) developed an efficient and generally multi-criteria method- ology based on AHP for choosing the optimal ultrasound device. Nukman et al. (Nukman, Ariff et al., 2009) evaluated various conceptual design alter- natives using AHP and selected the best wheelchair design concept. The AHP, which is a flexible strategy for organizing complex concerns by decomposing them into simple components, is suitable to this specific application making judgment in VR devices’ wearing comfort.
This study studies the assessment of wearing comfort in VR devices and highlights the essential components of user experience as decontaminated from a comprehensive literature review, factor analysis (FA), and the Del- phi method. It further defines the efficacy of the AHP methodology in the judgement and hierarchical prioritization of criteria, thereby offering a robust framework for evaluating user comfort in VR applications.
METHODOLOGY
Related Work
In order to ascertain the principal factors influencing the comfort of VR devices, a comprehensive examination of relevant literature was conducted. Chen et al. (Chen, Wang et al., 2021) discussed that the comfort analy- sis of VR devices focused on human performance, pressure, fatigue and visual induced motion sickness. Ito et al. (Ito, Tada et al., 2019, Ito, Tada et al., 2021) conducted a subjective evaluation for HMDs with respect to five aspects, well fitted, well focused, fatigue by vision, fatigue by weight and fatigue by balance. Jin et al. (Jin 2018) decomposed the suitability of VR
184 Du and Wang
devices into three indicators: shading, stability, and heat dissipation. Gener- ally, researches on VR wearing comfort have centered around fit, mechanical factors and thermodynamics.
Fit. Due to the complexity and variety of human facial morphology, the fit of a VR device is key to its ability to provide a satisfying virtual immersion experience. Zhuozhe (Chi 2020) explored the effect of head type on VR inter- face wearing preference using fit as the indictor. Wang et al. (Wang and Chi 2021) proposed a method testing the fit of VR headsets combining real and virtual procedure and obtained the correspondence between the deviation analysis and subjective evaluation.
Mechanical factors. Mechanical studies of VR devices have focused on the effects of weight factors and human posture. Theis et al. (Theis, Alexander et al., 2013) evaluated the physical and perceptual load of an HMD through visual clarity, visual area, muscle electricity, and postural analysis. Yan et al. (Yan, Chen et al., 2018) and Song et al. (Song, Liu et al., 2018) explored the effect of weight and weight distribution of HMDs on wearing discom- fort. Lee et al. (Lee, Park et al., 2023) investigated the effect of different head positions on the perceived wearing comfort. In addition, surface elec- tromyography (sEMG) is often used to record muscle activity and contraction reflecting the degree of muscle fatigue. For example, neck joint loading was an important indicator for assessing the load exerted by the HMD (Chihara and Seo, 2018); head position while using the HMD had a significant effect on trapezius muscle activity (Knight and Baber, 2007).
Thermodynamics. To compare the thermodynamic performance of dif- ferent brands of VR devices, Wang et al. (Wang, He et al., 2020) used a Thermal Imaging Camera to obtain the temperature change of VR devices after a short-term wearing. Rupp et al. (Rupp, Michaelis et al., 2018) pointed out that discomfort occurred if the forehead temperature exceeds 38 degrees when wearing VR devices.
Subsequent to the literature review, the Delphi method was employed to identify discomfort points experienced during the use of VR devices. Thir- teen subjects who are physically healthy and have rich experience in using VR devices were invited to participate in the interviews. Participants were provided with five VR devices from well-regarded market brands such as Pico, Oculus, and HTC for random usage. They were instructed to detail and assess their psychological and physical sensations throughout the dura- tion of wear. The frequently mentioned concerns of the users were compiled, drawing from both the literature and interview findings, and are tabulated as high-priority dimensions in Table 1.
Table 1. Indicators obtained based on literature review and Delphi method.
Table available in the original PDF paper.
Number Indicator Number Indicator
1 Loose 11 Balance 2 Squeeze 12 Hot and humid 3 Pressure 13 Fatigue
(Continued)
Evaluation of Wearing Comfort in Virtual Reality Devices 185
Table 1. Continued Number Indicator Number Indicator
Factor Analysis FA was conducted to evaluate the indicators and cluster the related indi- cators into one group to extracted main elements of comfort. A total of ninety questionnaires with the importance of each indicator rated accord- ing to a five-point Likert were collected. Bartlett’s Sphere Test (df =754.734, p <0.001) was used to assess the appropriateness of the correlation matrix for FA and the Kaiser–Meyer–Olkin (KMO) (0.810) was used to measure of overall sampling adequacy. The results were well above the accepted level indicating the suitability of the FA performed (Sharma, 1995). Finally, six factors were extracted based on Kaiser criterion (Kaiser, 1960), and a clear factor structure was derived that explaining 70.09 % of the variance of var- ious indicators using Varimax rotation (Table 2). Based on the results of FA, the initial set of 20 indicators were reduced to six underlying factors. The variables in each factor provide a heuristic of labeled suggestions signifying different dimensions of VR devices’ wearing comfort. The titles of the fac- tors, presented in the first column of Table 2, were derived from a descriptive methodology that captures the essence of the items grouped under each factor. Definitions for each designated factor are illustrated as below.
Table 2. Wearing comfort elements of VR devices, their explained variance and the primary variables of each element.
Stability relates to the steadiness maintained by the device; it should move in unison with head movements without causing slippage or looseness. Fit involves the degree to which the device conforms to the user’s facial contours, avoiding issues such as light leakage or uncomfortable gaps. Tenderness relates to the absence of excessive pressure or squeezing sensations. Weight- iness is characterized by the minimization of fatigue in the head and neck regions, as well as the reduction of sensations such as vertigo. Breathability is concerned with preventing a hot and humid microclimate against the facial skin, thereby avoiding a stifling experience. Skin friendliness is defined by the gentle interaction between the device’s contact surfaces and the skin, ensuring no irritation or tingling sensations occur.
Analytic Hierarchy Process This paper discusses the use of AHP in the area of VR devices’ wearing comfort evaluation. Generally, there are three main steps consisted in AHP, hierarchy framework, priority analysis and consistency verification.
Develop a hierarchy model. According to the AHP, the general objective to be met is defined first, followed by the criteria to achieve this objective, the possible sub-criteria into which the criteria can possibly be broken down, etc. Specifically, the overall goal of this application was to evaluate the wear- ing comfort level of VR devices. Six criteria were obtained through previous work, and sub-criteria was presented as twenty indicators described by body regions. The hierarchy model for evaluating wearing comfort of VR devices is introduced as Figure 1.
Construct a pair-wise comparison matrix. In order to determine the matri- ces’ coefficients, a 9-point semantic Saaty scale (Saaty, 1990) was employed to quantify qualitative judgments. The pair-wise comparisons generated a matrix of relative rankings for each level of the hierarchy. 30 subjects who have rich experience in using VR devices were recruited to complete the comparisons based on their experience and knowledge.
Evaluation of Wearing Comfort in Virtual Reality Devices 187
Figure 1: The hierarchy model for evaluating wearing comfort of VR devices.
Perform the consistency. To ensure the reliability of the subjects’ evalua- tions, a final step known as consistency verification was implemented. First, a process of averaging over the normalized columns was conducted to synthe- size the pairwise comparison. The eigenvalue (λmax) was computed, and the right matrix of judgements was multiplied by the priority vector or eigenvec- tor (1) to obtain matrix size. Then the consistency index (CI) was calculated according to (2). The consistency is determined by the consistency ratio (CR) which is the ratio of consistency index (CI) to random index (RI) for the same order matrices (3). A CR value below 0.1 signifies that the judgments are within acceptable consistency thresholds.
(AW)i
n X
(1)
Equation reproduced from the original PDF.
Upon the completion of consistency checks across all levels, further calcu- lation of the overall priority vector to obtain the final wearing comfort level was performed. The elements in Table 3 correspond to the priority vectors assigned to the criteria and sub-criteria.
Table 3. Priority vectors assigned to the criteria and sub-criteria.
Table available in the original PDF paper.
Factors Priority Vec- tor (PV)
Primary Variables Priority Vec- tor (PV) Stability 0.2320 Head stability 0.7143 Occipital stability 0.2857 Fit 0.1155 Forehead fit 0.2460 Temporal fit 0.2010 Zygomatic fit 0.2768 Nasal fit 0.2762
REALITY TEST: A CASE STUDY To verify the applicability of the methodology for comfort evaluation of VR devices through AHP, fifteen participants were invited to participant a wear- ing test. They were instructed to use the Oculus Quest 2 and Pico 4 for thirty minutes each, on separate occasions. Each participant was required to undergo the wear test three times, on different days. Following each ses- sion, they were asked to rate their overall comfort as well as the individual criteria and sub-criteria, based on the Borg CR-10 scale, across the three trials.
Statistical analysis was conducted in SPSS 26. Overall comfort reported from the test was named as C, comfort which was calculated by criteria PV and criteria scores named C1, comfort which was calculated by the PV of sub-criteria and sub-criteria scores was named C2. The three sets of data all satisfy the conditions of normal distribution, and Pearson coefficients were used to test the correlation among them. The results are shown in Table 4. The correlation coefficients, exceeding 0.8, indicated a strong consistency in the wearing comfort scores obtained through the three different evalu- ation instances. This high degree of correlation highlighted the validity of the AHP-based methodology, confirming that the scores computed at both the criteria and sub-criteria levels were robust predictors of overall wearing comfort. Consequently, the decision-making process regarding the comfort of VR devices can be effectively guided by the practical methods presented in this study.
Evaluation of Wearing Comfort in Virtual Reality Devices 189
Table 4. Corelation analysis results of C, C1 and C2.
Table available in the original PDF paper.
Various Mean SD C C1 C2 C 3.417 1.930 1 C1 2.945 1.414 0.903** 1 C2 2.439 1.167 0.842** 0.905** 1
Equation reproduced from the original PDF.
CONCLUSION
Wearing comfort is intrinsically linked to the user experience of VR devices and is a decisive factor in product satisfaction. This study introduces an AHP-based methodology designed to facilitate the assessment of VR device comfort levels. A hierarchical model was constructed, comprising six criteria—skin friendliness, breathability, weightiness, stability, tenderness, and fit—and twenty sub-criteria. The practicality of this methodology was tested in a case study, revealing that the AHP-derived results were in har- mony with the subjective comfort ratings provided by the participants. The case study underlined the efficacy and user-friendliness of the proposed hier- archical approach. Findings indicate that “tenderness” is the predominant factor affecting comfort during VR use, with zygomatic tenderness identified as the most critical element. Stability emerged as the second most important, with head stability being a key concern. Subsequent priorities in descending order were weightiness, fit, breathability, and skin friendliness.
The insights from this study provide a more streamlined and thorough method for users to evaluate the comfort performance of wearable devices. Additionally, it offers manufacturers a useful benchmark for pinpointing specific design elements that require enhancement. The implications of this research are substantial for the fields of ergonomics and industrial design, with potential applications that extend well beyond VR devices to a wide array of head-wearables.
ACKNOWLEDGMENT
The authors would like to acknowledge the staff in the Anthropometry and Wearables Design Lab at Hunan University. Special thanks are given to the participants involved in this study.
REFERENCES
Chen, Y., X. Wang and H. Xu (2021). “Human factors: Ergonomics evaluation for virtual reality headsets: a review.” CCF Transactions on Pervasive Computing and Interaction. Chi, Z. (2020). Fit Research for the Facial Interface of Virtual Reality Head Mounted
Display. Master, Hunan university. Chihara, T. and A. Seo (2018). “Evaluation of physical workload affected by mass and center of mass of head-mounted display.” Applied ergonomics 68: 204–212. Dumur, E., Y. Barnard and G. Boy (2004). “Designing for comfort.” Human factors
in design: 111–127.
190 Du and Wang
Ito, K., M. Tada, H. Ujike and K. Hyodo (2019). Effects of Weight and Balance of Head Mounted Display on Physical Load. HCII. Ito, K., M. Tada, H. Ujike and K. Hyodo (2021). “Effects of the Weight and Balance
of Head-Mounted Displays on Physical Load.” Applied Sciences 11(15). Jin, W. (2018). Research on product fit design based on knowledge of head and face 3D shape. Doctor, Hunan university. Kaiser, H. F. (1960). “The application of electronic computers to factor analysis.”
Educational and psychological measurement 20(1): 141–151. Knight, J. F. and C. Baber (2005). “A tool to assess the comfort of wearable computers.” Human factors 47(1): 77–91. Knight, J. F. and C. Baber (2007). “Effect of head-mounted displays on posture.”
Human factors 49(5): 797–807. Knight, J. F., D. Deen-Williams, T. N. Arvanitis, C. Baber, S. Sotiriou, S. Anastopoulou and M. Gargalakos (2006). Assessing the wearability of wearable computers. 2006 10th IEEE International Symposium on Wearable Computers, IEEE. Kokosis, G., A. Gould, H. Darrach, K. Chopra, S. T. Hollenbeck, B. T. Lee and D.
Coon (2022). “Use of a Wearable Posture-Correcting Device to Train Residents in Plastic Surgery: A Novel Approach to Surgical Ergonomics and Prevention of Associated Musculoskeletal Disorders.” Plastic and Reconstructive Surgery 149(1): 166E–168E. Kouchi, M. and M. Mochimaru (2004). “Analysis of 3D face forms for proper sizing and CAD of spectacle frames.” Ergonomics 47(14): 1499–1516. Lee, Y., D. Park and Y. M. Kim (2023). “The effect of wearing a head-mounted dis-
play on the boundaries of the cervical range of motion based on perceived comfort in a static posture.” Virtual Reality 27(2): 815–828. Lin, L. Z., Z. Ze-yuan, W. Jing-jing, J. Li-jun, J. Shan-xiao, Y. Yu-fei and D. Xianghong (2021). “Review on human factor design and evaluation of head-mounted products “ Packing Engineering 42(16): 49–60. Liu, S. F., Y. L. Lee and J. C. Liang (2011). “Shape design of an optimal comfortable
pillow based on the analytic hierarchy process method.” J Chiropr Med 10(4): 229–239. Marciano, F., D. Rossi, P. Cabassa and P. Cocca (2018). “Analytic Hierarchy Process to support ergonomic evaluation of ultrasound devices.” IFAC-PapersOnLine 51(11): 328–333. Nukman, Y., H. Ariff and M. Salit (2009). “Use of analytical hierarchy process (AHP)
for selecting the best design concept.” Jurnal Teknologi 49(A): 1–18. Pearson, E. J. M. (2009). “Comfort and its measurement–a literature review.”
Disability and Rehabilitation: Assistive Technology 4(5): 301–310. Rupp, M. A., J. R. Michaelis, D. S. McConnell and J. A. Smither (2018). “The role of individual differences on perceptions of wearable fitness device trust, usability, and motivational impact.” Applied Ergonomics 70: 77–87. Saaty, T. L. (1990). “How to make a decision: the analytic hierarchy process.”
European journal of operational research 48(1): 9–26. Sharma, S. (1995). Applied multivariate techniques, John Wiley & Sons, Inc. Song, H., G. W. Shin, Y. Yoon and S. Bahn (2020). “The effects of ear dimensions and product attributes on the wearing comfort of wireless earphones.” Applied Sciences 10(24): 8890. Song, Y., Y. Liu and Y. Yan (2018). The Effects of Center of Mass on Comfort of Soft
Belts Virtual Reality Devices. AHFE.
Evaluation of Wearing Comfort in Virtual Reality Devices 191
Theis, S., T. Alexander, M. p. Mayer and M. Wille (2013). Considering ergonomic aspects of head-mounted displays for applications in industrial manufacturing. Digital Human Modeling and Applications in Health, Safety, Ergonomics, and Risk Management. Human Body Modeling and Ergonomics: 4th International Conference, DHM 2013, Held as Part of HCI International 2013, Las Vegas, NV, USA, July 21-26, 2013, Proceedings, Part II 4, Springer. Vink, P. and S. Hallbeck (2012). Comfort and discomfort studies demonstrate the
need for a new model, Elsevier. 43: 271–276. Wang, H. and Z. Chi (2021). “Fit Test Method Study of Wearable Products by Combining Virtuality and Reality “ Packing Engineering 42(12): 84-90+97. Wang, Z., R. He and K. Chen (2020). “Thermal comfort and virtual reality headsets.”
Appl Ergon 85: 103066. Yan, Y., K. Chen, Y. Xie, Y. Song and Y. Liu (2018). The Effects of Weight on Comfort of Virtual Reality Devices. AHFE.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Ergonomics In Design
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-05-2
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional