Zhang Ping, Wenjing Ma, Feiyang Li, Zhenhua Yu (2025). Impact of EEG-Based Virtual Reality Haptic Force Feedback on User Experience. In: Metaverse, Virtual Environments and Game Design. AHFE Open Access, vol. 178, pp. 107–117. https://doi.org/10.54941/ahfe1006350
This study investigates the effects of haptic force feedback on brain neurofunctional connectivity and user immersion in virtual reality (VR) rehabilitation training. We used a VR setup with wearable force feedback devices to compare task performance between conditions with force feedback and those without it across different difficulty levels. By collecting Electroencephalography (EEG) signals and subjective data, we gained valuable insights into cognitive and emotional responses. Results showed enhanced neural activity and stronger immersion in the beta and gamma frequency bands under force feedback conditions. Multi-modal stimulation improved cognitive memory and user experience, with effects positively correlated to task difficulty. These findings show that combining natural interactions with our senses can improve virtual reality (VR) training and help develop better rehabilitation methods in the future.
Virtual Reality (VR) technology aims to create a seamless mapping to the real world through natural interaction and has become a research hotspot in the field of human-computer interaction. It plays a key role in areas such as intelligent manufacturing, rehabilitation and medical treatment, competitive training, and psychological counselling. Despite its stunning visuals and interactivity, VR technology still lacks realistic physical haptic feedback, with most headsets offering only limited haptic feedback, such as controller vibration (Galdieri, Camardella, Carrozzino, & Frisoli, 2022). Most studies focus on the visual channel and provide subjective analyses of users’ broad emotional and psychological characteristics (Jang, Kwon, Sun Gu Nam, Kim, & Hyun Kyoon Lim, 2022). Relatively little attention has been paid to the potential impact of other sensory channels on user immersion, thus neglecting the exploration of deep interaction between virtual and real. As a result, studying how multisensory interaction affects human-computer interaction has become a key area for future research (Li et al., 2023).
NEURAL BASIS OF HAPTIC INTERACTION AND VIRTUAL REALITY IMMERSION Haptic Interaction
Real-world perceptual experiences are natural and multi-sensory. Simulating this in virtual reality (VR) requires presenting information through multiple channels. Multisensory integration contributes to cognitive rehabilitation (Qu et al., 2023), and tactile stimuli significantly enhance the perceived intensity of visual stimuli (Xie et al., 2023). However, simply increasing the number of stimuli does not linearly correlate with user experience and may add cognitive load if not aligned with the virtual experience (Bauer & Andringa, 2020). Wearable devices such as haptic sensors and force feedback gloves are widely used for better user experiences. Research indicates that force feedback haptics are better than vibration feedback and non-haptic options in virtual reality (VR) (Moon et al., 2022).
Electroencephalography (EEG) EEG tracks brain activity using sensors on the scalp. It helps study how information types influence decision-making and behavior (Tan et al., 2020). It reflects the speed of response to different stimuli and can be used as a temporal estimate of neural activity behind attention and memory operations. EEG-based neurofeedback in VR devices can improve rehabilitation effects by simulating real-world motor skills (Dubovi, 2022).
The Research Framework This study uses a framework that combines humans, virtual reality, and physical elements to measure how multisensory interactions help each other, as illustrated in Figure 1. Behavioral data from VR tasks are combined with EEG data to form a comprehensive evaluation method. It is believed that when tasks get harder, using multiple senses can improve focus and brain function, which in turn impacts movement skills and how different parts of the brain connect (Shen et al., 2021). These findings provide evidence for cognitive enhancement and offer a new assessment method for rehabilitation interventions.
Figure 1: Virtual reality man-machine system architecture.
Impact of EEG-Based Virtual Reality Haptic Force Feedback on User Experience 109
HAPTIC INTERACTION PROTOTYPING Interactive System Architecture
A virtual training program was developed to help people improve their finger and wrist movements. This program uses different tasks to fully engage the senses and effectively assess and train the participants’ thinking skills, as shown in Figure 2. The user wears a head-up display device and an EEG device, receiving visual and auditory information through the head- up display. The force feedback glove’s finger tracking performs the two-way interaction of operation-haptic feedback. At the same time, the EEG signal analyzes the user’s cognitive-emotional regulation feedback mechanism, adopting the optimal interaction scheme for different task hierarchies.
Figure 2: Multi-modal interactive training system framework.
Force Feedback Glove Module The haptic feedback system has two main parts: creating a prototype for the haptic device and setting up how the virtual hand works in Unity3D. This system provides force feedback that gives users a feeling of grip. The haptic gloves developed are made of fine-fiber woven fabric with thermosetting platens added at the second and third joints of the fingers to ensure that the rotation center of the linkage structure coincides with the human joint axis. The motor creates a rebound force that travels to the finger joints through a stretchy nylon cord. A wire guide is used to prevent the fingers from moving together during rehab training. The device uses vibration to enhance the touch experience in the virtual environment. Users who interact with virtual objects would feel their presence and can perform more natural manipulations. The corresponding motor attached to each finger on the physical glove vibrates when the virtual hand touches the virtual object. The
system finally adopts 10 SG90 micro servo motors with an ultimate output torque of 9 kg·cm.
Virtual Training Scenarios To connect this study to typical hand action games in virtual reality, tasks incorporated motor and cognitive components: sorting different blocks and stacking them to create higher levels. Participants were allowed 10 minutes to complete the tasks, aiming to solve them as many times as possible and enhance their scores. Various levels of cognitive load (low and high) were induced under two different difficulty tasks by adapting specific features of the task and asking participants to memorize visual stimuli while performing the assembly task. After completing tasks with difficulty of different levels, subjects recalled as many details as possible of what they had just experienced. The number of stimuli to be remembered varied for each condition: for the simple condition, the type, color, number, and placement of the blocks needed to be remembered, and for the complex condition, the order of appearance, color, and predicted self-performance needed to be remembered.
VIRTUAL REALITY EEG EXPERIMENTAL RESEARCH Experimental Subjects
We divided twelve volunteers, six of whom were males from the university, aged 20–30 years (mean = 24.33, SD = 1.67), into two groups: no- feedback and force-feedback. Prior to the experiment, participants had no head trauma, neurological disorders, or consumption of substances affecting mental states. They also had normal vision and were well-rested.
Experimental Materials The EEG signal acquisition module used the g.USBamp-Research UHF Bioelectric Signal Acquisition and Analysis System, sampling at 256 Hz with 16-channel wet electrodes arranged by the international 10–20 system, as shown in Figure 3. The VR headset was a PICO Neo 3, and the force feedback glove module provided tactile illusions.
Figure 3: EEG channel location division.
Impact of EEG-Based Virtual Reality Haptic Force Feedback on User Experience 111
Experimental Design We conducted a 2-by-2 within-group experiment. Subjects were divided into no-feedback and force-feedback groups, each completing three experimental components corresponding to calm, simple, and complex cognitive load levels. Tasks included Two-Step (building blocks) and Three-Step (shooting balloons), with time and accuracy recorded. See Figure 4 for mission planning.
Figure 4: Experimental task planning.
Experimental Procedures Before the experiment, subjects were briefed on the tasks and instructed to minimize head movements. The experiment consisted of a 10-minute resting state followed by two 10-minute task states (simple and complex tasks), see Figure 5. The force feedback group used a control glove, while the joystick group used the PICO Neo 3’s joystick controller without force feedback and performed a task-state test, as shown in Figure 6.
ANALYSIS OF EXPERIMENTAL DATA EEG Signal Pre-Processing
Due to EEG signal susceptibility to artifacts, inappropriate data were removed from each participant’s experimental data. Triple filtering and independent component analysis (ICA) methods were used to remove these artifacts.
Analysis of Brain Activation States EEG time-frequency transformations started with a task-state cue, and neural activity began at 500 ms after receiving instructions. The extraction interval of interest was 0 to 1500 ms, as shown in Figure 7. The brain region map shows where different EEG signal types are located, highlighting the differences in these signals, as seen in Figure 8. EEG topographic maps were created in the alpha, beta, and gamma bands. The results showed that in the gamma band, the force feedback group had much higher power during complex tasks than the other groups, mainly in the frontal and parietal areas.
Figure 7: Time-frequency analysis of EEG signal.Figure 8: Topographic map of brain electrical channels.
FC Functional Connectivity Analysis Using pre-processed EEG data, the MNE-Connectivity library in the EEG Lab calculated the functional connectivity between six brain regions through a method called Phase Locked Value (PLV). Figure 9 shows that the strength of connections in the prefrontal and parietal areas of the brain increases
Impact of EEG-Based Virtual Reality Haptic Force Feedback on User Experience 113
during complex tasks. Figure 10 highlights the strongest connections in the FP1 and FP2 channels, as well as the strongest connectivity of the FP1 and FP2 channels.
Subjective Data Subjective information was collected after each experiment using a seven- point Likert scale. The results were analyzed using repeated measures ANOVA with SPSS software. The results are shown in Table 1, Figures 11 and Figures 12. The force feedback group had greater immersion with complex tasks but reduced usability, indicating that the inclusion of force feedback affected user learning and operational costs.
Table 1: Immersion scale (IPO).
Table available in the original PDF paper.
Event Sample Average Standard
F P
value deviation
handle 24 4.846 0.446 5.627 P = 0.027* mitten 24 5.340 0.396 Note: Average of all IPO questionnaires in all conditions. Error lines indicate the standard error of the mean.*p <0.05
Figure 11: Experimental subjective questionnaire score (* significant p < 0.05).Figure 12: The force feedback group was significantly different (* significant p < 0.05).
DISCUSSION
This study evaluated the effects of adding multi-channel feedback in virtual reality on brain function, including EEG signal time-frequency analysis,
Impact of EEG-Based Virtual Reality Haptic Force Feedback on User Experience 115
brain functional connectivity (FC), cognitive level, and behavioral skill performance (Riaz, Khan, Jawaid, & Shahid, 2021). A highly controllable natural interaction paradigm was designed to quantify emotional arousal, neurophysiological signals, and behavioral performance in immersive virtual reality at different task levels (simple vs. complex) by building a virtual training game and providing wearable force feedback devices.
The results showed that the multimodal interaction (audio-visual-tactile VAT) experience significantly enhanced subjects’ immersion and facilitated cognitive decision-making and learning memory. However, task performance showed a decreasing trend as task difficulty increased and cognitive load increased. EEG-based analyses and subjective scales indicated that adding haptic feedback to simple tasks benefited task performance and memory. For more complex tasks, one can adjust the strength of the feedback mechanism.
Consistent with previous literature, multimodal stimulation significantly enhanced subjects’ performance in virtual reality conditions compared to uni- sensory stimulation (Morton et al., 2022). Complex tasks invoke a higher perceptual load, making subjects think more when acting compared to simple tasks. This may cause subjects to miss more targets when multisensory channels intervene due to issues such as pseudo-tactile sensations. Due to the higher mental load induced by this situation, multimodal (VAT) stimuli significantly affected emotional arousal and stimulation. However, it may reduce response and information processing, confirmed by EEG time- frequency analyses, where evoked potentials appeared later for multisensory stimuli compared to sensory stimulus presentation in the no-feedback group.
The prefrontal cortex is closely associated with higher cognitive functions in humans (Morton et al., 2022). Along with other brain structures, the prefrontal cortex plays an important role in attention, perception, motivation, planning, sustained behavior, working memory, language, interference control, and executive functions (Liu Mingyu, Jue, Nan, & Qin, 2005). The area of the cerebral cortex associated with the production of movement is known as the motor area, and stimulation of this area elicits muscle movements in various parts of the body. The occipital lobe is the predominant motor imaginal cortex, and FC measurements reveal statistical dependencies between activity patterns in anatomically separated brain regions. They are commonly used to assess functional relationships between brain regions (Qu, Cui, Guo, Ren, & Bu, 2022). In this study, PLV was chosen as an index of FC, and the brain functional connectivity data showed that in the β and γ bands, the PLV values between the three brain regions in the force feedback group were significantly higher than those in the no-feedback group. Including tactile experience may have induced this change associated with the cognitive level. Incorporating multi-channel feedback affects many aspects of brain structure and function and helps enhance training effectiveness.
CONCLUSION
This study used EEG to monitor subjects’ brain signals in virtual reality during resting and task states, and VR was utilized for experimental design.
Brain function was evaluated from multiple perspectives using a multimodal information dataset incorporating both subjective and objective assessments. The results showed that haptic stimulation in virtual reality led to functional connectivity (FC) changes. In comparison to the no-feedback group, the force-feedback group showed that multimodal experiences enhance sensory- motor abilities and cognitive levels, leading to greater user immersion and emotional arousal. These findings contribute to a new validated assessment method that may be useful in the fields of rehabilitation and brain-computer interfaces. This physiological computational approach to multimodal data combines measurements from VR, force feedback wearables, behavioral metrics, subjective scales, and electroencephalographic (EEG). Using data from multiple sources can help accurately identify patients’ conditions and adjust training programs promptly in the field of virtual reality interaction.
ACKNOWLEDGMENT
This paper is supported by the 2020 Humanities and Social Science Research Foundation of the Ministry of Education of China, Title: Research on the Design Strategy of Elderly Products Based on Sensory Compensation (Item No. 20YJA760101).
REFERENCES
Bauer, A., & Andringa, G. (2020). The Potential of Immersive Virtual Reality for Cognitive Training in Elderly. Gerontology, 66(6), 1–10. https://doi.org/10.1159/ 000509830 Dubovi, I. (2022). Cognitive and emotional engagement while learning with VR: The
perspective of multimodal methodology. Computers & Education, 183, 104495. https://doi.org/10.1016/j.compedu.2022.104495 Galdieri, R., Camardella, C., Carrozzino, M., & Frisoli, A. (2022). Designing multipurpose devices to enhance users’ perception of haptics. Journal on Multimodal User Interfaces. https://doi.org/10.1007/s12193-022-00391-5 Jang, K.-M., Kwon, M., Sun Gu Nam, Kim, D., & Hyun Kyoon Lim. (2022).
Estimating objective (EEG) and subjective (SSQ) cybersickness in people with susceptibility to motion sickness. Applied Ergonomics/Applied Ergonomics, 102, 103731–103731. https://doi.org/10.1016/j.apergo.2022.103731 Li, T., Wang, H., Zhou, B., Li, Z., Chen, Z., Lan, Q., & Fan, D. (2023). Design and feasibility study of a HCPS framework-based VR alpine skiing decision-making training system. Computers & Graphics, 114, 138–149. https://doi.org/10.1016/ j.cag.2023.06.007 Liu Mingyu, Jue, W., Nan, Y., & Qin, Y. (2005). Development of EEG Biofeedback
System Based on Virtual Reality Environment. Conference Proceedings. https:// doi.org/10.1109/iembs.2005.1615693 Moon, H. S., Orr, G., & Jeon, M. (2022). Hand Tracking with Vibrotactile Feedback Enhanced Presence, Engagement, Usability, and Performance in a Virtual Reality Rhythm Game. International Journal of Human–Computer Interaction, 1–12. https://doi.org/10.1080/10447318.2022.2087000 Morton, J., Zheleva, A., Van Acker, B. B., Durnez, W., Vanneste, P., Larmuseau, C.,
… Bombeke, K. (2022). Danger, high voltage! Using EEG and EOG measurements for cognitive overload detection in a simulated industrial context. Applied Ergonomics, 102, 103763. https://doi.org/10.1016/j.apergo.2022.103763
Impact of EEG-Based Virtual Reality Haptic Force Feedback on User Experience 117
Qu, J., Cui, L., Guo, W., Ren, X., & Bu, L. (2022). The Effects of a Virtual Reality Rehabilitation Task on Elderly Subjects: An Experimental Study Using Multimodal Data. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 30, 1684–1692. https://doi.org/10.1109/tnsre.2022.3183686 Qu, J., Zhang, Y., & Bu, L. (2023). Functional near-infrared spectroscopy in the
assessment of rehabilitation efficacy of virtual reality products for people with cognitive disorders. International Journal of Industrial Ergonomics, 97, 103500. https://doi.org/10.1016/j.ergon.2023.103500 Riaz, W., Khan, Z. Y., Jawaid, A., & Shahid, S. (2021). Virtual Reality (VR)-Based Environmental Enrichment in Older Adults with Mild Cognitive Impairment (MCI) and Mild Dementia. Brain Sciences, 11(8), 1103. https://doi.org/10.3390/ brainsci11081103 Tan, W., Xu, Y., Liu, P., Liu, C., Li, Y., Du, Y., … Zhang, Y. (2020). A method of
VR-EEG scene cognitive rehabilitation training. Health Information Science and Systems, 9(1). https://doi.org/10.1007/s13755-020-00132-6 Xie, J., Lan, P., Wang, S., Luo, Y., & Liu, G. (2023). Brain Activation Differences of Six Basic Emotions Between 2D Screen and Virtual Reality Modalities. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 31, 700–709. https://doi.org/10.1109/tnsre.2022.3229389
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Metaverse, Virtual Environments and Game Design
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-54-0
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional