Yanjie Zhang, Fan Li, Su Han, Donglin Li (2024). Investigation of Potential fNIRS-based Biomarkers in Multi-Domain Virtual Reality Tasks for MCI Assessment. In: Accessibility, Assistive Technology and Digital Environments. AHFE Open Access, vol. 121, pp. 161–171. https://doi.org/10.54941/ahfe1004624
Alzheimer's disease (AD) is a progressive neurodegenerative condition and is currently the fourth leading cause of death in advanced nations. The primary cause of AD is the deterioration of neurons in areas of the brain crucial for memory, typically presenting symptoms like loss of memory and a decline in cognitive abilities. Mild Cognitive Impairment (MCI) represents a transitional phase between normal cognitive health and AD. Recent studies have shown that within five years, 32% of individuals diagnosed with MCI experience a progression to Alzheimer's disease. Hence, the early detection and treatment of MCI are vital in decreasing the likelihood of developing AD. Traditionally, MCI assessment has relied on neuropsychological tests such as the Mini-Mental State Examination (MMSE) and the Montreal Cognitive Assessment (MoCA). Nevertheless, these methods have limitations, including inducing participant anxiety and fatigue, cultural biases, and the need for skilled administrators. This has shifted towards more innovative assessment methods, particularly Virtual Reality (VR) technology. VR's engaging and multisensory environment offers the potential for more effective MCI assessment. Various VR tasks, such as the Virtual Supermarket Task (VST) and VR adaptations of the Morris Water Maze and Trail Making Test, have shown promise in delivering insightful performance metrics. However, existing research has primarily focused on VR task performance evaluation, often overlooking the corresponding brain activation patterns these tasks stimulate. Compared with the task performance, the stimulated brain patterns could more directly reflect the cognitive function changes resulting from MCI. Whether these VR tasks can induce distinguishable changes in functional near-infrared spectroscopy (fNIRS) data between MCI and healthy individuals and which fNIRS parameters could be useful for MCI assessment is still unknown. To address this research gap, we investigated human brain activity across MIC and healthy individuals in multi-domain VR tasks. First, we selected a VR drumming task which engages multiple cognitive domains, including motor skills, rhythm, and spatial-temporal orientation. Second, we extracted some potential MCI indicators, such as functional connectivity from fNIRS data to analyse brain activity across MIC and healthy individuals in the VR task. Lastly, we examined the statistically significant parameters and discussed the underlying brain activity patterns and their potential for MCI assessment. Our findings revealed that specific brain activity and functional connectivity parameters indicated significant differences between healthy and MCI groups, suggesting the potential value of these parameters as biomarkers for VR-based MCI assessment. This study introduced the potential fNIRS parameters for MCI assessment and discussed their implications and underlying reasons. In conclusion, our study lays a promising foundation for developing and refining VR-based MCI assessments. We anticipate our findings will lead to more effective VR task designs and promote widespread MCI screening in larger populations, ultimately aiding early detection and intervention in individuals at risk of dementia. Future research should address the identified limitations and explore further enhancements in MCI and related condition assessments.
Dementia presents significant challenges for individuals, families, and soci- ety, with no current effective treatments highlighting the need for preventive strategies (Kasper, Bancher et al. 2020). Early detection and intervention in Mild Cognitive Impairment (MCI), a precursor to dementia, are crucial (Mancioppi, Fiorini et al. 2019). Traditional neuropsychological assessments like the MMSE and MoCA, while useful, have limitations influenced by
various factors and require professional administration (Weintraub, Salmon et al. 2009). Consequently, researchers have been exploring new digital cog- nitive assessment methods, particularly those employing virtual reality (VR) technology.
VR has been utilized in tasks assessing spatial navigation, memory, and attention in MCI, offering metrics like completion time and error rate (Serino, Morganti et al. 2015, Cogné, Taillade et al. 2017). However, these may not capture subtle motor changes in MCI. This study explores functional near- infrared spectroscopy (fNIRS) as a non-invasive method to assess cerebral blood flow and oxygenation changes in the brain, offering insights into cog- nitive function (Yoo, Woo et al. 2020). fNIRS is advantageous in VR settings due to its lower sensitivity to movement artefacts and ability to probe deeper into cerebral blood flow and oxygenation changes, which are indicative of cognitive functioning. Previous research using fNIRS in visual tasks showed decreased activation in MCI patients’ occipital and parietal cortices (Doi, Makizako et al. 2013), suggesting deficits in visual processing.
Proposing that VR’s immersive nature might induce unique cognitive demands and neural responses in MCI patients, this study aims to bridge research gaps by using a multi-domain VR task and fNIRS to analyze brain activity in healthy and MCI individuals. However, several challenges need to be addressed:
1. Select a VR task that effectively elicits comprehensive cognitive informa- tion while stimulating multiple cognitive domains. 2. Extract the potential features from the fNIRS data and analyze the underlying reasons why it could be used to assess MCI effectively.
The manuscript is organized as follows: First, the Materials and Meth- ods part offers an in-depth explanation of the experimental procedures, data preparation, and feature extraction methodologies. Subsequently, the find- ings segment presents the results from the brain functionality data. The Interpretation segment illustrates the outcomes and potential applications for MCI evaluation. Finally, the Closing segment summarizes the principal discoveries and restrictions and proposes potential research direction.
MATERIALS AND METHODS
VR Device and Cognitive Task In this study, the Beat Arena VR drumming task was employed to engage a broad spectrum of cognitive functions, including executive function, mem- ory, orientation, attention, language, and visuospatial abilities. The task initiates with a welcome screen, progressing to an interactive stage where users respond to color-coded drums and circles to score points. Despite initial challenges for some with cognitive impairments, overall feedback sug- gests the task’s potential for cognitive stimulation and assessment. Cognitive engagement is multifaceted:
Executive Function: Users exercise cognitive flexibility and inhibitory control by timely drum strikes.
Investigation of Potential fNIRS-Based Biomarkers in Multi-Domain Virtual Reality Tasks 163
Memory: Recollection of game rules and timing is critical. Orientation: Spatial recognition of drum locations is required. Attention: Concentration is maintained through the tracking and striking of drums.
Language: The task incorporates verbal instructions and necessitates the recognition of visual cues.
Visuospatial Abilities: Coordination between visual signals and motor responses is essential, enhanced by visual, auditory, and tactile stimuli.
The Oculus Quest 2, a versatile wireless VR system powered by Qualcomm Snapdragon XR2, was utilized for its high-resolution display and adaptable refresh rates, supporting comprehensive gesture tracking. Beat Saber was selected for its rhythm-based gameplay, promoting significant upper limb movement and cognitive engagement.
Participants “A total of 27 healthy controls (13 males and 14 females, average age 36 ±10 years) and 20 MCI patients (8 males and 12 females, average age 66 ±9 years) were recruited from a local community. Demographic informa- tion for each group is provided in Table I. The MCI participants underwent screening using the Mini-Mental State Examination (MMSE) and the Mon- treal Cognitive Assessment (MOCA) to evaluate their overall cognitive status. Participants with a history of heavy alcohol consumption or drug abuse were excluded within the four weeks before the study. The research design adhered to professional and ethical guidelines for conducting human subject research.
Experiment Procedure As illustrated in Fig. 1, virtual scenes from the VR glasses were displayed on a television screen to facilitate assistance for the tester.
fNIRS Measurement The 1[HbO2] signals were obtained utilizing a multichannel commercial fNIRS apparatus (Nirsmart; Danyang Huichuang Medical Equipment Co, Ltd, PR China), featuring wavelengths of 760 and 850 nm. A 10 Hz sam- pling frequency was employed. The system consisted of a headpiece, optical fibres, a signal acquisition unit, a transmission component, and a primary computer. The headpiece was securely wrapped around the subject’s head, ensuring the emitter and detector probes were properly fastened around the participant’s head. The fNIRS system featured 18 channels placed over the patients’ lateral prefrontal cortex (LPFC), the lateral motor cortex (LMC), lateral occipital lobe (LOL), and right and left prefrontal cortex (RPFC and LPFC, respectively) as well as the motor cortex (RMC and LMC, respec- tively). The channel positions of the fNIRS system are displayed in Fig. 2. The head cap was securely fastened around the participants’ heads to ensure accurate probe placement. Previous research has demonstrated the reliability and stability of this system (Li, Ng et al. 2021).
Figure 1: The experiment setup.Figure 2: The channel position of fNIRS.
fNIRS Pre-Processing The HbO data were extracted for analysis, and the subsequent preprocessing steps were implemented using NIRS-KIT Matlab software (Hou, Zhang et al. 2021). First, polynomial regression models were employed to estimate linear or nonlinear trends for detrending purposes. Next, the TDDR method (Fish- burn, Ludlum et al. 2019) was applied to correct motion artefacts. Lastly, a
Investigation of Potential fNIRS-Based Biomarkers in Multi-Domain Virtual Reality Tasks 165
third-order IIR filter (0.01 to 0.08 Hz) was utilized to filter the fNIRS data. Fig. 3 compares the raw data and preprocessed data. The red line depicts the unprocessed fNIRS data, while the green line demonstrates the preprocessed data. Upon processing the data, the following parameters are calculated.
Figure 3: Preprocess and raw fNIRS data.
Functional Brain Network Indicators Brain Area Activation For 18 channels, we calculate the average hemoglobin concentration. Then we calculate the total hemoglobin concentration of each brain region. Fig. 4 is the brain activation of 18 channels.
Figure 4: Oxyhemoglobin concentration amplitude of all channels.
Functional Connectivity Previous research has demonstrated that changes in functional connectivity, as indicated by Pearson correlation coefficients calculated as (1), can dif- ferentiate MCI patients from healthy individuals (Fishburn, Ludlum et al. 2019).
rij = r (CHi, CHj ) = Cov(CHi, CHj) pVar (CHi) Var(CHj) (1) Statistical Analysis To examine differences in activation levels and functional connectivity (FC) between MCI patients and healthy individuals, independent samples t-tests were employed for between-group comparisons. Additionally, the impact of the cognitive state on brain function was explored by utilizing paired t-tests to investigate differences between task and resting states within each group. A statistical significance (P) threshold was set at 0.05 for all analyses.
RESULTS
Level of Activation The analysis results demonstrate statistically significant differences in brain activation between the MCI and Healthy groups. In the left occipitotempo- ral lobe (LOL), the MCI group exhibits higher activation than the Healthy group (F = −2.453, P = 0.018). Additionally, the MCI cohort exhibits sig- nificantly increased activity in the left prefrontal cortex (LPFC) compared to the Healthy cohort (F = −2.145, P = 0.037) throughout the resting condi- tion. Fig. 5 displays the mean brain activation across various brain regions, where the x-axis represents the mean brain activation values, and the y-axis indicates the group and corresponding state. Fig. 6 depicts the statistically sig- nificant differences in mean brain activity values, including error bars derived from the standard error of the mean, which convey the variability of the mean values.
Figure 5: The mean brain activation of brain regions.
Investigation of Potential fNIRS-Based Biomarkers in Multi-Domain Virtual Reality Tasks 167
Figure 6: The statistically significant result of the mean brain activity.
Functional Connectivity The analysis employing independent samples t-tests disclosed significant distinctions in functional connectivity (FC) metrics between the MCI and healthy cohorts. In the resting condition, the MCI group demonstrated notably elevated FC metrics in the LOL - RMC (F = −2.296, P < 0.05) and LMC - LPFC (F = −2.674, P < 0.05) links in comparison to the healthy cohort. In the task condition, the healthy cohort displayed considerably reduced FC metrics for the ROL - LMC (F = −2.864, P < 0.05), ROL - RPFC (F = −2.057, P < 0.05), RMC - LMC (F = −2.325, P < 0.05), and LMC - LPFC (F = −2.815, P < 0.05) connections relative to the MCI cohort.
Paired t-test outcomes revealed that the healthy cohort exhibited sig- nificantly elevated FC metrics in the task condition compared to the rest condition for multiple connections, including ROL - RMC (F = 5.613, P < 0.05), ROL - LMC (F = 6.058, P < 0.05), ROL - RPFC (F = 4.315, P < 0.05), and ROL - LPFC (F = 5.401, P < 0.05), RMC - LMC (F = 8.050, P < 0.05), RMC - RPFC (F = 3.469, P < 0.05), RMC - LPFC (F = 5.407, P < 0.05), LMC - RPFC (F = 2.888, P < 0.05), LMC - LPFC (F = 7.066, P < 0.05). Conversely, the MCI cohort displayed significantly decreased FC met- rics in the task condition relative to the rest condition for connections such as ROL - RMC (F = 4.799, P < 0.05), ROL - LMC (F = 3.614, P < 0.05), ROL - RPFC (F = 3.662, P < 0.05), ROL - LPFC (F = 5.123, P < 0.05), LOL - RMC (F = 2.198, P < 0.05), LOL - LMC (F = 2.169, P < 0.05), LOL - RPFC (F = 3.246, P < 0.05), RMC - LMC (F = 4.823, P < 0.05), RMC - RPFC (F = 3.766, P < 0.05), RMC - LPFC (F = 5.153, P < 0.05), LMC - RPFC (F = 4.316, P < 0.05), LMC - LPFC (F = 5.985, P < 0.05), RPFC - LPFC (F = 3.224, P < 0.05).
Fig. 7 and 8 provide visual representations of the functional connectivity, with Fig. 7 depicting FC metrics for an individual data point in the quiescent state of the Healthy cohort, while Fig. 8 displays the FC metrics across the four cases.
Figure 7: FC values of one rest state subject in the healthy group.Figure 8: FC values across the four cases.
DISCUSSION
This study investigated brain activation disparities in individuals with Mild Cognitive Impairment (MCI) and healthy controls using fNIRS during rest and a VR drumming task. The task involved interacting with virtual drums in VR, with data analysis showing significant brain activation and functional connectivity differences between groups. These results underline unique neu- ral patterns in MCI, contributing to early detection and monitoring of cognitive decline.
Analysis of Brain Activation Experimental Results During the rest state, the MCI group displayed significantly elevated activa- tion in the left prefrontal cortex (LOL) compared to the Healthy group. The increased activation observed in the left occipitotemporal lobe (LOL) and left prefrontal cortex (LPFC) might indicate altered connectivity in individu- als with MCI. Two competing theories, compensation and dedifferentiation
Investigation of Potential fNIRS-Based Biomarkers in Multi-Domain Virtual Reality Tasks 169
(Yu, Wang et al. 2020) could explain the increased activation in the LPFC during the resting state. Compensation theory suggests that the increased activation is a mechanism to counteract age-related decline. In contrast, ded- ifferentiation theory proposes that this increase is due to reduced brain ability to differentiate between various cognitive processes (Park and Reuter-Lorenz 2009). The increased activation could result from MCI individuals’ brains attempting to compensate for cognitive deficits by recruiting additional brain regions to maintain performance (Hillary and Grafman 2017).
During the task state, the MCI group also displayed significantly ele- vated activation in the left prefrontal cortex (LPFC) compared to the Healthy group. The prefrontal cortex is crucial for cognitive control processes, such as working memory, attention, and inhibitory control (Miller and Cohen 2001). Increased activation in the LPFC might reflect the MCI group’s need for additional cognitive control to maintain performance on the VR drum- ming task. The observed differences in brain activation patterns between MCI and healthy individuals could have implications for developing interventions targeting MCI, such as cognitive training programs targeting the LPFC to improve cognitive function in individuals with MCI (Belleville, Bherer et al. 2008).
Analysis of Functional Brain Connectivity Experimental Results In the resting state, the MCI group’s increased FC values in the LOL - RMC and LMC - LPFC connections might reflect compensatory mecha- nisms or disruptions in the default mode network (DMN). Studies suggest that MCI individuals might recruit additional neural resources to maintain cognitive function, increasing connectivity (Hillary, Roman et al. 2015). Moreover, disruptions in the DMN have been associated with cognitive decline and MCI.
In the task state, the healthy group demonstrated significantly lower FC values for the ROL - LMC, ROL - RPFC, RMC - LMC, and LMC - LPFC connections compared to the MCI group. This finding could be related to a more efficient neural network organization in the healthy group, as higher connectivity does not always indicate better cognitive performance. In fact, research has shown that greater connectivity might represent a less efficient use of neural resources, whereas lower connectivity can be associated with more efficient information processing (Meunier, Achard et al. 2009). More- over, the lower FC values in the healthy group’s task state may indicate better allocation of neural resources to task-specific regions while suppressing irrel- evant connections (Kelly, Uddin et al. 2008). This would result in a more streamlined neural network, enabling the healthy group to perform the task more efficiently than the MCI group.
The healthy cohort exhibited notably elevated FC metrics in the task con- dition compared to the resting condition for multiple connections, including ROL - RMC, ROL - LMC, ROL - RPFC, ROL - LPFC, RMC - LMC, RMC - RPFC, RMC - LPFC, LMC - RPFC, and LMC - LPFC. The increased FC val- ues during the task state for several connections in the healthy group might
be related to efficient information processing, attention, and working mem- ory. These cognitive processes are crucial for successful task performance and have been associated with increased connectivity in healthy individuals (Seeley, Menon et al. 2007).
In contrast, the MCI cohort demonstrated substantially reduced FC met- rics in the task condition relative to the resting condition for several connec- tions, including ROL - RMC, ROL - LMC, ROL - RPFC, ROL - LPFC, LOL - RMC, LOL - LMC, LOL - RPFC, RMC - LMC, RMC - RPFC, RMC - LPFC, LMC - RPFC, LMC - LPFC, and RPFC - LPFC. The MCI group’s reduced FC values during the task state might be due to impaired neural plasticity or dysfunctional recruitment of neural resources. Studies have shown that MCI individuals exhibit decreased connectivity in task-relevant networks (Damoi- seaux, Prater et al. 2012) and have difficulty adapting to cognitive challenges (Gardini, Venneri et al. 2015).
CONCLUSION
Our research underscores the importance of VR-based assessments for early detection of Mild Cognitive Impairment (MCI) by exploring multi-cognitive domain stimulation and its effects on brain activity. Key contributions of our study include: (1) We introduced a VR drumming task that engages multiple cognitive domains, including motor skills, rhythm, and spatial-temporal orien- tation, aiming for a comprehensive stimulation of cognitive functions. (2) To investigate brain activity changes during the VR task, we uti- lized brain activation markers and functional connectivity derived from fNIRS data. (3) We analyzed MCI-related fNIRS parameters, identifying potential indi- cators for MCI assessment and discussing their implications and under- lying causes.
Our findings reveal significant brain activity and connectivity differences between healthy and MCI individuals, highlighting their potential as MCI biomarkers. Despite the study’s limitations, such as a small sample size and a limited range of fNIRS parameters, our research lays a promising groundwork for VR-based MCI assessments. Future studies should expand participant diversity and explore broader data metrics to enhance the reli- ability and generalizability of these findings. Ultimately, our work aims to advance VR task designs for MCI screening, fostering early intervention and better outcomes for those at risk of cognitive decline.
ACKNOWLEDGMENT
This work was supported by the Hong Kong Polytechnic University under Grant P0038827 and Grant P0038933. This study has been granted human ethics approval from the PolyU Institutional Review Board of The Hong Kong Polytechnic University (IRB Reference Number: HSEARS20211117002).
Investigation of Potential fNIRS-Based Biomarkers in Multi-Domain Virtual Reality Tasks 171
REFERENCES
Belleville, S., et al. (2008). “Task switching capacities in persons with Alzheimer’s disease and mild cognitive impairment.” Neuropsychologia 46(8): 2225–2233. Cogné, M., et al. (2017). “The contribution of virtual reality to the diagnosis of
spatial navigation disorders and to the study of the role of navigational aids: A systematic literature review.”Annals of physical and rehabilitation medicine 60(3): 164–176. Damoiseaux, J. S., et al. (2012). “Functional connectivity tracks clinical deterioration in Alzheimer’s disease.” Neurobiology of aging 33(4): 828. e819-828. e830. Doi, T., et al. (2013). “Brain activation during dual-task walking and executive func-
tion among older adults with mild cognitive impairment: A fNIRS study.” Aging clinical and experimental research 25: 539–544. Fishburn, F. A., et al. (2019). “Temporal derivative distribution repair (TDDR): A motion correction method for fNIRS.” Neuroimage 184: 171-179. Gardini, S., et al. (2015). “Increased functional connectivity in the default mode
network in mild cognitive impairment: A maladaptive compensatory mechanism associated with poor semantic memory performance.” Journal of Alzheimer’s Disease 45(2): 457–470. Hillary, F. G. and J. H. Grafman (2017). “Injured brains and adaptive networks: The benefits and costs of hyperconnectivity.” Trends in cognitive sciences 21(5): 385–401. Hillary, F. G., et al. (2015). “Hyperconnectivity is a fundamental response to
neurological disruption.” Neuropsychology 29(1): 59. Hou, X., et al. (2021). “NIRS-KIT: A MATLAB toolbox for both resting-state and task fNIRS data analysis.” Neurophotonics 8(1): 010802-010802. Kasper, S., et al. (2020). “Management of mild cognitive impairment (MCI): The
need for national and international guidelines.” The World Journal of Biological Psychiatry 21(8): 579–594. Kelly, A. C., et al. (2008). “Competition between functional brain networks mediates behavioral variability.” Neuroimage 39(1): 527–537. Li, Q., et al. (2021). “A human-centred approach based on functional near-infrared
spectroscopy for adaptive decision-making in the air traffic control environment: A case study.” Advanced Engineering Informatics 49: 101325. Mancioppi, G., et al. (2019). “Novel technological solutions for assessment, treatment, and assistance in mild cognitive impairment.” Frontiers in Neuroinformatics 13: 58. Meunier, D., et al. (2009). “Age-related changes in modular organization of human
brain functional networks.” Neuroimage 44(3): 715–723. Miller, E. K. and J. D. Cohen (2001). “An integrative theory of prefrontal cortex function.” Annual review of neuroscience 24(1): 167–202. Park, D. C. and P. Reuter-Lorenz (2009). “The adaptive brain: aging and neurocog-
nitive scaffolding.” Annual review of psychology 60: 173–196. Seeley, W. W., et al. (2007). “Dissociable intrinsic connectivity networks for salience processing and executive control.” Journal of Neuroscience 27(9): 2349–2356. Serino, S., et al. (2015). “Detecting early egocentric and allocentric impairments
deficits in Alzheimer’s disease: An experimental study with virtual reality.” Frontiers in Aging Neuroscience 7: 88. Weintraub, S., et al. (2009). “The Alzheimer’s disease centers’ uniform data set (UDS): The neuropsychological test battery.” Alzheimer disease and associated disorders 23(2): 91. Yoo, S.-H., et al. (2020). “Diagnosis of mild cognitive impairment using cogni-
tive tasks: A functional near-infrared spectroscopy study.” Current Alzheimer Research 17(13): 1145–1160. Yu, S., et al. (2020). Multi-scale enhanced graph convolutional network for early mild cognitive impairment detection. Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part VII 23, Springer.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Accessibility, Assistive Technology and Digital Environments
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-97-1
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional