Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Development of a Home-based Augmented Reality Rehabilitation System for the Elderly with Disabilities

Department of Biomedical Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong SAR, China. ↗
Publication year
Digital Object Identifier10.54941/ahfe1003462

Abstract

Ageing is accelerating rapidly worldwide. Rehabilitation services are important to maintain the quality of life of the elderly. Clinical therapists share heavy workloads to deliver professional rehabilitation services to the elderly people. Home-based rehabilitation systems are proposed to provide professional training with more flexibility at a convenient time slot and venue. However, most existing home-based rehabilitation systems only provide game for fun or a video-conference platform to have one-to-one training with a therapist. In this study, we designed a home-based augmented reality (AR) rehabilitation system which integrates more than 45 professional rehabilitation training exercises designed by physical therapists and occupational therapists, combined with real-time AR guidance to provide feedback to users’ motions. The new platform does not require the therapist to be online during the time of training and it provides real-time guidance based on users’ 3D body segment motion. A pilot trial was conducted by recruiting 10 elderly subjects with disability to receive a 20 sessions of rehabilitation training using the AR rehabilitation training system. The AR rehabilitation training system can provide an objective and comprehensive performance report for users after each exercise session, which include the score of the corresponding exercise, the curves and the measurement of the significant biomechanics data. The results showed that the system could effectively improve the joint movement and body balance. A questionnaire survey was conducted among all the subjects after they finished the training, the results showed that they were very satisfied with the AR rehabilitation system. This study demonstrates that a home-based AR rehabilitation system has the potential to be applied in clinical application to support the elderly people to improve their physical dysfunction and maintain their quality of life.

INTRODUCTION

Population ageing is accelerating rapidly worldwide. By 2030, 1 in 6 peo- ple in the world will be aged 60 years or over (Ageing and health, 2022). A large proportion of the elderly are lived with some chronic diseases and disabilities, which request rehabilitation services to maintain their quality

© 2023. Published by AHFE Open Access. All rights reserved. 111

of life (Fong, 2019). All countries in the world are facing major chal- lenges in providing adequate rehabilitation resources for the older people (Hughes, 2020).

Home-based rehabilitation system is proposed as a promising method to alleviate the shortage of offline elderly rehabilitation resources. Telerehabili- tation, robotic devices and virtual reality are the latest technologies designed to assist the elderly to conduct rehabilitation training at home (Chen et al., 2019). Telerehabilitation is usually achieved through video conferencing with a therapist to observe the patient’s movements as they perform rehabilitation tasks (Rintala et al., 2019)(Ganesan et al., 2021). The limitations of telereh- abilitation are first, it requires the participation of the therapist in the whole training process. Second, major policies for home-based telerehabilitation including, for example, cost, privacy, liability, and system security are not well defined and understood (Wolf et al., 2015). The robotic devices mainly assist the movement of body segments to improve the joint mobility (Cherry et al., 2015). In robot-assisted rehabilitation training sessions, patients can perform simple traditional joint training exercises or interact with games for motivating. However, there can be safety concerns when using robotic devices in the unsupervised home, as robots may generate uncontrolled large forces at times (Li et al., 2022). Moreover, researchers have proposed robot-assisted therapy may serve as an adjunct to, but not an effective replacement for, tra- ditional therapy (Qassim and Wan Hasan, 2020). Virtual reality devices are usually designed as interactive games or virtual exercises. Patients can enjoy a safe and controlled virtual training environment at home. However, the cli- nical outcome of solely using virtual reality devices for rehabilitation remains to be verified.

In addition to the limitations mentioned, the features that are of greatest concern to home users are not well covered by one of the existing home- based rehabilitation technologies: to follow the training exercises from their therapists to have a better recovery; to know that they are doing the exercises in the right way in real time to avoid mistakes and unnecessary injuries; and to get their performance results after each training session to stay motivated (McClincy et al., 2021).

In this study, we designed a low-cost and easy-set-up home-based augmen- ted reality (AR) rehabilitation training system for the elderly with disabilities and chronic diseases. The system integrates more than 45 professional reha- bilitation training exercises designed by therapists, combined with real-time AR guidance to provide feedback to patients’ motions. After finishing trai- ning, users can get a performance report of each exercise to know their training results and progress. Unlike virtual reality devices, the system is designed strongly based on traditional training exercises and experiences, which can better meet the needs of patients. Different from telerehabi- litation, the system does not require the real-time participation of the- rapists and is easy to use. Moreover, there is no need for any physical equipment attached to the body and they can conduct training at sea- ted posture if they have the risk of fall, so this can ensure their safety during training.

Development of a Home-Based Augmented Reality Rehabilitation System 113

METHOD

System Design and Development We designed and developed a home-based AR rehabilitation training system, which is composed of a TV screen, an RGB-D camera (Microsoft Kinect v2) and a personal computer (see Figure 1(a)). The elderly can use this system to carry out professional rehabilitation training program tailored by therapists at home. To use the system, the user stands/sits 1.5 meters in front of the TV screen and follows the live standard exercise video. At the same time, the Kinect captures the movement of the user and displays it on the TV screen, and AR graphics with sound effects are also superimposed on the screen in real time to guide the user to perform the exercise correctly (see Figure 1(b)). After the user completes the training program, an intelligent performance report will be displayed to inform the user of their performance results.

Specially-Designed Exercise Program The home users would like to follow the training exercises from their the- rapists to have a better recovery and this function was realized in the AR rehabilitation training system. More than 45 exercises, covering upper limb, lower limb, coordination and balance training, specially designed by experi- enced physiotherapists and occupational therapists are incorporated into the system as an exercise library (see Figure 2(a)). Configuration functions were designed for customizing individualized training program for different users, including selecting suitable exercises for the user from the exercise library, specifying the number of repetitions for each exercise, and setting the training sequence of the selected exercises (see Figure 2(a)).

Real-Time AR Guidance To guide the users to perform the exercises correctly, AR graphics with sound effects were designed for each exercise. The AR guidance is based on the Kinect depth sensor combined with the-state-of-the-art 3D body tracking technology and artificial technology. When users stand in front of the TV screen and do not perform exercises, they can see themselves together with

Figure 1: Home-based AR rehabilitation system ((a) system view (b) a subject was performing the exercise using the system).
Original PDF Figure 1:
Figure 2: Details of the AR rehabilitation system ((a) exercise program design (b) AR guidance (c) performance report).
Original PDF Figure 2:

a whole-body AR skeleton model on the screen. During the training, the whole-body skeleton model will disappear while specially designed AR gra- phics with sound effects for the exercise will be overlapped and displayed in real time to guide and highlight the important movement features of the exercise (see Figure 2(b)).

Intelligent Performance Report The AR rehabilitation training system can provide an objective and com- prehensive performance report for users after each exercise session, which include the score of the corresponding exercise, the curves and the measu- rement of the significant biomechanics data, and the 3D skeleton animation (see Figure 2(c)). The biomechanics data, including angles of joint movement, the positions of the centre of mass (COM), reaching distance etc., were calcu- lated based on the 3D skeleton joint positions tracked by the Kinect sensor, human body modelling algorithm and inverse kinematic algorithms (Junata et al., 2021) (Yang et al., 2022) (Yeung et al., 2014) (Yeung et al., 2021). Data network services and a shared cloud database were designed and imple- mented to support the data storage and transmission. Users can review their historical performance reports at any time after they have completed an exe- rcise session. If they want, the performance reports can also be provided to the therapists to monitor their progress and modify their exercise program accordingly.

Testing Protocol A pilot trial was conducted to investigate the acceptance and efficacy of the home-based AR rehabilitation training system. The participants were required to receive a 20 sessions of rehabilitation training using the AR reh- abilitation system (40 minutes/session, 2–4 sessions/week), a total of 800

Development of a Home-Based Augmented Reality Rehabilitation System 115

minutes. For each subject, the exercise program was designed by a clinical therapist based on his/her evaluation of this subject. The therapist selected exercises from the exercise library, set the sequence and repetitions of the exercises. The research team set up the system in the subject’s home and trai- ned them how to use it. Kinematic data produced during the training process, including the elbow, shoulder, knee and hip joint angles, as well as the positi- ons of COM were recorded in real-time by the system and stored in the cloud database. A questionnaire survey was conducted among all the participants after they finished their training, The questions were rated on a five-point Likert scale. The grade was from “Strongly disagree” to “Strongly agree”.

Recruitment Ten elderly people were recruited in this study. The subjects were eligible if they meet the inclusion criteria: 1) aged 60 or over; 2) has motor dysfunction in upper-limb, lower-limb, balance or coordination; 3) is able to understand the purpose of the study, follow the instructions and use the system. This study has been registered at https://clinical trials.gov (NCT05627050). All subjects signed a consent form before the experiment, which declared the details of the study and the collection and use of data.

Results and Discussion

All subjects (5 females and 5 males, aged 63.8±3.09) finished more than 800 minutes training in their homes by the end of the study. No adverse effects were observed during the training. All subjects completed the questionnaire survey after they finished their last day of training.

Kinematic Data The kinematic data, including joint motion angles and COM positions, were exported from the database and analysed. We used the median filter to filter the noise and smooth the data. The results showed that the system could accurately capture the motion of joints (see Figure 3(a)).

Figure 3: Data preprocessing ((a) data filtering (b) peaks and troughs extracting).
Original PDF Figure 3:

To observe the motor function improvements in subjects, for each subject, we analysed the changes of the range of motion (ROM) of the joints involved in the training programme during the 800-minute training period. Eight sub- jects performed the elbow flexion and extension exercise, six did the shoulder flexion exercise, nine conducted the hip abduction exercise and eight perfor- med the knee flexion exercise. First, we divided the data of 800 minutes into every 100 minutes. Then within each 100 minutes, we extracted the peaks and troughs (see Figure 3(b)) to obtain the ROM of the joints.

The ROM of elbow flexion and extension is computed by subtracting troughs (maximum elbow flexion angle) from the peaks (maximum elbow extension angle). Figure 4(a) shows the changes of ROM of elbow flexion and extension in the subjects over the training period. Overall, after the trai- ning, there are improvements in the ROM of elbow flexion and extension in subjects. For subject 5 and subject 7, the ROM has been increased by nearly 15 degrees. The ROM of shoulder flexion is the peaks (maximum shoulder flexion angle). The changes of ROM of shoulder flexion in the subjects were shown in Figure 4 (b). On the whole, no significant improvement was observed. But subject 5 has an improvement in the ROM of nearly 20 degrees.

The ROM of hip abduction and knee flexion are also the peaks, which is the maximum hip abduction angle and the maximum knee flexion angle respectively. The changes are shown in Figure 5. The improvement in the

Figure 4: The changes of ROM of the upper limb joints((a) the ROM of elbow flexion and extension (b) the ROM of shoulder flexion).
Original PDF Figure 4:
Figure 5: The changes of ROM of the lower limb joints((a) the ROM of hip abduction (b) the ROM of knee flexion).
Original PDF Figure 5:

Development of a Home-Based Augmented Reality Rehabilitation System 117

ROM of knee flexion is quite impressing while no significant improvement was found in the ROM of hip abduction. Specially, subject 5 and subject 9 have more than 15 degree’s improvements in the ROM of knee flexion after the training.

The four exercises of elbow flexion and extension, shoulder flexion, hip flexion, and knee flexion all highlight the joint movements as the AR guidance. However, compared to the AR guidance for shoulder and hip movements, the guidance for elbow and knee flexion is more intuitive (see Figure 6), which may have somewhat contributed to differences in training effects between these exercises.

Among the ten subjects, two have balance dysfunctions and they recei- ved balance training in the study period. We analysed the changes of the COM positions of these two subjects over the training period and the results (see Figure 7) showed that the COM positions gradually contracted to a smaller area, which indicated that their balance functions were gradually improved during the training.

Questionnaire Survey

Table 1 shows the questionnaire survey results. The subjects showed high satisfactions towards the home-based AR rehabilitation training system. 90% felt that they had functional improvements in at least one of the three areas (upper limb, lower limb and balance). Five out of the ten subjects felt that they had upper limb functional improvements, Six out of the ten subjects felt

Figure 6: AR guidance of exercises ((a). elbow and knee exercises (b). shoulder and hip exercises).
Original PDF Figure 6:
Figure 7: The changes of the COM positions ((a). subject 2 (b). subject 3).
Original PDF Figure 7:
Table 1. Questionnaire survey results.

Table available in the original PDF paper.

Mathematical equation reproduced from the original PDF
Equation reproduced from the original PDF.

4.5 5 90%

Mean Score Median Score Positive (score>4) Satisfied with the device 4.4 4.5 90% Satisfied with the user interface

4.3 4 90%

Satisfied with the training experience

3.5 3.5 50%

Improved upper limb function

3.7 4 60%

Improved lower limb function

5 5 100%

Improved balance function

4.8 5 100%

Would like to continue to use this device at home

4.6 5 100%

Would like to recommend this device to other people

that they had lower limb function improvements and the two subjects with balance dysfunctions both felt that their balance function had improved.

Interestingly, we found similar problems in the questionnaire survey results as reflected in the kinematic data. In the question of whether the system hel- ped improve the upper limb and lower function, 50% of the subjects gave the positive answer. For the users, the direct feeling comes from the interaction with the AR guidance, dissatisfaction indicates that we need to continue to improve our AR graphics, such as incorporating user suggestions into the future AR graphic design.

CONCLUSION

This study designed and developed an easy-set-up home-based AR rehabi- litation training system for the elderly with disabilities. This system covers the essentials that home rehabilitation users are most concerned about, including the therapist-design exercise program, real-time AR motion gui- dance and intelligent performance report. The pilot trial conducted among ten elderly people demonstrated high acceptance of the system and veri- fied that the system helped improve the users’ joint mobility and bala- nce after an 800-minute training. A further study with more subjects should be conducted to evaluate the feasibility and effectiveness of the system.

ACKNOWLEDGMENT

This study is supported by the Hong Kong Innovation and Technology Fund (SST/105/20GP). We are grateful for the support from therapists and participants in this study.

Development of a Home-Based Augmented Reality Rehabilitation System 119

REFERENCES

Aging and Health. (Oct 1, 2022). World Health Organization Website: https://www.

who.int/news-room/fact-sheets/detail/ageing-and-health Chen, Y, Abel, K. T, Janecek, J. T, Chen, Y, Zheng, K, and Cramer, S. C. (2019). Homebased technologies for stroke rehabilitation: A systematic review. International journal of medical informatics, 123, 11–22. Cherry, C. O, Chumbler, N. R, Richards K. (2015). Expanding stroke telerehabilita-

tion services to rural veterans: a qualitative study on patient experiences using the robotic stroke therapy delivery and monitoring system program, Disabil. Rehabil. Assist. Technol, 3107, 1–7 Fong, J. H. (2019). Disability incidence and functional decline among older adults with major chronic diseases. BMC geriatrics, 19(1), 1–9. Ganesan, B, Fong, K. N. K, Meena, S. K, Prasad, P, & Tong, R. K. Y. (2021). Impact

of COVID-19 pandemic lockdown on occupational therapy practice and use of telerehabilitation–A cross sectional study. Eur Rev Med Pharmacol Sci, 25(9), 3614–3622. Hughes, M. (2020). Older people, ageing and social work: Knowledge for practice.

Routledge. Junata, M, Cheng, K. C. C, Man, H. S, Lai, C. W. K, Soo, Y. O. Y, & Tong, R. K.

Y. (2021). Kinect-based rapid movement training to improve balance recovery for stroke fall prevention: a randomized controlled trial. Journal of NeuroEngineering and Rehabilitation, 18(1), 1–12. Li, L, Fu, Q, Tyson, S, Preston, N, and Weightman, A. (2022). A scoping review of design requirements for a home-based upper limb rehabilitation robot for stroke. Topics in Stroke Rehabilitation, 29(6), 449–463. McClincy, M, Seabol, L. G, Riffitts, M, Ruh, E, Novak, N. E, Wasilko, R, and Bell, K.

M. (2021). Perspectives on the gamification of an interactive health technology for postoperative rehabilitation of pediatric anterior cruciate ligament reconstruction: user-centered design approach. JMIR Serious Games, 9(3), e27195. Qassim, H. M and Wan Hasan, W. Z. (2020). A review on upper limb rehabilitation robots. Applied Sciences, 10(19), 6976. Rintala, A, Päivärinne, V., Hakala, S, Paltamaa, J, Heinonen, A, Karvanen, J, and

Sjögren, T. (2019). Effectiveness of technology-based distance physical rehabili- tation interventions for improving physical functioning in stroke: a systematic review and meta-analysis of randomized controlled trials. Archives of physical medicine and rehabilitation, 100(7), 1339–1358. Wolf SL, Sahu K, Bay RC. (2015). The HAAPI (Home Arm Assistance Progression Initiative) Trial: A Novel Robotics Delivery Approach in Stroke Rehabilita- tion. Neurorehabilitation and Neural Repair, 29(10): 958–968. Yang, Z. Q, Du, D, Wei, X. Y, & Tong, R. K. Y. (2022). Augmented Reality

For Stroke Rehabilitation During COVID-19. Journal of NeuroEngineering and Rehabilitation 19, 136 Yeung, L. F, Cheng, K. C, Fong, C. H, Lee, W. C, & Tong, K. Y. (2014). Evaluation of the Microsoft Kinect as a clinical assessment tool of body sway. Gait & posture, 40(4), 532–538. Yeung, L. F, Yang, Z, Cheng, K. C. C, Du, D, & Tong, R. K. Y. (2021). Effects of

camera viewing angles on tracking kinematic gait patterns using Azure Kinect, Kinect v2 and Orbbec Astra Pro v2. Gait & posture, 87, 19–26.

Cover of the proceedings volume Health Informatics and Biomedical Engineering Applications

Explore the complete digital volume

Health Informatics and Biomedical Engineering Applications

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-54-4

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →