Yehua Shi, Xiaoyi Wang, Cathy Lau, Kai-Yu Tong (2023). Is it possible to use Kinect sensor for lying position rehabilitation exercise? Kinect V2 versus Azure Kinect. In: Health Informatics and Biomedical Engineering Applications. AHFE Open Access, vol. 78, pp. 72–79. https://doi.org/10.54941/ahfe1003457
The availability of low-cost portable depth sensor camera brings opportunity to be applied in home-based rehabilitation exercise for stroke and other chronic disease patients. Kinect V2 seemed not feasible to easily track motion in a lying position, while the latest Microsoft Azure Kinect has improved the sensor. This paper experimentally explores the feasibility of Azure Kinect and Kinect V2 for lying position rehabilitation exercises and evaluate the tracking performance by changing the camera viewing angles. Two healthy subjects performed upper and lower limb rehabilitation exercise trial on the bed according to supine position and lateral position. The Kinect sensor was tested at 6 viewing angles in human body coronal plane and sagittal plane. Subject motion data and video were recorded and evaluated by two Kinect camera systems. The results showed that the hardware improvement such as resolution enhancement and the neural network motion tracking algorithm of the Azure Kinect depth camera led to higher performance in lying body motion recognition than Kinect v2 for most of the viewing angles. In conclusion, Azure Kinect could improve the lying position body tracking accuracy and it has great potential in the field of rehabilitation with lying position exercises.
In the early stages of stroke, rehabilitation begins with motor exercises of limbs, and patients are required to perform task-oriented training. For some stroke patients who cannot sit or stand for a long time, lying position rehabilitation exercises are necessary. The availability of low-cost porta- ble depth sensor camera brings opportunity to be applied in home-based rehabilitation exercise for stroke and other chronic disease patients. In the past ten years, several depth sensing modules based on vision systems have been proposed. The mainstream technology routes include Structured Light (Microsoft Kinect v1), Active Stereovision (Intel RealSense series), Time-of- Flight (Microsoft Kinect v2, iPad Pro, iPhone). By using the manufacturer’s skeletal tracking software solution or other available human pose estimation
Is It Possible to Use Kinect Sensor for Lying Position Rehabilitation Exercise? 73
methods, people are able to conduct human body tracking test, while the recognition accuracy vary widely.
Kinect V2 performed well with subjects in standing and seat posture and can be competent as a clinical assessment tool of body sway (Yeung et al., 2014), but it supports poorly for tracking motion in a lying position. The new generation Microsoft Azure Kinect has promoted the resolution and offers significantly higher accuracy than other commercially available cameras (Kramer et al., 2020). The development of Deep Learning and Convolutio- nal Neural Networks implemented in the Azure Kinect Body Tracking SDK, has further improved the results in terms of reliability of human body skele- ton recognition, even adding more details, such as face key points4 (Romeo et al., 2021). From the recent research, the comparison between Azure Kinect and Kinect V2 mainly focuses on the recognition accuracy of posture and gait. Albert et al. provided the evaluation of human pose tracking perfor- mance of the Azure Kinect and Kinect v2 for gait analysis (Albert et al., 2020). Tölgyessy et al. compared the skeleton tracking accuracy and preci- sion of Kinect V1, Kinect V2, and the Azure Kinect (Tölgyessy et al., 2021). There is a lack of the analysis in lying position body movement recognition.
This study was designed to evaluate the feasibility of using the Azure Kinect and Kinect V2 to track the body movement in the lying position on bed for rehabilitation training. We evaluated the tracking performance with different camera viewing angles by comparing human skeleton tracking points. This paper first explained the hardware/software setup and experi- mental design. Two subjects were asked to perform multiple rehabilitation movements according to 6 angles and then all data has been recorded and compared in the system. We discuss our results and draw the conclusion in the end.
METHOD
Two healthy subjects were required to perform a total set of upper and lower limb rehabilitation training trial, including shoulder abduction/adduction, shoulder flexion/extension for upper limb training; hip abduction/adduction on supine position and leg crossing exercises on lateral position for lower limb training) on the bed. The informed consent was given by the participants for scientific research purposes.
The height of bed & camera and the horizontal distance between the camera and the centre the human body is 0.3m, 1.5m, 1.2m respectively. The Kinect sensor was tested at 45
(default) viewing angles in human body sagittal plane (which 90
was viewing from the vertical level direction to the ground), with 5 viewing angles in human body coronal plane (0
, 45
, 90
, 135
, 180
for Angle 1 - 5, which 90
viewing angle was perpendicular to the body sagittal plane). Then, the camera is set perpendicular to the coro- nal plane - Angle 6 (which the camera lens is placed vertically facing the ground). The experiment stage and Kinect camera (Fig. 1&2) for the image and data acquisition was configured by allowing the best integration between the systems. As the light in the Kinect area was considered sufficient (Mauro et al., 2021). Default experiments are not affected by lighting conditions.
Figure 1: The layout of the experiment ((a) Testing stage (b) Two kinds of Kinect cameras).Figure 2: Viewing planes of Kinect camera (a) viewing from human coronal plane (b) viewing from human sagittal plane.
The subjects were recorded by two Kinect camera sensor systems. The sof- tware tools for storing the skeleton image and kinematic data of Azure Kinect & Kinect V2 was based on the Azure Kinect body tracking SDK & Kinect for Windows 2.0 SDK and was implement using the C# language tailor-made codes for this experiment. The laptop used to run Azure Kinect body tracking SDK was equipped with an Nvidia GeForce RTX 3050 GPU. The target sam- pling rate of the Kinect v2 and Azure Kinect was 30 Hz. We chose to record the coordinates of the elbow and knee joints because the elbow and knee joints were considered as the centre movement points of upper and lower limb respectively, which can reduce the error while retaining the motion track. The tracking performance was evaluated by comparing the elbow and knee’s X, Y coordinates, motion trajectories and captured human skeleton model images.
Is It Possible to Use Kinect Sensor for Lying Position Rehabilitation Exercise? 75
Figure 3: Human body framework from viewing angle (0◦-180◦) in body coronal plane, Azure Kinect (Left), Kinect V2 (Right) ((a) 0◦Viewing angle (b) 45◦Viewing angle (c) 90◦Viewing angle (d) Viewing angle - perpendicular to the ground).
RESULT
For each subject, they were asked to perform lying upper body and lower body movements within half a minute respectively. The recorded movement point of the upper body is the elbow joint (left), and the recorded movement point of the lower body is the knee joint (left). We preliminarily verified the possibility of Azure Kinect and Kinect V2 in the lying down rehabilitation by comparing the kinematic data (in Fig. 4).
After analysing the 2D motion coordinate trajectory of joint points and the X-axis and Y-axis coordinate motion cycles, we evaluated the performance of Azure Kinect and Kinect V2 for each viewing angle and summarized it in Table 1.
In conclusion, Azure Kinect is stable in most of the angle scenes (stable in 10/12 scenes and the trajectory has tracking problems only when in the lower limbs with 135
s viewing angle and the upper limbs with 45
). Kinect’s performance in the half scene (7/12) is relatively stable, but it shows a larger error compared to the Azure Kinect tracking trajectory.
Among the 5 angles comparisons (0
/45
/90
/135
/180
) of Azure Kinect and Kinect V2, the performance of 0
, 90
and 180
angle of view was
(a)
(b)
Figure 4: Continued relatively better than other angles. In addition, in experiments at 45◦s and 135◦s viewing angle, both devices were not able to keep long-term sta- ble state, we find that upper limbs or lower limbs joint nodes are often incorrectly recognized, we should be cautious when choosing these angles for rehabilitation training.
One of the findings we got from the experiment is that both devices show extremely strong tracking performance at the angle - perpendicular to the ground (Also have the highest average normal tracking time ratio: AK: 96.7%, V2: 88.9% in Table 1.)
Is It Possible to Use Kinect Sensor for Lying Position Rehabilitation Exercise? 77
(c)
Figure 4: The 2D motion coordinate trajectory of joint points ((a)0◦and 45◦viewing angle of upper limb and lower limb (b) 90◦and 135◦viewing angle of upper limb and lower limb (c) 180◦and Angle 6 of upper limb and lower limb).Table 1. Azure Kinect and Kinect V2 lying position human body tracking performance and average normal tracking time ratio (Performance Rate: Excellent > Well > Average > Bad).
Table available in the original PDF paper.
Performance rate / Viewing angle
Azure Kinect Relative Performance
Kinect V2 Relative Performance f
Average normal tracking time of joint points (%) during whole movement period (Kinect V2)
Average normal tracking time of joint points (%) during whole movement period (Azure Kinect)
Excellent Excellent 96.7% 88.9%
0
Well Average 87.0% 25.3% 45
Well Bad 54.5% 33.9% 90
Well Well 85.1% 67.6% 135
Average Bad 57.8% 32.7% 180
Well Average 81.2% 28.1% Angle 6 (perpendicular to ground)
In the comparison of the human frame between Kinect V2 and Azure Kinect in Fig. 3. The picture shows that the skeleton frame sometimes drifts much and cause serious recognition error, often in Kinect V2. And this is a potential flaw of the Kinect V2 using in lying rehabilitation.
In summary, it is often difficult to install a camera vertically looking down at the human body in space. If the conditions permit, perpendicular to the
ground is indeed the best angle for the lying test of Azure Kinect. The results showed that the hardware improvement such as resolution enhancement and the neural network motion tracking algorithm of the Azure Kinect depth camera led to higher performance and accuracy in lying body motion reco- gnition than Kinect v2 in most of the viewing angles. We also found Kinect was able to identify human skeleton with a more complete body framework during test.
CONCLUSION
To sum up, the feasibility of Azure Kinect and Kinect V2 for lying posi- tion rehabilitation exercise has been analysed and the best viewing angle suggestion was given in this paper. For the training of reclining rehabilitation movements using the Kinect sensor, we first recommend the vertical viewing angle (Angle 6). Under this viewing angle, both Kinect devices can complete the whole process of human body tracking well. If the conditions are limited, we can consider 0
/90
/180
for Azure Kinect instead. Although the tracking performance is reduced, but still usable referring to Table 1. And we do not recommend camera viewing angles lower than the well performance.
Our analysis showed that Kinect V2 performance was relatively not enough in the experiment, but the lying position could be tracked by Azure Kinect in most time and it has great potential in the field of rehabilitation with lying position exercises.
Future work will focus on conduct the experiment with more subjects and will quantitatively compare the motion coordinates of more joint points of Kinect V2 and Azure Kinect against the Vicon reference on the joint angle time series (Yeung et al., 2021) to draw further conclusions.
ACKNOWLEDGMENT
This study is supported by The Social Innovation and Entrepreneurship Development Fund (KPF22GWP34). We are grateful to all members who participated in or contributed to this research.
REFERENCES
Albert, J. A., Owolabi, V., Gebel, A., Brahms, C. M., Granacher, U., & Arnrich, B. (2020). Evaluation of the Pose Tracking Performance of the Azure Kinect and Kinect v2 for Gait Analysis in Comparison with a Gold Standard: A Pilot Study. Sensors, 20(18), 5104 Antico, M., Balletti, N., Laudato, G., Lazich, A., Notarantonio, M., Oliveto, R.,...
& Simeone, J. (2021). Postural control assessment via Microsoft Azure Kinect DK: An evaluation study. Computer Methods and Programs in Biomedicine, 209, 106324. Kramer, J. B., Sabalka, L., Rush, B., Jones, K., & Nolte, T. (2020). Automated depth video monitoring for fall reduction: A case study. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (pp. 294–295). Romeo, L., Marani, R., Malosio, M., Perri, A. G., & D’Orazio, T. (2021, June). Per-
formance analysis of body tracking with the microsoft azure kinect. In 2021 29th
Is It Possible to Use Kinect Sensor for Lying Position Rehabilitation Exercise? 79
Mediterranean Conference on Control and Automation (MED) (pp. 572–577). IEEE Tölgyessy, M., Dekan, M., & Chovanec, ˇL. (2021). Skeleton tracking accuracy and precision evaluation of Kinect V1, Kinect V2, and the azure kinect. Applied Sciences, 11(12), 5756. Yeung, L. F., Cheng, K. C., Fong, C. H., Lee, W. C., & Tong, K. Y. (2014). Evalu-
ation of the Microsoft Kinect as a clinical assessment tool of body sway. Gait & posture, 40(4), 532–538. Yeung, L. F., Yang, Z., Cheng, K. C., Du, D., & Tong, R. K. (2021). Effects of camera viewing angles on tracking kinematic gait patterns using Azure Kinect, Kinect v2 and Orbbec Astra Pro v2. Gait & posture, 87, 19–26.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Health Informatics and Biomedical Engineering Applications
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-54-4
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional