Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Analysis of Stair-Ascent Activities with Handrail Use in Daily Living Space and Motion Features using RGBD Camera

1 Tokyo Institute of Technology, Tokyo, 152-8552, Japan ↗
2 National Institute of Advanced Industrial Science and Technology, Tokyo, Japan ↗
Publication year
Digital Object Identifier10.54941/ahfe1004353

Abstract

The geriatric population has increased worldwide over the past few decades. Older adults rarely make a sudden transition from a healthy state to a state requiring nursing care; more often they transition through an intermediate stage called frailty. To assess frailty quantitatively using ambient sensing technology, our group developed a system to automatically and continuously measure and analyze human ascent and descent motions and handrail-use behaviors in homes, using an RGBD camera. This study developed a whole-body motion feature analysis method using principal component analysis (PCA), and analyzed the features of whole-body motions related to handrail use when ascending stairs. Daily stair-ascent motion was measured in two houses, with two participants in their 20s and two in their 50s in the first house, and two in their 70s in the second house. A method for extracting the characteristic motion of ascending stairs while using a handrail was developed using principal component analysis of whole-body skeleton data. The results showed that the third principal component was the characteristic motion of holding the handrail. The developed method makes it possible to evaluate dependence on handrails and clarify the characteristics of movements associated with changes in physical function, through continuous measurement and motion feature extraction techniques for daily stair ascent.

INTRODUCTION

The geriatric population has increased worldwide over the past few decades (WHO 2021). Older adults tend to transition from health, through a period of frailty, before reaching a state requiring nursing care (Fried et al., 2001). A self-administered questionnaire was used to assess frailty in older peo- ple (Shinkai et al., 2010). However, this checklist-based frailty assessment requires older adults to complete the questionnaires voluntarily, regularly, and conscientiously, placing an administrative burden on respondents. If daily physical activities like stair ascent and descent could be measured auto- matically and continuously, and the measurement data used for quantitative frailty assessment, this would both reduce the administrative burden and

© 2023. Published by AHFE Open Access. All rights reserved. 8

Analysis of Stair-Ascent Activities With Handrail Use in Daily Living Space 9

support better detection of declines in older adults’ physical function. Early detection of frailty by observation of daily life activities using ambient sens- ing technology is expected to promote and maintain social participation by older people. Therefore, our group developed a system to ambiently, automatically, and continuously measure and analyze human ascent and descent motions and handrail-use behaviors in homes using an RGBD cam- era (Miyazaki et al., 2023). The system is triggered when a person appears within the camera’s angle of view, and then automatically stores the envi- ronmental depth information and the 3D skeletal coordinates of the person. We measured handrail dependency by determining the walking speed and handrail grasping state during stair ascent and descent using a decision- tree algorithm and automatically calculated the handrail grasping points and grasping rate.

However, the three-dimensional whole-body motions during stair ascent with handrails, and the features of these motions that could be evaluated for handrail dependence, remained unknown. Therefore, this study aims to develop a method for whole-body motion-feature analysis using principal component analysis to analyze the features of whole-body motions related to handrail-use behavior when ascending stairs.

Measurement Data of Stair Ascent Activity With Handrails in Daily-Life Spaces A previous study measured the stair ascent and descent data for stairs with handrails in two housing environments (Miyazaki et al., 2023). Environment 1 was a two-story house with four participants: a male in his 20s (p1), a female in her 50s (p2), a male in his 50s (p3), and a female in her 20s (p4). Environment 2 was a two-story house with two participants living in it: a male in his 70s (p5) and a female in her 70s (p6). Of these participants, one male and one female in their 70s were determined to be pre-frail using a con- ventional self-evaluated questionnaire. An Azure Kinect DK (AK) (Microsoft Co.), which automatically stored the three-dimensional ascent or descent motion using handrails, was installed on the staircases in the two environ- ments. In each living environment, the participants ascended and descended stairs naturally. When they entered the camera’s field of view, the device was automatically activated, and their activity (ascent or descent) was cap- tured. The data used in the analysis were 49 cases from p1, 14 cases from p2, 28 cases from p3, 10 cases from p4, five cases from p5, and five cases from p6. In all cases, full-body motion was measured over the whole assess- ment period. In a previous study, the measured handrail-use frequency and points of contact during ascent and descent of stairs were analyzed using a decision-tree algorithm to visualize the frequency of use of the handrail and grabbing points, and to evaluate handrail dependence (Figure 2). As given in Table 1, in Environment 1, the dependence of the participants on the handrails increased with age: the two participants in their 20s (p1 and p3) were the least dependent, the two in their 50s (p2 and p4) were more dependent, and the two pre-frail participants in their 70s (p5 and p6) were highly dependent.

Figure 1: Measurement of stair-ascent motion using Azure Kinect (Miyazaki et al., 2023).
Original PDF Figure 1:
Figure 2: Visualization of handrail grasping position and frequency during stair ascent (Miyazaki et al., 2023).
Original PDF Figure 2:
Table 1. Handrail grabbing ratio when ascending stairs (Miyazaki et al., 2023).

Table available in the original PDF paper.

p1 p2 p3 p4 p5 p6 Ascending 4.6% 25% 2.5% 19.0% 81.6% 76.3%

Feature Extraction Method for Stair Ascent Motion With Handrail by Principal Component Analysis We developed a method to extract the features of stair ascent and descent motions using a principal component analysis method for 3D whole-body motion and analyzed the motion features. The skeletal-position data for 32 3D joint positions were obtained using Azure Kinect. Because this is high-dimensional data (96-dimensional, consisting of 32 joint positions × 3-dimensional coordinates), extracting and analyzing the motion features was challenging. An effective approach to the analysis problems posed by multivariate data, such as whole-body joint motion data, is feature data extraction using principal component and motion analysis. Performing prin- cipal component analysis on the time-history waveform of three-dimensional

Analysis of Stair-Ascent Activities With Handrail Use in Daily Living Space 11

motion data enables whole-body motion to be decomposed into principal- component motions, thus enabling the compression and analysis of the motion features (Ito et al., 2016, Federolf et al., 2013).

First, of the 31 joint position data acquired by the Azure Kinect, 20 joints, excluding NOSE, EAR, EYE, HANDTIP, and THUMB positions, were used. This gave us a total of 60-dimensional time-history data in three-dimensional coordinates. In addition, it was necessary to establish a coordinate sys- tem to compare whole-body motions in different environments. Therefore, using the method developed in a previous study, a staircase coordinate sys- tem was set up with the x-axis in the direction of the longitudinal axis of the handrail using the shape features of the handrail. Subsequently, a three-dimensional coordinate transformation was performed from the cam- era coordinate system to the staircase coordinate system (Miyazaki et al., 2023). Moreover, the coordinate values were normalized from the length between the PELVIS and SPINE_NAVAL positions so that differences in each partici- pant’s height would not affect the results. Principal component analysis was performed on the normalized 60-dimensional time-history data of the joint positions to achieve dimensionality decomposition of the motion data. Factor loadings were calculated for each joint position in the obtained principal com- ponents, and the joints contributing to each principal component of motion were examined. Finally, the motions constituting the principal components were reconstructed, and their characteristics were discussed.

RESULTS

The PCA results of the principal component analysis showed that the cumu- lative contribution of the components up to the third principal compo- nent was approximately 60% for all participants (p1=63.0%, p2=55.4%, p3=64.6%, p4=59.9%, p5=59.8%, and p6=66.2%). Therefore, the fea- tures of the ascent motion were analyzed up to the third principal component. Factor loadings were calculated for the 60-dimensional data of x, y, and z coordinates of the 20 measured joint positions. The factor loadings were divided into four groups, and the sum of the absolute values of the loadings for the joint coordinates belonging to each group is shown in Figure 3.

The four groupings are as follows: Trunk-neck-head is a grouping of Head, Neck, and Spine; Upper limb is a grouping of left and right shoulders and elbows; Hand is a grouping of left and right wrists and hands; and Lower limb is a grouping of left and right hips, knees, and ankles.

First, focusing on the joints that constitute the first principal component of each participant, the Lower limb showed the largest factor loadings for p1 and p4 for a male and female in their 20s, and for p2 and p3 for a male and female in their 50s. However, the factor loadings of the Hand were large in p5 and p6 for the pre-frail participants in their 70s. In the second principal component, the factor loadings for the Hand were large for all participants, and no differences were observed. The Hand had the largest factor loadings for p1, p2, p3, and p4 in the third principal component. Compared to p1 and p4, who were in their 20s, p2 and p3, who were in their 50s, showed higher

hand-factor loadings. In contrast, the factor loadings for the Lower limb were largest for p5 and p6, who were in their 70s.

Figure 3: Comparison of the sum of the absolute value of principal component loading for 1st to 3rd principal component for each participant.
Original PDF Figure 3:

DISCUSSION

During the stair ascent, the Lower limb contributed to the first principal com- ponent in p1–p4 and the Hand contributed to the third principal component. In addition, for the Hand, the factor loadings on the other joints were greater for p2 and p3 (in their 50s) than for p1 and p4 (in their 20s). In contrast, for p5 and p6 (in their 70s) the Hand contributed the most to the first principal

Analysis of Stair-Ascent Activities With Handrail Use in Daily Living Space 13

component, and the Lower limb contributed the most to the third principal component.

Figure 4 shows sequence diagrams of the restored first and third principal component motions of p1, p3, and p6 in their 20s, 50s, and 70s, respectively. Figure 4(a) displays the first principal component motion for p1 in their 20s, while Figure 4(c) shows the same for p3 in their 50s. Both participants dis- play a similar motion pattern: the right leg moves forward, and the left leg moves backward at the beginning. However, as time progresses, the right leg moves backward, and the left leg moves forward. Additionally, there is almost no motion in the hands and trunk except for the legs. This indicates that the first principal component of whole-body motion during stair ascent in p1 and p3 is the motion from which the walking component is extracted. For the third principal component, the Hand contributed the most to both participants’ motions. Therefore, when we observe the principal component motions of p1 in Figure 4(b) and p3 in Figure 4(d), p1 does not show any clear motion features, even after time elapses. In contrast, in the case of p3, the hand motions along the long axis of the handrail were extracted. There- fore, the features of the grasping motion of the handrail appeared in the third principal component.

However, the Hand was dominant in the first principal component motion of the participants in their 70s. The sequences of the principal component motions in Figure 4(e) show that the first principal component motion of p6 differs from those of p1 and p3, in that the hand and upper limbs move along the longitudinal axis of the handrail. In contrast, the third principal component was dominated by the Lower limb, and the motion sequence in Figure 4(f) shows that the left and right legs alternated. Therefore, the contribution of the handrail-grasping motion component increased, and the contribution of the walking motion component decreased in pre-frail participants in their 70s.

Considering the above findings, the results of this study indicate that the contribution of hand motions along the handrail’s longitudinal axis increases from the third principal component to the first principal component in par- ticipants in their 70s who are in a pre-frail state. Furthermore, the results suggest that the contribution of the hand increases more between partici- pants in their 50s and those in their 20s, who are not in a pre-frail state, even within the same principal component. Hand motion along the handrail’s longitudinal axis could be an important feature that expresses the degree of handrail dependence during stair ascent, which is related to the assessment of frailty; it is therefore important to extract and evaluate this motion from multidimensional whole-body motion.

Although the contribution of the Hand to the second principal compo- nent was high, there was no significant difference among the participants. The restoration of the second principal component motions showed that the feature motion was affected by errors, owing to the poor accuracy of joint position prediction when the forearms were hidden by the upper body, and it was therefore excluded from this analysis. In future studies, it will be nec- essary to optimize camera placement to enable the accurate measurement of whole-body motion in daily life. We also plan to increase the measurement environment and number of participants in the future.

Figure 4: Sequential feature motion for the 1st or 3rd principal components for participant 1, 3 and 6.
Original PDF Figure 4:

Analysis of Stair-Ascent Activities With Handrail Use in Daily Living Space 15

CONCLUSION

This study developed a method for extracting characteristic motions in handrail-use behavior while ascending stairs using handrails. Daily stair- ascent motion was measured in two houses, with two participants in their 20s and two in their 50s in the first house, and two participants in their 70s in the second house. The results indicate that the contribution of hand motions along the handrail’s longitudinal axis increases in their 70s who are in a pre-frail state. Therefore, hand motion along the handrail’s lon- gitudinal axis could be an important feature that expresses the degree of handrail dependence during stair ascent, which is related to the assessment of frailty; it is therefore important to extract and evaluate this motion from multidimensional whole-body motion.

REFERENCES

Federolf, P., Roos, L. and Nigg, B. M. (2013) ‘Analysis of the multi-segmental postural movement strategies utilized in bipedal, tandem and one-leg stance as quantified by a principal component decomposition of marker coordinates’, J Biomech, 46(15), 2626–33. Fried, L. P., Tangen, C. M., Walston, J., Newman, A. B., Hirsch, C., Gottdiener, J.,

Seeman, T., Tracy, R., Kop, W. J., Burke, G. and McBurnie, M. A. (2001) ‘Frailty in older adults: evidence for a phenotype’, J Gerontol A Biol Sci Med Sci, 56(3), M146-56. Ito, M., Mano, A., Miyazaki, Y., Watanabe, R. and Nishida, Y. (2016) ‘Extraction and difference quantification of protective motion in case of indoor infant fall behaviors by multivariate analysis’, Journal of Biomechanical Science and Engineering, 11(3), 15–00680-15-00680. Miyazaki, Y., Shoda, K., Kitamura, K. and Nishida, Y. (2023) ‘Assessing Handrail-

Use Behavior during Stair Ascent or Descent Using Ambient Sensing Technology’, Sensors, 23(4), 2236. Shinkai, S., Watanabe, N., Yoshida, H., Fujiwara, Y., Amano, H., Lee, S., NIshi, M.

and Tsuchiya, Y. (2010) ‘Research on screening for frailty: development of “the Kaigo-Yobo Checklist’, Japanese journal of public health, 57(5), 345–354. WHO (2021) ‘Ageing and health’, [online], available: https://www.who.int/newsroom/fact-sheets/detail/ageing-and-health [Accessed October 31 2021].

Cover of the proceedings volume Emerging Technologies in Healthcare and Medicine

Explore the complete digital volume

Emerging Technologies in Healthcare and Medicine

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-92-6

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →