Takumi Ohashi, Honoka Hayashi, Sota Takagi, Yu Miyaji, Ryota Izawa, Miki Saijo (2022). Towards Accident Prevention: An Aspiration Risk Warning System for Older Adults During Meals. In: Human Factors in Accessibility and Assistive Technology. AHFE Open Access, vol. 37, pp. 108–116. https://doi.org/10.54941/ahfe1001649
The aim of this study is to create a system to support cooperative prevention of accidents during mealtime among users of elderly care facilities. Experiments were conducted with 10 healthy adults to define abnormal postures during swallowing and extract responses to abnormal posture warnings by a communication robot. The participants were asked to tilt their bodies at 10° intervals from a comfortable posture and perform a food-intake test in the resulting posture. Based on observation of swallowing and feedback about posture from the participants, the angle at which each posture became dangerous was determined. As a result, abnormal posture thresholds were identified as leaning forward 30°, tilting left/right 30°, and chin tucking 20°. It was determined that the communication robot should not issue a warning in the event of repeated detection of abnormal posture. Moreover, it was considered that the warning alone is insufficient to prevent danger, and that the user and the people around them should be instructed on how to respond in the event of a warning. If the system is used in a care facility, the warning should be communicated to users and caregivers, facilitating cooperative accident prevention.
Machine learning has been widely applied to assistive technologies for older adults to resolve the severe workforce shortage at care facilities. Numerous studies have proposed real-time posture estimation models with simple neu- ral network classifiers for fall detection (e.g., Wang et al., 2008; De Miguel et al., 2017; Asif et al., 2020; Serpa et al., 2020). However, such intervention has mainly focused on diagnosing and treating fall risks or detecting falls, and alerting caregivers. Few studies have tried to directly detect signs of an accident in real time, notify stakeholders of such, and encourage prevention behavior. Especially in elderly care, pulmonary aspiration due to poor posture during meals is a significant problem (Care Work Foundation, 2018). Instead of fall detection, it is thus necessary to detect abnormal postures that pose a risk of aspiration and encourage prompt accident prevention response. We developed an abnormal posture warning system that can detect the angles
Towards Accident Prevention: An Aspiration Risk Warning System for Older Adults 109
of chin tucking and postures of leaning forward and tilting left/right using Raspberry Pi and PoseNet (Ohashi et al., 2021; Takagi et al., 2021). When an abnormal posture is detected, the system notifies the user and others nearby by voice and the caregiver by text message/image, thereby promoting coo- perative accident prevention. Our previous research examined this concept of cooperative prevention; however, no studies have determined the thresh- olds of abnormal posture angles or notification targets of the system to avoid serious accidents.
In this study, as a preliminary survey for conducting experiments on elderly people, we conducted experiments on 10 healthy young people to 1) define abnormal postures during swallowing and 2) extract responses to warning notifications of abnormal postures. The rationale for the selection of young people is based on the literature showing that when posture control and cognitive tasks are performed simultaneously, younger adults control their posture better and react faster than older people (Huxhold et al., 2006; Laughton et al., 2003; Woollacott and Shumway-Cook, 2002). A meal can be regarded as a dual task consisting of controlling sitting posture and eating. Abnormal posture angles and reactions to alerts are assumed to be shallower and slower/milder, respectively, in older compared with younger people. The findings obtained for young people in this study will thus lead to safe and effective experiments for older people.
RESEARCH METHOD Definition of Abnormal Postures During Swallowing
Postures to be evaluated. The postures that affect swallowing are leaning forward/backward, tilting left/right, tilting forward/backward of the head and neck, and tilting left/right of the head and neck. However, this study examined only the following postures: a) leaning forward; b) tilting left; c) tilting right; and d) tilting forward of the head and neck (chin tucking). The reason is that the first three not only affect swallowing but can also cause a fall from a chair, making it necessary to identify the threshold angle at which these postures become dangerous. By contrast, leaning backward may be supported by a backrest or posture-maintaining cushion. In addition, it has been reported that a sitting position with a 30° or 60° backward tilt using a reclining bed actually facilitates swallowing (Saitoh et al., 1986). Given the above considerations, we evaluated the postural collapse of the trunk in the sitting position for leaning forward and tilting left/right.
Correct head and neck posture has the chin slightly retracted, but it has been pointed out that too much retraction can be dangerous. Therefore, it is necessary to determine the threshold at which the degree of forward leaning becomes dangerous. On the other hand, it is generally known that tilting the head back results in a straight line between the pharynx and trachea, which greatly increases the risk of aspiration. If the head and neck are tilted back not only during a meal but also during other daily activities, saliva containing bacteria in the mouth may flow into the trachea without causing choking over, resulting in aspiration pneumonia. Therefore, the experiment did not include tilting the head and neck back because it is clear that such a posture
Figure 1: Postures for which swallowing was assessed.Table 1. Food Test evaluation criteria.
Table available in the original PDF paper.
Score Criteria 1 No swallowing. Choking over and/or respiratory distress. 2 With swallowing. Respiratory distress (suspicion of subclinical aspiration). 3 Swallowing present. Good respiration. Choking over and/or wet hoarseness. Moderate oral residue. 4 Swallowing present. Good respiration. No choking over. Almost no oral residue. 5 In addition to Point 4, swallowing twice within 30 seconds is possible.
during eating is dangerous. Figure 1 shows the postures for which threshold values are defined in this study.
Research participants. To define abnormal postures during meals, we con- ducted an experiment on swallowing difficulty in 10 healthy adults (male: 8, female: 2, mean age = 22.9 ± 1.9 years old, mean height = 170.3 ± 8.5 cm). In this experiment, the standard posture was “a comfortable posture whereby the subject lightly puts weight on the backrest and relaxes the whole body.” Subjects were asked to tilt at increments of 10 degrees, eat jelly for people with swallowing difficulties at each posture, and evaluate the difficulty of swallowing at each angle using a questionnaire.
Evaluation procedure. To evaluate swallowing, we used the Food Test (Tohara et al., 2002) and a questionnaire on swallowing difficulty that does not require swallowing videofluorography. Table 1 shows the Food Test eva- luation criteria. The Food Test is a method used to evaluate the swallowing function. A score of 3 or less indicates that the participant has difficulty eating and swallowing. The procedure of the experiment is shown in Table 2.
The tilt of the trunk was visually confirmed using the pelvic floor as the base point, and verbal instruction was given to the participants regarding the degree of tilt. The angle was checked by connecting a video camera set up directly in front of and beside the participants to a display, and attach- ing a sheet with a reference line printed every 10° to the display. The tilt of the head and neck was measured using PoseNet (Papandreou et al., 2017), a posture estimation model, as it is not possible to set a base point that does not move like the pelvic floor; instruction was then provided regarding the angle of the tilt. The questionnaire on difficulty with swallowing was admini- stered using a 5-point Likert scale (1: no problem, 2: slightly uncomfortable,
Towards Accident Prevention: An Aspiration Risk Warning System for Older Adults 111
Table 2. Experimental procedure of the food test.
Table available in the original PDF paper.
Step Procedure (i) Instruct the participant to have a “comfortable posture.” (ii) Instruct the participant to tilt 10° toward the experimental posture. (iii) Place about 4 g (a teaspoonful) of jelly on the tongue and instruct the participant to swallow. (iv) After the participant swallows the jelly, ask the participant to repeat swallowing (swallowing saliva) twice. (v) Two authors evaluate the swallowing using the evaluation criteria of the Food Test, and the lowest score is used as the score. (vi) If the Food Test score is 4 or more, repeat (i) to (v) at most twice. (vii) Repeat (i) to (vi), increasing the inclination by 10°. If the Food Test score is 3 or less, or if the author(s) judges that the posture is dangerous for the participant, stop the experiment.
3: uncomfortable, 4: very uncomfortable, 5: unable to swallow). In addition, a free-description column was provided in the questionnaire asking if the participant felt that the posture was dangerous. For the question about swal- lowing difficulty, a score of 3 or more was defined as an abnormal posture. In healthy adults without any problems in eating and swallowing functions, it is rare to feel discomfort during normal swallowing. On the other hand, since discomfort in swallowing is one of the earliest symptoms of a decline in eating and swallowing functions, we adopted the angle at which this occurs as the threshold for abnormal posture that affects eating and swallowing functions.
Experiment environment. Video cameras were set up in three directions: in front of, to the side, and above the head of the participants. The video cameras were used for the operation of PoseNet and to confirm the validity of the Food Test scores. In addition, another video camera was installed to film the entire experiment. This was used to analyze the participants’ responses to a communication robot (HATAPRO, Zukku, size: 68 × 100 × 68 mm, weight: about 170 g) that issues a dangerous posture warning, the details of which are explained later. A web camera (Logicool, HD WEBCAM C310N) and a Raspberry Pi (LABISTS, Raspberry Pi 4B, memory: 4 GB) were used for the posture detection module, which was placed in front of the participants. A USB Accelerator (Google, Coral USB Accelerator) was used to run PoseNet on the Raspberry Pi.
Response to Automatic Detection/Warning of Abnormal Postures Abnormal posture detection. We performed abnormal posture detection for tilting the head and neck backward, which is a dangerous posture likely to cause aspiration of food and saliva. The experiment was conducted with the same research participants above without swallowing jelly. The instructions for the participants are shown in Table 3. Collapse of the backward tilt of the head and neck was defined as when the magnitude of the outer product of the vector of the line segment connecting the participant’s nose and the shoulder
Table 3. Experimental procedure for response to warning.
Table available in the original PDF paper.
Step Procedure (i) Instruct the participant to have a “comfortable posture.” (ii) Instruct the participant to slowly tilt the head and neck backward. (iii) When the communication robot issues a warning, observe the reaction of the participant for 5–10 seconds, and end the experiment. (iv) If the communication robot does not issue a warning, repeat steps (i) to (iii) up to five times.
midpoint and that of the line segment connecting the shoulder midpoint and the left shoulder was 1.1 times larger than the comfortable posture.
Automatic warning. When collapse of the backward tilt of the head and neck is detected for more than 30 consecutive frames, the communication robot issues a warning, saying “Posture collapse detected.” Communication robot ZUKKU was used for the warning. The owl-shaped robot, which is small enough to fit in the palm of a person’s hand, speaks, moves its wings randomly, and emits light from its body. After the experiment, we conducted a free-description questionnaire asking the participants how they felt about the warning.
Evaluation of reaction to the warning. The video data taken during the experiment was used to analyze the participants’ reactions. The number of times the communication robot issued a voice warning and the number of times the participants responded to the warning were examined, and the par- ticipants were judged to have responded if any of the following took place within 10 seconds of the warning: A change in posture such as by returning to a comfortable posture or turning toward the robot; an action toward the author such as by looking at them or asking how to deal with the situation; the utterance of a sound, indicating that the warning was heard. It should be noted that warnings were issued during the Food Test of some participants, i.e., during the experiment of swallowing in a posture other than backward tilt of the head and neck. Therefore, we also counted the number of warnings and responses during that period.
RESULTS AND DISCUSSION
Definition of Abnormal Postures During Swallowing We conducted swallowing tests to define abnormal postures while eating. The participants sat in a chair, tilted in 10° increments, and ate jelly for people with swallowing difficulties. In addition, the angle at which each posture became dangerous was defined by asking the subjects to report on their posture. Figure 2 shows the relationship between the angles of each posture and subjective evaluation scores of difficulties in swallowing.
In general, eating and swallowing functions gradually decline with age. In particular, since the target population of our automatic warning system for abnormal postures is frail elderly people who receive day-care services (Ohashi et al., 2021), there is a high possibility that they have difficulties
Towards Accident Prevention: An Aspiration Risk Warning System for Older Adults 113
Figure 2: Subjective evaluation results of difficulty in swallowing for each subject in leaning forward, tilting left/right, and chin-tucking postures.
with eating and swallowing. The posture that the healthy adult participants scored as 3 points (uncomfortable) is therefore likely to be more dangerous for elderly people, who are likely to have weakened eating and swallowing functions. We thus defined an abnormal posture as a posture that received a score of 3 by at least one of the healthy adult participants. As a result, the abnormal posture angle thresholds were defined as leaning forward 30°, tilting left/right 30°, and chin tucking 20°. For tilting left/right, the posture with 3 points or more starts at 40°. However, based on the answer to the free-description question of participant No. 8, that “It was not difficult to swallow, but I thought 40° and 50° on both sides were dangerous postures,” we defined 30° as an abnormal posture threshold for tilting left/right as taking into account the risk of falling.
Response to Automatic Detection/Warning of Abnormal Postures
Table 4 shows the results of participant responses to the warnings issued by the communication robot. Five and four warnings were issued for partici- pants No. 3 and No. 5, respectively. This is because warnings were issued during the Food Test in postures other than backward tilt of the head and neck. In the experiments with tilting left/right, participant No. 3 tended to bring the spoon to their mouth with the chin slightly raised when putting jelly in their mouth for swallowing. At that time, the communication robot detected a collapse of the backward tilt of the head and neck, and a warning sound was emitted. Participant No. 5 sometimes moved their head and neck when listening to the instructions, which were given by the author from the side. In addition, for the evaluation of tilting the head and neck backward, warnings were issued twice during one trial, so the number of warnings was higher than for the other participants. The warning system was activated 17 times in total across all experiments. Of these, the participants responded seven times (41.2%). The following answers were received when we asked the participants who did not respond to the warnings why they did not: “I
Table 4. The number of warnings issued and responses by the participants.
could not decide what action to take”(No. 2); “I didn’t feel that [the posture] was dangerous. And I didn’t know how to respond because I didn’t know what posture was not dangerous” (No. 6); “I could not understand the voice of the warning and could not understand what it was saying” (No. 7); “I was not aware that the collapsing posture was dangerous” (No. 8).
About half of the abnormal posture warnings did not elicit a response from the participants, and the responses of participant Nos. 2, 6, and 8 were considered to be caused by problems with the content of the warning voice, i.e., “Posture collapse detected.” This made it difficult for the participants to understand that they were in a dangerous posture and what they should do next. As a result, the participants did not know how to act, and may not have been able to respond appropriately. It was considered that a possible solution to this would be to set the warning voice such that the participant can understand the situation as dangerous, and include the next action to be taken; for example, “This posture is dangerous. Please correct your posture.”
Participant No. 3, whose warning system was activated five times during the experiment, stated that “the warning is probably issued because I am eating badly, but I do not feel any danger” and did not show any reaction to the subsequent two warnings. Similarly, participant No. 5, whose warning system was activated four times, did not respond to the second warning or later. The reason for the lack of response may be that the repeated warnings during the 40-minute experiment reduced the sense of danger. For people who tend to frequently perform actions that are judged to be abnormal postures during meals, warnings will be frequently issued. Thus, one of the challenges is to develop a warning method that can provide such people with a sense of danger in a sustained manner.
The Future Direction of a Communication Robot The following responses were obtained regarding the functions of the com- munication robot: “It is good that there are unnecessary movements in a good sense. And I feel familiar with it because of its movements” (No. 1); “The appearance is cute. It is friendly” (No. 3); “The movement is a little anno- ying, and I think the lights emitted by the robot are hard to see” (No. 5);
Towards Accident Prevention: An Aspiration Risk Warning System for Older Adults 115
“I thought it would be a startling sound, but the sound was gentler than I expected. It would be easier to notice if the sound was louder” (No. 7); and “I didn’t realize that the communication robot was talking; I thought the per- son in charge of the experiment was talking because the robot’s voice was so sullen. I think it would be better to make the communication robot’s voice more suitable” (No. 10).
Based on this feedback, the authors discussed possible changes to the robot. First of all, in this experiment, we obtained the answer that the communica- tion robot provided a sense of familiarity. This suggests that the robot could be developed into a device that can prevent accidents during meals without causing anxiety to elderly people and accompany them in their daily lives. On the other hand, it was difficult to generate a sense of urgency. This can be solved not only by changing the voice of the communication robot, but also by displaying warnings in large letters, colors, and images on a small display connected to the robot. In addition to notifying the elderly people, the robot could be linked to the smartphones and wearable devices of care facility staff, thereby reducing their workload. The robot should therefore be used in com- bination with other devices at a care facility, with such devices connected by a communication network, to support cooperative accident prevention when a warning issued and reduce the burden on staff while staying close to the daily lives of the elderly.
CONCLUSION
In this study, we aimed to develop a system to prevent accidents during meal- time for older adults in nursing care facilities. We conducted an experiment to 1) define abnormal posture thresholds during swallowing and 2) extract responses to warnings of abnormal postures using a communication robot. As a result, the abnormal posture angle thresholds were defined as leaning forward 30°, leaning left/right 30°, and chin tucking 20°, taking into account the risk of falling. Regarding the warning of abnormal posture by the com- munication robot, it was found that the first warning was sometimes ignored and that subsequent warnings were often ignored due to repetition. As war- nings alone are insufficient to avoid danger, it is necessary to instruct the user and the people around them on how to respond. In addition, if the system is to be used in a nursing care facility, it is necessary for the warning to be communicated to both the user and caregiver. We will continue this resea- rch toward the creation of an accident prevention system that can support nursing care facilities facing a labor shortage.
ACKNOWLEDGMENT
The authors would like to express their appreciation to all the research participants.
Ethical Consideration. This study was approved by the Human Sub- ject Research Ethics Review Committee, Tokyo Institute of Technology (Approval No. 2020282).
Conflict of Interest. This study is partly financially supported by HATA- PRO, INC.
REFERENCES
Asif, U., Von Cavallar, S., Tang, J. and Harrer, S. (2020), “SSHFD: Single shot human fall detection with occluded joints resilience,” Frontiers in Artificial Intelligence and Applications, Vol. 325, pp. 2656–2663. Care Work Foundation. (2018), Report on the Research Survey Project on the
Prevention of Accidents Related to the Use of Nursing Care Services, Tokyo, available at: http://www.kaigo-center.or.jp/report/pdf/h30_kaigojiko_houkoku_ 20180402.pdf [in Japanese] Huxhold, O., Li, S.C., Schmiedek, F. and Lindenberger, U. (2006), “Dual-tasking postural control: Aging and the effects of cognitive demand in conjunction with focus of attention,” Brain Research Bulletin, Vol. 69, No. 3, pp. 294–305. Laughton, C.A., Slavin, M., Katdare, K., Nolan, L., Bean, J.F., Kerrigan, D.C., Phil-
lips, E., et al. (2003), “Aging, muscle activity, and balance control: Physiologic changes associated with balance impairment,” Gait and Posture, Vol. 18, No. 2, pp. 101–108. De Miguel, K., Brunete, A., Hernando, M. and Gambao, E. (2017), “Home camerabased fall detection system for the elderly,” Sensors (Switzerland), Vol. 17, No. 12, available at: https://doi.org/10.3390/s17122864 Ohashi, T., Ito, Y., Kurabayashi, D. and Saijo, M. (2021), “Designing an Abnor-
mal Posture Detection System to Prevent Accidents During Meal Assistance for Older Adults: A User-Centered Design Approach,” Lecture Notes in Networks and Systems, Vol. 263, pp. 345–352. Papandreou, G., Zhu, T., Kanazawa, N., Toshev, A., Tompson, J., Bregler, C. and Murphy, K. (2017), “Towards Accurate Multi-person Pose Estimation in the Wild,” available at: http://arxiv.org/abs/1701.01779 Saitoh, E., Kimura, A., Yamori, S., Mori, H., Izumi, S. and Chino, N. (1986),
“Videofluorography in rehabilitation of dysphagia,” The Japanese Journal of Rehabilitation Medicine, Vol. 23, No. 3, pp. 121–124. Serpa, Y.R., Nogueira, M.B., Neto, P.P.M. and Rodrigues, M.A.F. (2020), “Evaluating Pose Estimation as a Solution to the Fall Detection Problem,” 2020 IEEE 8th International Conference on Serious Games and Applications for Health, SeGAH 2020, available at: https://doi.org/10.1109/SeGAH49190.2020.9201701 Takagi, S., Hayashi, H., Morishita, Y., Hamaya, K., Ohashi, T. and Saijo, M. (2021),
“Designing an Abnormal Posture Warning System using a Pose Estimation Model for Meal Assistance for Older Adults,” Conference Program of 2021 International Conference on Frontiers of Artificial Intelligence and Machine Learning, p. 8. Tohara, H., Saitoh, E., Baba, M., Onogi, K. and Uematsu, H. (2002), “Swallowing Characteristics and Tongue Surface Movements of Persons with regard to Pasty Foods - A Dysphagia Evaluation System without Videofluorographic Study,” The Japanese Journal of Dysphagia Rehabilitation, Vol. 6, No. 2, pp. 196–206. Wang, C.C., Chiang, C.Y., Lin, P.Y., Chou, Y.C., Kuo, I.T., Huang, C.N. and Chan,
C.T. (2008), “Development of a fall detecting system for the elderly residents,”2nd International Conference on Bioinformatics and Biomedical Engineering, ICBBE 2008, IEEE, pp. 1359–1362. Woollacott, M. and Shumway-Cook, A. (2002), “Attention and the control of posture and gait: A review of an emerging area of research,” Gait and Posture, Vol. 16, No. 1, pp. 1–14.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors in Accessibility and Assistive Technology
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-13-1
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional