Worldwide demographics are changing; we are living longer and, in developed countries, the birth-rate is dropping. In this context and motivated by the challenge of sustainable ageing, this paper presents sustAGE, a multi-modal person-centred IoT platform, which integrates with the daily activities of ageing employees both at work and outside. The sensed information allows the system to assess the state of the users and context-related aspects with the aim to provide timely recommendations to support wellbeing, wellness, and productivity. Herein, we describe the use cases, outline the overall system architecture, and introduce the first prototype of the platform implemented up-to-date. Furthermore, the results from the usability evaluation conducted with real users who used the prototype for one month are presented.
Companies will have to deal with the challenge of an ageing workforce. Fewer younger workers are available and current workers are asked to postpone their retirement. The issues that arise from this prolonged working life are the increasing risk for longer drop outs due to injuries, an imbalance of job-related demands and worker resources due to the decline in physical and cognitive abilities, and a potential increment of absenteeism. Smart monitoring systems (Fdez-Arroyabe et al. 2020) and flexible work setups (Giakoumis et al. 2019) can support establishing a reactive working environment that considers the needs and capabilities of each individual worker to develop personalised strategies and proactively prevent and mitigate health risks.
In this regard, sustAGE develops a person-centred smart solution to support the employment and the later retirement of ageing workers. The solution is developed along three directions: a) improvement of occupational safety and health, based on the monitoring of the workplace conditions and the individual health states, b) provision of personalised recommendations for a physical and mental health improvement within and outside the work premises, and c) optimisation of the overall workforce productivity with the early detection and avoidance of risky and stressful conditions that affect the performance of individual workers or worker groups, and recommendations for task/job role modifications.
A key innovation of sustAGE is the adoption and integration of the Micro-Moments (MiMos) concept, a marketing term originally coined with the intent-driven interactions of humans with their smartphones in order to cover specific information needs. sustAGE defines information-rich MiMos to digitise interactions with the
Human Interaction & Emerging Technologies (IHIET-AI 2022) Artificial Intelligence & Future Applications https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8989-4 physical environment, repeated patterns, or events occurring in the workers’ daily living routines. The information-rich MiMos guide the triggering of timely recommendations to achieve a better recommendation acceptance.
The rest of the paper is laid out as follows. We first describe the use-cases where we validate the sustAGE solution, and then provide a high-level description of the overall system. The following sections introduce the first prototype with the MiMos and recommendations implemented, and discuss the main results from the usability evaluation in the real working settings. The last section concludes the paper, highlighting the key findings.
USE CASES Three use-cases are selected for the implementation and evaluation of the system functionalities: i) an indoor use-case addressing assembly line work in the manufacturing industry, ii) an outdoor use-case addressing cargo port operations in the maritime transportation & logistics industry, and iii) a cross-domain use-case for all workers outside their work environment.
Manufacturing Industry Assembly line workers in the manufacturing industry perform highly repetitive tasks for extended time intervals with minimal control over the pace and sequence of work. Such task conditions correspond to poor cognitive stimulation and may contribute to an accelerated decline of cognitive abilities with age (Gajewski et al. 2010). Moreover, prolonged, sub-optimal work-related body postures lead to an accumulation of physical strain and chronic over-straining of specific muscle groups that may, with increasing age, result in health impairments and an early retirement.
This use-case depicts this type of work. Workers repetitively perform a similar task sequence on a car part that is being slowly moved forward to the different work stations by a conveyor system. The assembly line operates continuously and slides at a constant speed. Workers at each work station perform a specific task sequence every 4 to 5 min for an 8 h shift, only interrupted by a 45 min lunch break and two shorter 10 min breaks. If workers need a short break, these are temporarily replaced by the team leader, who also masters the task.
The sustAGE interventions for OSH risk prevention and mitigation include alternative break-management (e.g., event-elicited micro-breaks), and task switching. In terms of MiMos, sustAGE monitors the environmental conditions of the workplace and the state of the workers with respect to physical/mental workload and prolonged awkward body postures. If these conditions meet user-specific criteria, then a recommendation for a short individual micro-break or for task switching to another workstation with different demands is sent to the shift manager, who makes the
Human Interaction & Emerging Technologies (IHIET-AI 2022) Artificial Intelligence & Future Applications https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8989-4 decision to accept or decline it. Also, recommendations to the shift manager for line speed adjustments rely on monitoring workers’ idle time between consecutive task cycles, along with their physical condition. During the workers’ short breaks, recommendations for physical exercises, cognitive training, or hydration are also issued.
Transportation & Logistics Industry This scenario addresses two occupational groups in port operations: dock workers and crane operators. Dock workers are exposed to strenuous physical work with increased risks for OSH (Cezar-Vaz et al. 2014). Despite technological developments, older workers are still confronted with substantial physical demands that can be increased through the exposition to adverse environmental conditions. Crane operators are exposed to mechanical vibrations, noise, and sun glare and experience also the need for sustained attention at a rather high repetitiveness of work activity, which increases mental workload (Yakub & Sidik 2014). Environmental conditions can also induce increased physical and mental workload for the crane operator. Both groups need to remain vigilant for potential safety hazards during cargo handling.
We focus on environmental and task-related demands for both dock workers and crane operators as well as on accident prevention during normal cargo operations. The sustAGE interventions address physical exertion of dock workers through extra breaks for hydration and recovery. Interventions for workload mitigation for crane operators include recommendations for stretching activities and alternative cognitive activities during breaks. Interventions for accident prevention are instantiated as alerts to dock workers for the avoidance of collisions with the moving crane and/or pending cargo.
In terms of MiMos, sustAGE monitors the environmental conditions of the workplace and the workers’ physical workload. When specific conditions match dynamic, personalised critical values, recommendations are issued to the lead worker, who makes the actual decision to grant a break or not. Recommendations to play a computerised Cognitive Game (CG) during a lunch break, to meet or to interact with colleagues at shift breaks or at the end of the day are also included. For the purpose of accident prevention, sustAGE monitors the workers’ location relative to a threat volume defined by the moving crane and container (Lourakis & Pateraki 2021). In case the threshold is exceeded, an immediate alert with a suggestion to move to a safe distance is issued to the worker.
Cross-domain – Outside the Workplace The third scenario targets physical and cognitive aspects of wellbeing outside the regular work schedule. The interventions include training activities for maintaining physical and mental fitness. In an attempt to achieve this without being intrusive, the system reminds workers to follow a scheduled training program in the out-of-work
Figure 1: Block diagram illustrating the overall sustAGE system and its components. The system receives multi-modal data in real-time as input, and outputs intelligent recommendations or actions targeted at the workers or their managers.
hours. Before issuing the reminder, the system considers the worker’s recent progress, the weather conditions to check if outdoor activities are permitted, and the worker status and availability for training.
In terms of MiMos, sustAGE monitors workers’ physical and mental fitness indicators, sleep duration and time of sedentary behaviour, workout time, and performance in the recommended health and mental activities. The average sleep duration is employed to recommend workers to get more rest if needed; e.g., going to sleep earlier or waking up later, especially when low sleep quality is accompanied by fatigue incidents at work. GPS information and geo-fencing are used to detect when the user arrives at or leaves from the workplace.
SYSTEM OVERVIEW This section introduces the overall system. Section 3.1 provides a high-level description of the system components, while Section 3.2 presents the user interfaces.
Figure 2: User interfaces of the sustAGE solution: (a) home screen of the smartphone app, (b) home screen of the smartwatch app, (c) display of the smartwatch app when receiving notifications (green), recommendations (yellow), and alerts (red).
Figure available in the original PDF paper.
Back-end Architecture The sustAGE system –cf. Figure 1– exploits an IoT ecosystem composed of vision, speech, positioning, wellness, and environmental sensors to capture and model multi- modal information for extracting workers- and context-related information. The system is conceptually divided into five layers.
The Monitoring layer receives real-time sensor data collected by the IoT devices and performs initial processing of user-centred aspects inside and partially outside the work environment. The components in this layer analyse worker’s behaviour through monitoring physical actions in the workplace, infer the sentiment conveyed by the workers when interacting with the system, and provide the system with workers’ position, velocity, and timing information.
The Streaming layer corresponds to the communication centre of the entire system and distributes the information. The sustAGE Bridge receives both raw data sent from the sensors, and inferred information from the components in the Monitoring layer. The messaging bus forwards the information to the high-level components and facilitates their real-time interaction and information exchange.
The Personalisation layer encapsulates the application logic of sustAGE and includes personalisation mechanisms. It distils the short- and medium-term states and long- term traits –including affect recognition and smartwatch-based human activity recognition (Mallol-Ragolta et al. 2021)– and extracts knowledge abstractions and regular updates on individual user profiles. The components in this layer detect/predict the corresponding MiMos, exploiting sensor data.
The Recommendation layer delivers the right recommendation at the right time to im- prove the physical and mental health, wellbeing, safety and productivity of the ageing workers. This layer employs rule-based recommenders, which trigger recommendations when the appropriate conditions are met and implements user feedback mechanisms to record user responses to recommendations.
Finally, the Interaction layer includes those components responsible for the users’ interaction with the system. The dialog manager focuses on the information exchange between users and the system itself. The CGs are the platform containing the serious games designed and created to train the cognitive skills of the users.
User Interfaces Users’ interaction with the system takes place through a smartphone and a smartwatch. Specifically, a Xiaomi Mi 9 and a Garmin Vivoactive 3. Both devices run customised apps to fit the purposes of the overall sustAGE solution. The home screen of the smartphone app is depicted in Figure 2 (a). The app functionalities include checking the history of recommendations received, reviewing the scores obtained while playing the CGs, recording a speech sample, retrieving information from the environmental sensors, consulting users’ profile information, and taking a heart rate measurement at rest, so it can be used as a baseline measurement for further computations at the back-end of the system. The front-end of the smartwatch app – cf. Figure 2 (b)– features a digital clock with the current heart rate of the user updated in real-time and a network icon to provide feedback about the data transmission status between the smartwatch and the system. System recommendations are issued to the users via their smartphones and smartwatches. In the smartphone, the recommendations appear as pop-up messages; in the smartwatch, we use screen notifications accompanied by vibrations. As the recommendations are categorised into three priority levels –notifications, recommendations, and alerts–, the smart- watch notifications convey this information visually using a colour coding approach –cf. Figure 2 (c)– and through the haptic modality with different vibration patterns.
FIRST PROTOTYPE The first prototype of the sustAGE system detects a set of MiMos both in and outside the work environment –cf. Table 1– and are linked with a specific recommendation(s). Some MiMos are designed to provide information to the system, feeding the users’ profile (Athanassiou et al. 2021), such as the inferred sleep time, the time when users leave the workplace, or the results obtained when playing the CGs.
USABILITY EVALUATION OF THE FIRST PROTOTYPE The usability evaluation is conducted with 8 (3 f, 5 m) and 15 (15 m) manufacturing and transportation & logistics industry workers with a mean age of 54.8 and 53.5 and a standard deviation of 4.3 and 4.3, respectively, who used the system for one month. A first impact and a task scenarios evaluation were designed to assess the usability of
Table 1: Summary of the MiMos detected with the corresponding recommendations issued inside and outside the work environment in the first prototype of sustAGE for the manufacturing industry (M) and the transportation & logistics industry (T&L).
Table available in the original PDF paper.
the smartphone and the smartwatch apps. The results obtained indicate users did not experience usability issues while interacting with the customised apps. We employ the System Usability Scale (SUS) (Brooke 1996) to evaluate the usability of the first prototype as a whole. The SUS is composed of 10 predefined items, which we adapted to the sustAGE solution, and users rate their level of agreement with the different statements in the questionnaire using a 5-point Likert scale. This instrument was administered at the end of the evaluation phase. The overall SUS score ranges from 0 to 100 for an intuitive interpretation. SUS scores above 90s indicate an exceptional usability, while scores below 70 indicate usability issues (Bangor et al. 2009).
Although the overall SUS score obtained in the transportation & logistics industry use-case is slightly above the threshold of 70, it is below for the manufacturing industry use-case. The latter reflects the negative experience of the users, who reported receiving recommendations that did not reflect the reality: users received
Table 2: Descriptive statistics computed from the overall SUS scores obtained from the questionnaires answered by manufacturing industry (M) and transportation & logistics industry (T&L) workers after using the first prototype for one month.
Table available in the original PDF paper.
recommendations they had just arrived at work although they had been working for a few hours already due to previously undetected technical issues. Users’ negative feedback was also reported regarding the recommendations issued during breaks, as they felt conditioned in their free time.
CONCLUSIONS
This paper presented the first prototype of sustAGE. We described the use-cases where we validate our solution, provided an overview of the overall system, and analysed the main results obtained from the usability evaluation of the first prototype as a whole. The results obtained especially from the users’ feedback allow us to highlight the importance of system trustworthiness. It is critical to model the environment accurately, so the recommendations issued match the current state of the user. Additionally, we would like to emphasise the challenge of persuading the users to follow the system recommendations, notably when these are targeted at their leisure time. Such systems should stress it is on the users’ own interest to follow the recommendations to improve, as in the case of sustAGE, their productivity, health state, and overall wellbeing.
ACKNOWLEDGMENTS
This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 826506 (sustAGE).
REFERENCES
Athanassiou, G., Pateraki, M., Varlamis, I. (2021), “Micro-moment-based interventions for a personalized support of healthy and sustainable ageing at work: Development and application of a context-sensitive recommendation framework.” In Proceedings of the 13th International Joint Conference on Computational Intelligence - SmartWork, Valletta, Malta – Virtual Event, 409 – 419, SciTePress.
Bangor, A., Kortum, P., Miller, J. (2009), “Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale.” Journal of Usability Studies, 4 (3), 114 – 123, UXPA. Brooke, J. (1996), “SUS: A ’Quick and Dirty’ Usability Scale.” In: Jordan, P. W.,
Thomas, B., McClelland, I. L., and Weerdmeester, B. (eds.). Usability Evaluation in Industry, ch. 21, 6 pages, Taylor & Francis. Cezar-Vaz, M. R., De Almeida, M. C. V., Bonow, C. A., Rocha, L. P., Borges, A.
M., Piexak, D. R. (2014), “Casual Dock Work: Profile of Diseases and Injuries and Perception of Influence on Health.” International Journal of Environmental Research and Public Health, 11 (2), 2077 – 2091, MDPI. Fdez-Arroyabe, P., Soliño Fernández, D., Bilbatua Andrés, J. (2020), “Work environment and healthcare: a biometeorological approach based on wearables.” In: Dey, N., Ashour, A. S., Fong, S. J., and Bhatt, C. (eds.). Wearable and Implantable Medical Devices, 7, 141 – 161, Academic Press. Gajewski, P. D., Wild-Wall, N., Schapkin, S. A., Erdmann, U., Freude, G.,
Falkenstein, M. (2010), “Effects of aging and job demands on cognitive flexibility assessed by task switching.” Biological Psychology, 85 (2), 187 – 199, Elsevier. Giakoumis, D., Votis, K., Altsitsiadis, E., Segkouli, S., Paliokas, I., Tzovaras, D.
(2019), “Smart, personalized and adaptive ICT solutions for active, healthy and productive ageing with enhanced workability.” In Proceedings of the 12th International Conference on PErvasive Technologies Related to Assistive Environments, Rhodes, Greece, 442 – 447, ACM. Lourakis, M., Pateraki, M. (2021), “Markerless visual tracking of a container crane spreader.” In Workshop Proceedings of the International Conference on Computer Vision, Montreal, Canada – Virtual Event, 2579 – 2586, CVF. Mallol-Ragolta, A., Semertzidou, A., Pateraki, M., Schuller, B. (2021), “harAGE: A
Novel Multi-modal Smartwatch-based Dataset for Human Activity Recognition.” In Proceedings of the 16th International Conference on Automatic Face and Gesture Recognition, Jodhpur, India – Virtual Event, 7 pages, IEEE. Yakub, N. W., Sidik, S. M. (2014), “Prevalence and contributing factors of job strain among crane operators in a port container terminal in Malaysia.” Malaysian Journal of Medicine and Health Sciences, 10 (2), 39 – 46, UPM.
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-7923-8989-4
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional