Olmo Alonso Moreno Franco, Christian Di Natali, Luigi Monica, Darwin Caldwell, Jesus Ortiz (2025). Advancing Occupational Exoskeletons: Usability Assessment of a Minimalist Calibration Interface. In: Human Interaction and Emerging Technologies (IHIET 2025). AHFE Open Access, vol. 197, pp. 433–440. https://doi.org/10.54941/ahfe1006736
Research on ergonomics for work-related musculoskeletal disorders (MSDs) remains a significant challenge for affected individuals, businesses, and society. As the most costly category of occupational health issues, MSDs affect more than one in three European workers, making them the most common work-related illness across all industries. A common occupational task in various industries is manual materials handling (MMH), which refers to the process of manually moving, lifting, lowering, pushing, pulling, or carrying materials, goods, or products. Nevertheless, it also poses a risk of injuries and MSDs due to different factors, such as the physical strain involved, repetitive motion, poor ergonomics, and environmental factors, such as uneven floors, cluttered workspaces, slippery surfaces, or poor lighting. A promising approach to address MSDs in the workplace is the use of an occupational back-support exoskeleton. This is a wearable technology designed to reduce lumbar spine physical strain during lifting tasks. Research has demonstrated that these wearable devices can decrease back-muscle activity by up to 40%, effectively reducing spinal loading during MMH tasks. According to the actuation principle, an active exoskeleton (with sensors, controller and actuators) can be more versatile in terms of configuration than a passive exoskeleton. A specific characteristic of active exoskeletons is the possibility of modifying the control strategy to provide appropriate assistive forces according to the task. This control strategy is modulated through a human–machine interface (HMI), which is the cornerstone of user interaction and the basis of cognition to modify and adjust parameters in a system. In an active occupational exoskeleton, more functions can be adjusted, such as calibration, user information (weight and height), and control gains.Purpose: This study presents the usability assessment of the motor calibration function in the novel User Command Interface Round (UCI-R), a minimally adaptable setup system for occupational exoskeletons. Calibration is the first step to set up the exoskeleton XoTrunk before starting the MMH task. A user study was conducted with 10 participants by comparing the original user interface with a newly designed, minimised version. The experiment assessed improvements in user experience and efficiency between the two interfaces.Methods: To systematically evaluate user interaction with the interface, we applied the GOMS (Goals, Operators, Methods, and Selection rules) model, which is a well-established cognitive modelling technique in HCI. In this study, the GOMS was used to compare user actions between the two interface versions with a focus on task flow related to motor calibration. The analysis aimed to estimate the cognitive load, execution time, and operational complexity, providing quantitative and qualitative insights into user experience. The goal of this study was exoskeleton calibration; the operators are interactive cards from the exoskeleton calibration section of the User Command Interface (UCI) and the UCI-R visual interface. The methods are the sequence functions to be performed (to achieve motor calibration) in both interfaces, and the calibration action has a specific rule, which is to remain still during the calibration.Results: Ten subjects participated in the assessment showing differences in usability attributes such as efficiency, satisfaction and time on ask favorable for the UCI-R visual interface.Conclusions: Usability attributes such as effectiveness, efficiency, ease of task, time on task, documentation organisation and satisfaction can lead to the acceptance of a new technology as an occupational exoskeleton. A well-designed user interface improves user experience and satisfaction. The minimised UCI-R yielded higher ASQ and SUM results than the standard version of the UCI.
Research on ergonomics for work-related musculoskeletal disorders (MSDs) remains a significant challenge for affected individuals, businesses, and society. As the most costly category of occupational health issues, MSDs affect more than one in three European workers, making them the most common work-related illness across all industries. Occupational MSDs can result from mechanical (physical) exposure and psychosocial factors in the workplace. However, determining whether these work-related factors are the actual cause of pain can be challenging because there may be non-work- related conditions, such as ageing process (Winkel et al., 2008).
A common occupational task in various industries is manual materials handling (MMH), which refers to the process of manually moving, lifting, lowering, pushing, pulling, or carrying materials, goods, or products. Nevertheless, it also poses a risk of injuries and MSDs due to different factors, such as the physical strain involved, repetitive motion (which can lead to fatigue), poor ergonomics (workplaces may not be designed to facilitate safe practices), and environmental factors, such as uneven floors, cluttered workspaces, slippery surfaces, or poor lighting (Yang et al., 2020).
A promising approach to address MSDs in the workplace is the use of an occupational back-support exoskeleton. This is a wearable technology designed to reduce lumbar spine physical strain during lifting tasks. Research has demonstrated that these wearable devices can decrease back-muscle activity by up to 40%, effectively reducing spinal loading during MMH tasks (Poliero et al., 2021). An exoskeleton is an electromechanical wearable device that operates in parallel with the body and can function through passive or active actuation to enhance limb capabilities (Anam et al., 2012).
The industries with the greatest interest in robotic exoskeleton technologies include construction, manufacturing, demolition, logistics, and shipbuilding, where physically demanding tasks, repetitive motions, and heavy lifting are common. Beyond these industries, sectors such as health and social care, as well as agriculture, are increasingly exploring exoskeletons to reduce occupational injuries and enhance worker performance (Bogue, 2018). According to the actuation principle, an active exoskeleton (with sensors, controller and actuators) can be more versatile in terms of configuration than a passive exoskeleton (Poliero et al., 2021). A specific characteristic of active exoskeletons is the possibility of modifying the control strategy to provide appropriate assistive forces according to the task (Lazzaroni et al., 2020). This control strategy is modulated through a human–machine interface (HMI), which is the cornerstone of user interaction and the basis of cognition to modify and adjust parameters in a system (Gong et al., 2009). In an active occupational exoskeleton, more functions can be adjusted, such as calibration, user information (weight and height), and control gains.
This study presents the usability assessment of the motor calibration function in the novel User Command Interface Round (UCI-R), a minimally adaptable setup system for occupational exoskeletons. Calibration is the first step to set up the exoskeleton XoTrunk before starting the MMH task. The first version of this HMI was presented by Moreno et al. (2022), this interface presented four functions: a) motor calibration, b) user profile management, c) control strategy configuration, and d) signal monitoring. A user study was conducted with 10 participants by comparing the original user interface with a newly designed, minimised version. The experiment assessed improvements in user experience and efficiency between the two interfaces.
METHODOLOGY
To systematically evaluate user interaction with the interface, we applied the GOMS (Goals, Operators, Methods, and Selection rules) model, which is
a well-established cognitive modelling technique in HCI (Ramkumar et al., 2017). In this study, the GOMS was used to compare user actions between the two interface versions with a focus on task flow related to motor calibration. The analysis aimed to estimate the cognitive load, execution time, and operational complexity, providing quantitative and qualitative insights into user experience. The goal of this study was exoskeleton calibration; the operators are interactive cards from the exoskeleton calibration section of the User Command Interface (UCI) and the UCI-R visual interface. The methods are the sequence functions to be performed (to achieve motor calibration) in both interfaces, and the calibration action has a specific rule, which is to remain still during the calibration.
System Description The Wearable Robots, Exoskeletons and Exosuits Laboratory (XoLab) of the Advanced Robotics Department (ADVR) is an interdisciplinary team at the Istituto Italiano di Tecnologia (IIT). Our group focuses on the research and development of wearable devices for occupational applications, such as the back-support exoskeleton XoTrunk (Poliero et al., 2022) and the upper-limb exoskeleton Shoulder-SideWINDER (Park et al., 2022). Both exoskeletons are active exoskeletons according to their actuation type, and their primary function in wearable active devices is to calibrate the actuators before they start to operate.
As mentioned, the XoTrunk is an active back-support exoskeleton that weighs 6 kg, and it was designed to assist with MMH tasks by reducing strain on the lower back and hips (see Fig. 1-a). The system features a rigid aluminium frame worn like a backpack, with three passive hip-to-thigh joints and two DC brushless motors that provide up to 30 Nm of assistive torque in the sagittal plane (Natali et al., 2020). The control system uses an accelerometer from an inertial measurement unit (IMU) on the user’s sternum to detect movement and modulate force assistance. Motor calibration is important because it captures the bias value of the torque sensor. This value is part of a constructed inverse rotation matrix Rbn at the time of calibration; thus, the gravity direction in the sagittal plane x is registered, affecting directly to the exoskeleton’s torque output (Lazzaroni et al., 2020). During calibration, the user must stand still.
The first HMI designed to interact with the XoTrunk exoskeleton was the UCI. The proposed system enhances the functionality of occupational exoskeletons by providing a user-friendly control system that allows users to interact with and customise exoskeleton settings, including secure identification, signal monitoring, user management, and control strategy adjustments. Designed with security and usability in mind, it includes a navigation wheel and buttons for efficient user input, and it has a display with a resolution of 800×480 pixels and a colour screen (see Fig 1-b).
The proposed UCI-R is a minimised version of the UCI with buttons for interface navigation and a colour display resolution of 480×480 pixels (see Fig 1-c). Both interfaces feature intuitive navigation elements, such as menus, submenus, cards, and decks. The graphic-user interface (GUI) follows a set
of visual guidelines proposed by Google Material (Yang et al., 2021), and the visual engine is mounted in the Processing framework (Reas and Fry, 2014). When in use, either the UCI or the UCI-R is attached to the exoskeleton, and the user wears the device during operation.
Figure 1: User-exoskeleton interaction: XoTrunk and the UCI/UCI-R. (a) Occupational exoskeleton XoTrunk. (b) User command interface motor calibration sequence. (c) User command interface-round motor calibration sequence.
Evaluation Metrics The standardised evaluation metrics chosen in this study were selected from the user-centred evaluation for wearable robotics devices (WRD), a user research platform provided by the Interactive Usability Toolbox (IUT) (Meyer et al., 2023). The scale of Cronbach’s alpha coefficient of reliability is defined in Tavakol and Dennick (2011). Two assessment metrics were selected:
1) The After-Scenario Questionnaire (ASQ) is a psychometric 3-item instrument with a 7-option Likert scale designed to assess user satisfaction immediately after participants complete specific tasks (or scenarios) in scenario-based usability studies (Lewis, 1991). 2) The Single Usability Metric (SUM) is a 3-item standardised and summated usability metric using a 5-option Likert scale developed to encapsulate into a single score the primary dimensions of usability, such as effectiveness, efficiency, and satisfaction.
EXPERIMENTAL EVALUATION The experiment assessed the usability of the motor calibration function of the UCI and the UCI-R in attributes such as effectiveness, efficiency, ease of task, time on task, documentation organization, and satisfaction.
Participants A group of 10 subjects participated in the experiments; among the participants, 3 were females and 7 were males. The experiment was conducted in accordance with the Declaration of Helsinki and was approved by the Ethics Committee of Liguria (protocol no.: CER Liguria 001/2019).
Experiment Deign The experiment was conducted to compare the usability attributes of the UCI (Activity 1) and miniimalistic version of the UCI-R (Activity 2). In this study, the exoskeleton was not used because we were interested in ensuring that the participant’s attention was focused on the interfaces. Both interfaces were designed in a simulator on a PC. First, in Activity 1, the participant interacted with the UCI simulator and was asked to perform a motor calibration sequence consisting of seven steps (from the main menu window) until the participant obtained a final result, either successful calibration or failure. At the time of calibration, the participant can select between successful and failed tasks. It was requested to first perform a successful task and then a failure. In Activity 2, the user interacts with the UCI-R novel interface in the simulator. In this version, the participant does not perform a sequence for this action; instead, the interface conducts the user to a process that always starts with the motor calibration action. Once the user pressed the start calibration button, the first displayed result was considered successful. Then, the simulator displays the start calibration button once again for a second trial, where the final result is a task failure. Once complete, the UCI-R interface displays the initial screen with no actions, indicating the end of the activity. At the end of each activity, the participant answered the ASQ and SUM questionnaires.
RESULTS AND DISCUSSION
Results of ten participants evaluation of the two user interfaces (UCI and UCI-R) in two activities are presented in the After-Scenario Questionnaire and Single Usability Metric sections.
After-Scenario Questionnaire The internal consistency of the survey data was evaluated using Cronbach’s alpha, which resulted in a coefficient of α = 0.694. As shown in Fig. 2, participants reported in Activity 1 (UCI interface) a score for task ease with a mean value of 3.0 ± 1.699, for task time a mean value of 3.1 ± 1.663, and for time documentation, a mean value of 2.5 ± 1.715. Scores reported by participants in Activity 2 (UCI-R interface) indicated task ease with a mean value of 1.2 ± 0.421, task time with a mean value of 1.2 ± 0.421, and time documentation with a mean value of 1.6 ± 0.843.
The After-Scenario Questionnaire score scale uses 1 as the highest value (strongly agree) and 7 as the lowest (strongly disagree); in this case, lower ASQ scores represent better grading. The ASQ scores demonstrated significantly greater user satisfaction during Activity 2 (UCI-R interface). The mean scores for task ease (1.2), task time (1.2), and task documentation (1.6) were substantially lower than those of Activity 1 (UCI interface), indicating that eliminating unnecessary navigation along the interface improved perceived usability across the exoskeleton calibration process.
Figure 2: After scenario questionnaire scores. Evaluation of Activity 1 (UCI interface) and Activity 2 (UCI-R interface) task ease, task time, and task documentation attributes.
Single Usability Metric The internal reliability of the scale was confirmed using Cronbach’s alpha of α = 0.701. As shown in Fig. 3, participants reported in Activity 1 (UCI interface) a score for task ease with a mean value of 3.5 ± 1.080, for satisfaction with a mean value of 3.5 ± 1.080, and for time on task, a mean value of 3.5 ± 1.354. Scores reported by participants in Activity 2 (UCI-R interface) indicated task ease with a mean value of 5.0 ± 0.0, satisfaction with a mean value of 4.9 ± 0.316, and time on task with a mean value of 5.0 ± 0.0.
Figure 3: Single usability metric scores. Evaluation of Activity 1 (UCI interface) and Activity 2 (UCI-R interface) in terms of task ease, satisfaction, and time on task attributes.
Participants reported significantly higher usability scores for Activity 2 than for Activity 1, suggesting that direct access to the calibration feature enhances user experience by improving task ease, satisfaction, and completion efficiency.
CONCLUSION
Calibration is a critical procedure for active occupational exoskeletons, such as XoTrunk. In order to achieve this step, an HMI device is required to interact with the exoskeleton. In this study, a usability comparison of two user interfaces was evaluated. Participants reported in Activity 1 (UCI interface) according to their scores a moderate to low satisfaction across the interface; users felt the activity was not particularly easy, took more time, and found a lack of supported information. Participants also experienced more friction, either in navigating, time taken, or clarity; making the UCI interface possibly less intuitive, even if task success was still achievable. For Activity 2 (UCI-R interface) scores, participants indicated significantly better usability, the task felt easier to complete, faster or with less effort, and they felt more satisfied, implying a simpler workflow that reduced both cognitive and interaction load.
ACKNOWLEDGMENT
This work was supported by the Italian Workers’ Compensation Authority (INAIL) and Istituto Italiano di Tecnologia (IIT) within the project Sistemi Ciberneteci Collaborativi - Esoscheletro Collaborativo 3.
REFERENCES
Anam, K., & Al-Jumaily, A. A. (2012). Active Exoskeleton Control Systems: State of the Art. Procedia Engineering, 41, 988–994. https://doi.org/10.1016/ j.proeng.2012.07.273 Bogue, R. (2018). Exoskeletons–a review of industrial applications. Industrial Robot:
An International Journal, 45(5), 585–590. Gong, C. (2009). Human-Machine Interface: Design Principles of Visual Information in Human-Machine Interface Design. 2009 International Conference on Intelligent Human-Machine Systems and Cybernetics, 2, 262–265. Lazzaroni, M., Tabasi, A., Toxiri, S., Caldwell, D. G., Momi, E. D., Dijk, W. van,
Looze, M. P. de, Kingma, I., Dieën, J. H. van, & Ortiz, J. (2020). Evaluation of an acceleration-based assistive strategy to control a back-support exoskeleton for manual material handling. Wearable Technologies, 1, e9. Lewis, J. R. (1991). Psychometric evaluation of an after-scenario questionnaire for computer usability studies: The ASQ. ACM SIGCHI Bulletin, 23(1), 78–81. https://doi.org/10.1145/122672.122692 Meyer, J. T., Tanczak, N., Kanzler, C. M., Pelletier, C., Gassert, R., & Lambercy, O.
(2023). Design and validation of a novel online platform to support the usability evaluation of wearable robotic devices. Wearable Technologies, 4, e3. https:// doi.org/10.1017/wtc.2022.31
Moreno Franco, O. A., Ortiz, J., & Caldwell, D. G. (2022). Evaluation of the User Command Interface, an Adaptable Setup System for Industrial Exoskeletons. 2022 9th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob), 01–07. Natali, C. D., Toxiri, S., Ioakeimidis, S., Caldwell, D. G., & Ortiz, J. (2020).
Systematic framework for performance evaluation of exoskeleton actuators. Wearable Technologies, 1. https://doi.org/10.1017/wtc.2020.5 Park, D., Natali, C. D., Caldwell, D. G., & Ortiz, J. (2022). Control Strategy for Shoulder-SideWINDER With Kinematics, Load Estimation, and Friction Compensation: Preliminary Validation. IEEE Robotics and Automation Letters, 7(2), 1278–1283. https://doi.org/10.1109/LRA.2021.3139320 Poliero, T., Iurato, M., Sposito, M., Natali, C. D., Toxiri, S., Anastasi, S., Draicchio,
F., Monica, L., Caldwell, D. G., Sanguineti, V., & Ortiz, J. (2021). A case study on occupational back-support exoskeletons versatility in lifting and carrying. The 14th PErvasive Technologies Related to Assistive Environments Conference, 210– 217. https://doi.org/10.1145/3453892.3453901 Poliero, T., Lazzaroni, M., Toxiri, S., Natali, C. D., Caldwell, D. G., & Ortiz, J.
(2022). Lifting and Carrying: Do We Need Back-Support Exoskeleton Versatility? In J. C. Moreno, J. Masood, U. Schneider, C. Maufroy, & J. L. Pons (Eds.), Wearable Robotics: Challenges and Trends (pp. 499–503). Springer International Publishing. https://doi.org/10.1007/978-3-030-69547-7_80 Reas, C., & Fry, B. (2014). Processing: A Programming Handbook for Visual Designers and Artists (2nd ed.). MIT Press. Ramkumar, A., Stappers, P. J., Niessen, W. J., Adebahr, S., Schimek-Jasch, T., Nestle,
U., & Song, Y. (2017). Using GOMS and NASA-TLX to Evaluate Human– Computer Interaction Process in Interactive Segmentation. International Journal of Human–Computer Interaction, 33(2), 123–134. https://doi.org/10.1080/ 10447318.2016.1220729 Sauro, J., & Kindlund, E. (2005). A method to standardize usability metrics into a single score. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 401–409. https://doi.org/10.1145/1054972.1055028 Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach’s alpha. International
Journal of Medical Education, 2, 53–55. https://doi.org/10.5116/ijme.4dfb.8dfd Winkel, J., & Westgaard, R. (2008). Risk factors of occupational MSDs and potential solutions: Past, present and future. European Trade Union Institute, 34, 3–7. Yang, B., Xing, Z., Xia, X., Chen, C., Ye, D., & Li, S. (2021). Don’t Do That!
Hunting Down Visual Design Smells in Complex UIs Against Design Guidelines. 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), 761–772. https://doi.org/10.1109/ICSE43902.2021.00075 Yang, S. T., Park, M. H., & Jeong, B. Y. (2020). Types of manual materials handling (MMH) and occupational incidents and musculoskeletal disorders (MSDs) in motor vehicle parts manufacturing (MVPM) industry. International Journal of Industrial Ergonomics, 77, 102954. https://doi.org/10.1016/j.ergon.2020.102954
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Interaction and Emerging Technologies (IHIET 2025)
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-73-1
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional