André Cardoso, Estela Bicho, Ana Braga, Carla Alves, Luís Louro, Duarte Fernandes, Pedro Arezes, Ana Colim (2024). Enhancing Worker Efficiency and Reducing Cognitive Workload Through Assistive Assembly: A Proof-of-Concept. In: Human Interaction and Emerging Technologies (IHIET-AI 2024): Artificial Intelligence and Future Applications. AHFE Open Access, vol. 120, pp. 71–80. https://doi.org/10.54941/ahfe1004557
Nowadays, fast-paced and competitive industry, assembly tasks demand significant levels of concentration, precision, and cognitive effort. These challenges often result in mental fatigue, errors, and decreased efficiency among workers. The concept of Assistive Assembly offers a promising solution, harmonizing human dexterity with cutting-edge technology. By integrating ergonomics and robotics, Assistive Assembly has the potential to provide invaluable support to assembly line workers, enabling them to achieve peak performance effortlessly. This study presents a proof-of-concept approach for a future Assistive Assembly, simulating this condition with human-human collaboration. This preliminary step aims to support a human-centered design for an assembly workstation comprising a human worker, a collaborative robot, and a video camera system. A total of 25 participants were recruited to perform a simulated window assembly task under two conditions: Assistive and Non-assistive. The NASA Task Load Index (NASA-TLX) and the number of errors committed were measured. The subjects reported significantly lower perceived cognitive workloads in the assistive condition. Related to the number of errors, a significant difference in median test scores between the two conditions was found, meaning a decrease in errors registered in the assistive setting. Although these preliminary results are promising, further development and testing are essential to refine the Assistive Assembly concept within collaborative robotics settings.
Currently, one pressing issue for companies has been related to the signif- icant physical burden on their workers along with its potential impact on product quality (Zare et al., 2016). Although manual assembly has histori- cally been physically demanding and continues to be so (Colim et al., 2021), another issue has arisen related to cognitive processing in complex assembly tasks (Wollter Bergman et al., 2021). Assembly tasks often demand unwa- vering concentration, precision, and significant cognitive effort (Bommer & Fendley, 2018). These challenges often result in cognitive workload, errors,
and decreased efficiency among workers (Tropschuh, Cegarra, and Battaïa, 2023). In the domain of Ergonomics and Human Factors, understanding cogni- tive workload and its impact on workers’ performance has emerged as one of the most critical challenges (Thorvald et al., 2019). It has become increas- ingly clear that cognitive overload, characterized by a state of high cognitive workload, can be detrimental to both human performance and safety (Biondi et al., 2021).
Direct measurement of cognitive workload is challenging due to its asso- ciation with internal processes in information processing. Consequently, researchers have developed several techniques/methods to measure mental effort, which serves as an indicator of cognitive workload (Atici-Ulusu et al., 2021). Cognitive workload is often measured using subjective and self-report measures. These measures are commonly employed either in isolation or together with performance measures, such as reaction time, number of errors, or accuracy, or physiological measures such as heart rate, eye tracking, or optical brain measures. In this field, subjective measures are the most applied due to their ease of use, cost-effectiveness, and minimal intrusion into work settings (Widyanti, Johnson, and de Waard, 2013), being the NASA Task Load Index (NASA-TLX) widely used (Grier, 2015). The importance of mental workload measurement has been further amplified by an increasing emphasis on safety, health, and worker comfort (Chenarboo et al., 2022).
Nowadays, the variety of products in manual assembly manufacturing implies several changes in assembly tasks, increasing their complexity (Wol- fartsberger, Hallewell Haslwanter, and Lindorfer, 2019). Novel mechanisms that provide support and flexibility for workers, decreasing their cogni- tive workload are needed. Assistive Assembly offers a promising solution to provide invaluable support to assembly line workers, enabling them to achieve high performance (Keshvarparast et al., 2023). Examples of Assistive Assembly technology include collaborative robots and instructive assistance systems, such as augmented reality systems (Wolfartsberger et al., 2019).
Related to instructive assistance systems, some studies have proved their effectiveness in decreasing the workers’ cognitive workload while enhancing their accuracy in specific assembly tasks (Funk, Kosch, and Schmidt, 2016; Vanneste et al., 2020). Relatively to the collaborative robots and their impact on workers’ cognitive workload, this is a topic that needs further research (Carissoli et al., 2023).
The contribution of collaborative robots to workers’ mental workload is a dynamic and context-dependent issue. While collaborative robots can promote the reduction of hazardous tasks, thereby promoting a reduced men- tal workload and increasing productivity (Welfare et al., 2019), they can also introduce new challenges. Recent studies have been focused on human- robot interactions intending to understand which factors influence workers’ cognitive workload and which solutions can be presented to optimize it (Baltrusch et al., 2022).
Therefore, it is essential to develop studies to maximize the benefits of collaborative robots while minimizing the cognitive workload. Based on this assumption, the current study presents a proof-of-concept approach to
Enhancing Worker Efficiency and Reducing Cognitive Workload 73
an Assistive Assembly workstation, simulating this condition with human- human collaboration. The focus of the study was to compare the cogni- tive workload and performance between two distinct assembly conditions: Non-assistive and Assistive. This preliminary step aims to support a human- centered design for a novel assembly workstation comprising a human worker, a collaborative robot, and a video camera system, that will allow the robot to get information about the human counterpart status, adapting its behavior accordingly.
METHODOLOGY
In a laboratory context, a window assembly task was replicated, based on a real-world industry scenario (as previously presented in Colim et al., 2023). This task consists of the assembly of three types of frames for windows with different dimensions, namely: 400 mm x 500 mm, 400 mm x 600 mm, and 500 mm x 600 mm. It should be noted that this task is associated with significant cognitive demands. Firstly, the assembly of these windows needs different components and requires a careful selection and proper alignment of these for successful assembly, adding complexity to the task. Secondly, all three windows shared common parts, thereby demanding participants to differentiate and correctly allocate components to the respective window type.
During the trials, the workbench was organized to provide some assembly components at the participants’ normal reach, considering relevant anthro- pometric data (Anacleto Filho et al., 2023). Other components were delivered by a human helper positioned in front of the participant, on the opposite side of the workbench (Figure 1). In front of the participants, a display presented the assembly instructions step-by-step (as exemplified in Figure 2), being this visualization entirely controlled by the participant through a command.
The primary goal of this work was to undertake a comparative analy- sis of cognitive workload and performance within two assembly conditions, namely Non-assistive and Assistive. Regarding the Non-assistive condition, participants did not receive active support from the human helper but rather had to assume control of the assembly performance. In this condition, the par- ticipants had to indicate the components needed, asking the human helper and the parts were delivered in a pre-established orientation, within their normal reach.
On the other hand, in the Assistive condition, the participants received active support in the following ways: components were delivered in the cor- rect assembly orientation, synchronized with the worker’s requirements, and on the respective mounting side. It is important to emphasize that the role of the human helper, has been designed with consideration to the features and constraints of a collaborative robot, as delineated in the simulation presented in our previous study (Colim et al., 2023). In addition, it is also important to clarify that, in the future, the human helper will be a collaborative robot, and these tests will help to define its role in creating a workstation with Human- Robot Collaboration (HRC) that minimizes both cognitive and physical load.
To minimize potential bias and enhance the rigor of the study, the assign- ment of participants to either the Assistive or Non-assistive conditions was randomized between experiments.
Figure 1: Setup of the experimental scenario.
A sample of 25 participants were considered. Participants were ran- domly recruited, considering researchers and students from a Portuguese university, balancing between genders. All participants in the study signed an Informed Consent Term in agreement with the Committee of Ethics for Research in Social and Human Sciences of the University of Minho (approval number CEICSH 038/2020), respecting the Declaration of Helsinki.
Each participant performed six trials, considering the Assistive and Non- assistive conditions and the three windows assembly (2 conditions for assis- tive mode x 3 windows). Then, to assess the cognitive workload NASA-TLX (Hart, 2006) and the number of errors committed were measured.
After each trial, participants reported their perceptions throughout the NASA-TLX, a well-established tool for assessing the perceived cognitive workload associated with the assembly task. This tool includes ratings from 5 to 100 points (Rossato et al., 2021), of six dimensions, namely mental demand, physical demand, temporal demand, own performance, effort, and frustration level (Evans & Fendley, 2017).
Relatively to the number of errors committed during the assembly task, these were systematically recorded using ZED 2i RGBD Camera by Stere- olabs®. These errors encompassed instances of choosing the incorrect part and assembling components in the wrong orientation.
Enhancing Worker Efficiency and Reducing Cognitive Workload 75
Figure 2: Example of a step-by-step assembly instruction.
Descriptive statistics and inferential analyses were used to compare the per- formance and cognitive workload between the Assistive and Non-assistive conditions, using SPPS® Version 29.0. The median was applied as the mea- sure of central tendency, and minimum and maximum were registered. For the data normality verification, we used the Shapiro-Wilk test, however, this condition was rejected. Based on this evidence, to test pairwise median differ- ences between the two conditions, the Wilcoxon test was applied. Significance was determined at p < 0.05.
RESULTS AND DISCUSSION
In the current study, as previously mentioned, a window assembly task in different conditions was considered, including 25 participants (with a mean age of 28.8 ± 6.5 years old, 11 male and 14 female), all of them right-handed and with no previous mental health issues.
The NASA-TLX results, including its six dimensions, are summarized in Table 1. These results demonstrate significant differences between the Non-assistive and Assistive conditions, providing valuable insights into the effectiveness of the Assistive Assembly. Regarding the NASA-TLX six dimen- sions, the findings denote several significant differences. Firstly, in terms of Mental Demand (MD), participants in the Assistive condition reported a significantly lower level of MD (median = 40.0 points) compared to the Non-assistive condition (median = 30.0 points) (p ≤0.001). This reduction in mental demand suggests that the Assistive Assembly has the potential to alleviate the cognitive workload.
Furthermore, the results indicate a substantial improvement in Perfor- mance (PE) in the Assistive condition, with participants reporting a signifi- cantly better PE score (median = 20.0 points) compared to the Non-assistive condition (median = 10.0 points) (p = 0.012). This finding underscores a positive impact on participants’ ability to perform the task efficiently. It has been shown that high mental demands are associated with a negative effect on workers’ performance (Biondi et al., 2021). Therefore, the opposite is also
true, that is lower cognitive workload can lead to better performance, as our results demonstrate.
Effort (EF) also exhibited a significant difference between the two con- ditions. In the Assistive condition, a reduced level of effort (median = 35.0 points) was reported, compared to the Non-assistive condition (median = 20.0 points) (p ≤0.001). This finding suggests that the Assistive Assembly effec- tively reduced the combined physical and cognitive EF required to complete the task within a certain perceived level of performance (Hart, 2006).
While several dimensions of cognitive workload demonstrated significant improvements in the Assistive condition, it is important to note that Phys- ical Demand (PD), Temporal Demand (TD), and Frustration (FR) did not yield significant differences between the two conditions. This suggests that the Assistive Assembly primarily impacted mental demand, task performance, and effort, while other aspects of the task remained relatively consistent.
Regarding the overall cognitive workload, the findings show significantly lower cognitive workload in the Assistive condition (median = 16.7 points) compared to the Non-assistive condition (median = 29.2 points) ( p = 0.008). This supports the evidence that assisting in the assembly resulted in a reduced cognitive workload.
Table 1. Median (min.; max.) NASA-TLX results, including its six dimensions (MD: Mental Demand; PD: Physical Demand; TD: Temporal Demand; PE: Perfor- mance; EF –Effort; and FR – Frustration.) and overall cognitive workload for Non-assistive and Assistive conditions. Significant results (p < 0.05) are denoted with *.
Table available in the original PDF paper.
MD PD TD PE EF FR Overall cognitive workload Non- assistive
Moreover, the results concerning the number of errors (Table 2) corrob- orate the overall NASA- TLX results. A significant reduction in errors was observed in the Assistive condition (median = 0.0 points) compared to the Non-assistive condition (median = 1.0 points) (p = 0.002). This highlights the efficacy of the Assistive Assembly in enhancing task accuracy. Accu- rate assembly is of utmost importance for improving the efficiency of the manufacturing process. When workers make mistakes, they may need to spend additional time correcting those mistakes, which can potentially impact assembly line efficiency and reduce productivity (Pimminger et al., 2021).
Globally, our findings underscore the advantages of Assistive Assembly in reducing MD and EF and enhancing task PE. In this condition, the results point out a decrease in the overall cognitive workload, and improving task accuracy. These results are in line with previous studies. For example,
Enhancing Worker Efficiency and Reducing Cognitive Workload 77
Vanneste et al., (2020) showed that Assistive Assembly employing augmented reality instructions, has the potential to cognitively support the workers dur- ing the assembly tasks, which can lead to better work quality, contributing to lower perceived complexity of the task. Additionally, Funk, Kosch, and Schmidt, (2016) introduced a work involving a projection-based assistance system. The results indicated a significantly lower perceived cognitive work- load and lower median number of errors in the group using the assistive system.
Table 2. Median (min.; max.) number of errors for non-assistive and assistive conditions.
Table available in the original PDF paper.
Non-assistive Assistive Median (min.; max.) 1.0 (0.0; 4.0) 0.0 (0.0; 1.0) Monte Carlo Sig. (1-tailed) 0.002
These outcomes are of utmost importance for the design and implementa- tion of assistive technologies aimed at enhancing user experiences and task efficiency in diverse contexts. Notwithstanding the promising nature of these initial findings, it is imperative to emphasize the need for further development and testing of the Assistive Assembly concept.
Future work should focus on employing more precise techniques for direct measurement of cognitive workload, ensuring a comprehensive assess- ment. Also, a biomechanical assessment is intended to evaluate the physical demands placed on workers during these assembly tasks. Globally, these data will enable the creation of an HRC workstation, essentially contributing to defining the robot’s behavior to minimize the workload in the assembly task. The definition of robot behavior taking into account the workload reduction, associated with a framework for real-time ergonomic assessment capable of continuously monitoring both the cognitive and physical conditions of the workers, will enable the creation of the next generation of workstations. It is crucial to emphasize that, in the future, the role of the human helper will be carried out by a collaborative robot. This role/behavior, in addition to tak- ing into account the results of the current study, will also take into account our previous developments in Dynamic Neural Field models for natural and efficient collaboration with human workers (Cunha et al., 2020; Erlhagen & Bicho, 2014; Silva et al., 2016; Wojtak et al., 2021, 2023). Examples of that include the ability of the robot to cope with dynamically changing joint action situations (Cunha et al., 2020), the ability to execute a shared human- robot task plan (Wojtak et al., 2021), the ability to recognize the emotional status of the worker and act accordingly (Silva et al., 2016), and the ability to close temporal coordination of actions and goals (Wojtak et al., 2023).
Within this future HRC workstation, worker data will be acquired through a vision system and seamlessly integrated into the robot’s architecture, ensur- ing that the robot’s actions align with the framework’s outcomes in real-time, which will allow to take into account the working conditions and evaluate posture as well as, cognitive variables, and extrapolate corrective measures. This endeavor will contribute toward advancing Assistive Assembly as a
transformative approach for improving efficiency and reducing cognitive and physical burdens faced by workers performing assembly industrial tasks.
ACKNOWLEDGMENT
This work was supported by FCT within Project I-CATER: Intelligent Robotic Coworker Assistant for industrial Tasks with an Ergonomics Ratio- nale, reference PTDC/EEI-ROB/3488/2021.
REFERENCES
Anacleto Filho, P. C., da Silva, L., Mattos, D., Pombeiro, A., Castellucci, H. I., Colim, A., Carneiro, P., & Arezes, P. (2023). Establishing an anthropometric database: A case for the Portuguese working population. International Journal of Industrial Ergonomics, 97. https://doi.org/10.1016/j.ergon.2023.103473 Atici-Ulusu, H., Ikiz, Y. D., Taskapilioglu, O., & Gunduz, T. (2021). Effects of
augmented reality glasses on the cognitive load of assembly operators in the auto- motive industry. International Journal of Computer Integrated Manufacturing, 34(5), 487–499. https://doi.org/10.1080/0951192X.2021.1901314 Baltrusch, S. J., Krause, F., de Vries, A. W., van Dijk, W., & de Looze, M. P.
(2022). What about the human in human robot collaboration?: A literature review on HRC’s effects on aspects of job quality. In Ergonomics (Vol. 65, Issue 5, pp. 719–740). Taylor and Francis Ltd. https://doi.org/10.1080/00140139.2021. 1984585 Biondi, F. N., Cacanindin, A., Douglas, C., & Cort, J. (2021). Overloaded and at Work: Investigating the Effect of Cognitive Workload on Assembly Task Performance. In HUMAN FACTORS (Vol. 63, Issue 5). Bommer, S. C., & Fendley, M. (2018). A theoretical framework for evaluating mental
workload resources in human systems design for manufacturing operations. Inter- national Journal of Industrial Ergonomics, 63, 7–17. https://doi.org/10.1016/j.er gon.2016.10.007 Carissoli, C., Negri, L., Bassi, M., Storm, F. A., & Delle Fave, A. (2023). Mental Workload and Human-Robot Interaction in Collaborative Tasks: A Scoping Review. International Journal of Human-Computer Interaction. https://doi.org/ 10.1080/10447318.2023.2254639 Chenarboo, F. J., Hekmatshoar, R., & Fallahi, M. (2022). The influence of phys-
ical and mental workload on the safe behavior of employees in the automobile industry. Heliyon, 8(10). https://doi.org/10.1016/j.heliyon.2022.e11034 Colim, A., Cardoso, A., Arezes, P., Braga, A. C., Peixoto, A. C., Peixoto, V.,Wolbert, F., Carneiro, P., Costa, N., & Sousa, N. (2021). Digitalization of Musculoskeletal Risk Assessment in a Robotic-Assisted Assembly Workstation. https://doi.org/10.3390/safety Colim, A., Cardoso, A., Bicho, E., Louro, L., Alves, C., Ribeiro, P., Pereira, D.,
Monteiro, S., Costa, N., Carneiro, P., Mould, S., Oliveira, J., Fernandes, D., & Arezes, P. (2023). On the development of an ergonomic approach for the design of an industrial robotic coworker. Human Interaction & Emerging Technologies (IHIET 2023): Artificial Intelligence & Future Applications, 111. https://doi.org/ 10.54941/ahfe1004004 Cunha, A., Ferreira, F., Sousa, E., Louro, L., Vicente, P., Monteiro, S., Erlhagen, W., & Bicho, E. (2020). Towards Collaborative Robots as Intelligent Co-workers in Human- Robot Joint Tasks: what to do and who does it? N ISR 2020; 52th International Symposium on Robotics, 1–8.
Enhancing Worker Efficiency and Reducing Cognitive Workload 79
Erlhagen, W., & Bicho, E. (2014). A dynamic neural field approach to natural and efficient human-robot collaboration. In Neural Fields: Theory and Applications (Vol. 9783642545931, pp. 341–365). Springer-Verlag Berlin Heidelberg. https: //doi.org/10.1007/978-3-642-54593-1_13 Evans, D. C., & Fendley, M. (2017). A multi-measure approach for connecting
cognitive workload and automation. International Journal of Human Computer Studies, 97, 182–189. https://doi.org/10.1016/j.ijhcs.2016.05.008 Funk, M., Kosch, T., & Schmidt, A. (2016). Interactive worker assistance: Comparing the effects of in-situ projection, head-mounted displays, tablet, and paper instructions. UbiComp 2016 - Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 934–939. https://doi.org/ 10.1145/2971648.2971706 Grier, R. A. (2015). How high is high? A meta-analysis of NASA-TLX global
workload scores. Proceedings of the Human Factors and Ergonomics Society, 2015-January, 1727–1731. https://doi.org/10.1177/1541931215591373 Hart, S. G. (2006). NASA-task load index (NASA-TLX); 20 years later. Proceedings of the Human Factors and Ergonomics Society, 904–908. https://doi.org/10.1177/ 154193120605000909 Keshvarparast, A., Battini, D., Battaia, O., & Pirayesh, A. (2023). Collabora-
tive robots in manufacturing and assembly systems: literature review and future research agenda. In Journal of Intelligent Manufacturing. Springer. https://doi.or g/10.1007/s10845-023-02137-w Pimminger, S., Kurschl, W., Panholzer, L., & Schönböck, J. (2021). Exploring the Learnability of Assembly Tasks Using Digital Work Instructions in a Smart Factory. Procedia CIRP, 104, 696–701. https://doi.org/10.1016/j.procir.2021.11. 117 Rossato, C., Pluchino, P., Cellini, N., Jacucci, G., Spagnolli, A., & Gamberini, L.
(2021). Facing with Collaborative Robots : The Subjective Experience in Senior and Younger Workers. Cyberpsychology, Behavior, and Social Networking, 24(5), 349–356. https://doi.org/10.1089/cyber.2020.0180 Silva, R., Louro, L., Malheiro, T., Erlhagen, W., & Bicho, E. (2016). Combining intention and emotional state inference in a dynamic neural field architecture for human-robot joint action. Adaptive Behavior, 24(5), 350–372. https://doi.org/10. 1177/1059712316665451 Thorvald, P., Lindblom, J., & Andreasson, R. (2019). On the development of a
method for cognitive load assessment in manufacturing. Robotics and Computer- Integrated Manufacturing, 59(April 2018), 252–266. https://doi.org/10.1016/j.rc im.2019.04.012 Tropschuh, B., Cegarra, J., & Battaïa, O. (2023). Integrating physiological and mental aspects in employee scheduling: an overview for practitioners in production management. International Journal of Production Research. https://doi.org/10. 1080/00207543.2023.2217278 Vanneste, P., Huang, Y., Park, J. Y., Cornillie, F., Decloedt, B., & Van den
Noortgate, W. (2020). Cognitive support for assembly operations by means of augmented reality: an exploratory study. International Journal of Human Computer Studies, 143. https://doi.org/10.1016/j.ijhcs.2020.102480 Welfare, K. S., Hallowell, M. R., Shah, J. A., & Riek, L. D. (2019). Consider the Human Work Experience when Integrating Robotics in the Work- place (ACM/IEEE international conference on human-robot interaction, Ed.; pp. 75–84).
Widyanti, A., Johnson, A., & de Waard, D. (2013). Adaptation of the Rating Scale Mental Effort (RSME) for use in Indonesia. International Journal of Industrial Ergonomics, 43(1), 70–76. https://doi.org/10.1016/j.ergon.2012.11.003 Wojtak, W., Ferreira, F., Louro, L., Bicho, E., & Erlhagen, W. (2023). Adaptive timing
in a dynamic field architecture for natural human–robot interactions. Cognitive Systems Research, 82. https://doi.org/10.1016/j.cogsys.2023.101148 Wojtak, W., Ferreira, F., Vicente, P., Louro, L., Bicho, E., & Erlhagen, W. (2021).
A neural integrator model for planning and value-based decision making of a robotics assistant. Neural Computing and Applications, 33(8), 3737–3756. https: //doi.org/10.1007/s00521-020-05224-8 Wolfarts Berger, J., Haslwanter, J. D. H., & Lindorfer, R. (2019). Perspectives on Assistive Systems for Manual Assembly Tasks in Industry. Technologies, 7(1). https://doi.org/10.3390/technologies7010012 Wollter Bergman, M., Berlin, C., Chafi, M. B., Falck, A. C., & Örtengren, R.
(2021). Cognitive ergonomics of assembly work from a job demands–resources perspective: Three qualitative case studies. International Journal of Environmental Research and Public Health, 18(23). https://doi.org/10.3390/ijerph182312282 Zare, M., Croq, M., Hossein-Arabi, F., Brunet, R., & Roquelaure, Y. (2016).
Does Ergonomics Improve Product Quality and Reduce Costs? A Review Arti- cle. Human Factors and Ergonomics In Manufacturing, 26(2), 205–223. https: //doi.org/10.1002/hfm.20623
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Interaction and Emerging Technologies (IHIET-AI 2024): Artificial Intelligence and Future Applications
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-96-4
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional