Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Evaluation of Feedback in Manual Assembly Assistance Systems

1 Technology Faculty, University of Brasilia, Brasilia-DF, 70910-900, Brazil ↗
2 Chair of Production Metrology, Quality Management, and Information Management in Mechanical Engineering - Laboratory for Machine Tools and Production Engineering (WZL) of RWTH Aachen University, Aachen, 52074, Germany ↗
Publication year
Digital Object Identifier10.54941/ahfe1005645

Abstract

Despite the technological advancements of Industry 4.0 and automation in many industries, the variability and complexity of products to meet market demands require a level of flexibility that is not yet achieved with machinery. Consequently, manual assembly processes have become the core of manufacturing in organizations that aim to keep up with the accelerated pace of market growth. However, increased flexibility and manual assembly have the disadvantage of increased manufacturing errors, which are more likely due to the complexity of processes, operator fatigue, etc. This paper highlights the crucial role of feedback in the assembly process, presenting an evaluation of human operator performance using a simulation of two types of intelligent assembly assistance systems, one that only provides task instructions and another that, in addition to instructions, displays errors in task execution. A 3D-printed toy truck model was used to simulate assembly. As a result, a total of 12 participants participated in the experiment. The research primarily evaluates the metrics of assembly completion time and the number of errors. Data analysis suggests a difference in the two groups' assembly performance. The group of participants whose assistance system provided feedback on errors appears to have been more efficient, taking less time to recover from errors.

INTRODUCTION

As modern manufacturing systems strive to offer more variants per model and reduce the introduction and development times of new products to maintain their competitive edge in today’s fast-paced market, the complexity of assembly lines increases considerably. Consequently, providing assembly instructions at workplaces has become a major challenge for industrial environments, especially considering that investigations have revealed that the assembly process accounts for approximately 50% of the total production development time. Although advances in automation technology

© 2024. Published by AHFE Open Access. All rights reserved. 783

are significant, the degree of complexity of some products means that certain assembly operations continue to be carried out manually. This paper aims to address this challenge by evaluating the potential impact of providing feedback on errors in improving human operator performance and efficiency in manual assembly processes.

Even though industries have developed some systems to design printed instructions on paper and based on the experience and previous knowledge of process engineers or the traditional introduction of more experienced colleagues to newer ones, the benefits of these mechanisms are quite limited. This is because paper instructions quickly become obsolete and must be replaced with each change in the product lines being assembled, experienced workers are not always available, and new instructions need to be memorized, which can considerably affect employee performance (Funk and Schmidt, 2015). In this sense, to ensure the necessary flexibility in this type of process without reducing the level of product quality, intelligent assistance systems have been implemented that automatically adapt to changes in the process and are capable of detecting, monitoring, characterizing, and assisting workers in decision-making (Rodríguez-Gasca et al., 2024; Thamm et al., 2021). In this scenario, one of the technologies that stands out as a tool for optimizing the information transfer process is augmented reality (AR). AR allows visual information to be integrated directly into the work environment. Compared to reading paper manuals, it has proven effective in facilitating understanding, especially when performing complex tasks, and leading to more efficient manual operations (Büttner et al., 2017; Heine et al., 2023). Therefore, incorporating any of the Industry 4.0 enabling technology applications within industries requires certain conditions and maturity levels to function properly (Dantas & Barbalho, 2020) and avoid the islands of improvement phenomena. The design of the user interfaces, as well as the assembly instructions, need to be flexibly adapted to satisfy the specific needs of the users of the system. This is crucial to efficiently use the strengths of both the competencies of the human profile and the advantages of technologies (Fischer et al., 2017; Metzmacher et al., 2019). Generally, the instructional systems do not adequately recognize the different stages of learning, different forms of assembly, or the way of using it is complex, which can lead to ineffective orientations in situations where assistance becomes an obstacle rather than a resource, compromising or operative performance and consequently the results of the process (Funk et al., 2015; Kosch et al., 2017).

Accordingly, this research was proposed where we carried out an experiment using the simulation of a projection system that provides visual instructions and generates alerts to the experiment participants to guide the assembly of the parts of a 3D printed truck or provide feedback when an error was made. The structure of the proposed system is conceived by (Thamm et al., 2021) according to Figure 1. The experiment sought to compare assembly efficiency between two groups: one group received only assembly instructions, and the other group received the instructions and automated feedback for assembling errors.

Evaluation of Feedback in Manual Assembly Assistance Systems 785

Figure 1: Diagram of an assembly assistance system using augmented reality (adapted from Thamm et al., 2021).
Original PDF Figure 1:

The remaining paper is organized as follows: Section 2 illustrates the applied research methodology. Based on the practical experiment performed, Section 3 briefly describes and discusses the results of the proposed solution. Finally, the paper concludes with the study’s conclusions and formulates implications for future research.

METHODOLOGY

This methodology describes the design of the experiment to compare the two modes of the system and evaluate the time spent on the assembly task and the user error rate.

System Design To understand how feedback is perceived when provided by a manual assembly assistance system (AAS), the experiment was conducted based on the methodology proposed by (Markus et al., 2024) which would mimic an assembly assistance system providing feedback during a process of assembly compared to a system where the same process would be executed without a system’s help. A room was created where a researcher could be hidden from the volunteer participant using fake walls. This setup was developed so that the participant would not suspect that a person was controlling the system, and the perception of feedback would take the form of a computerized system. In addition to the hidden researcher, another person oversaw the description of how the experiment would be executed to the participants. The layout of the system structure for the experiment can be seen in Figure 2.

The physical system was composed of two workbenches separated by improvised walls. A laser projector (WEMAX Model L032FGN, 16:9, 1080p) and a webcam (Logitech C505e Webcam Model 960-001372, 720p). In addition, one of the workbenches had a rectangular area to delimit where the participant had to assemble the object to be captured by the camera. The product chosen to be assembled consisted of a toy truck with movable and

interchangeable parts made with 3D printing. The image of the truck can be seen in Figure 3.

Figure 2: System design for the experiment.
Original PDF Figure 2:
Figure 3: The product was chosen to be assembled (with instructions in Portuguese).
Original PDF Figure 3:

The projector projected the system’s graphical user interface onto the workbench at an angle slightly above the participant’s head. This interface consisted of assembly instructions designed in PowerPoint that were projected and manipulated from the computer by the hidden researcher (Figure 4). The hidden researcher followed the assembly sequence executed by the participant step by step through the real-time camera images connected to the computer, allowing the system to simulate rapid responses to the participant’s interactions with the interface.

It was enough to touch the arrows positioned at the bottom of the workbench to manipulate the interface. An arrow on the left side serves to go back to the previous step, and another on the right side allows one to advance to the next.

Evaluation of Feedback in Manual Assembly Assistance Systems 787

Figure 4: User interface of the system.
Original PDF Figure 4:

Participants A total of 12 voluntary participants (men and women) were recruited at the University of Brasilia, Brazil, using convenience sampling. Subjects were selected because they had no previous experience in assembly tasks. The total average age was 24 years. One of the participants was a Doctor of Medicine, and the other 11 were engineering students. All of them were interested and favorable towards innovative technologies. Considering this background, the assembly tasks of the experiment had a basic level of complexity (i.e., no complex parts or tool usage) so that the experience and motivation of the participants would not affect the results.

Procedure To experiment, six participants were assigned to first perform the mode that only showed instructions (control group) without providing error feedback. Then, the other six were assigned to perform the mode that showed feedback (intervention group). They were told the objective of the experiment, the image of the product they were to assemble, and to complete each step according to the instructions shown on the interface. In addition, they were given brief training on navigating the interface and the visual and textual information provided by the system. No specific time was stipulated for assembling the product to prevent time pressure from influencing performance.

In the control group, the slide presentation moved forward or backward when a volunteer participant pressed any buttons. For the intervention group, the presentation moved forward if the assembly was corrected for each step completed. However, if the assembly was made with any mistake, the presentation went to an error page until the participant corrected the error. An example of how the error message was displayed can be seen in Figure 5.

The assembly sequence was based on a sequence that considered common tasks such as part location, component identification, instruction identification, and execution of simple movements such as turning, inserting, sliding, screwing, pushing, and connecting. The assembly sequence was the

same for both groups, the only difference being the instantaneous error feedback for the intervention group.

Figure 5: An example of an error page containing the message ‘The cabin platform is larger than the cargo bay platform’ in Portuguese.
Original PDF Figure 5:

The system with feedback was designed to detect seven possible errors, plus a generic error for cases where specific errors could not be included. The feedback related to the errors contemplated by the system were the following:

1. Generic error: used when the identified error does not match the predicted set of assembly errors. 2. “The cabin platform is smaller than the container platform”: used when the participant mistakes the cabin platform for the cargo bay platform. 3. “Wrong Direction, Slide the cabin through the other direction”: used when a participant slides the cabin in the wrong direction. 4. “Assemble the other three wheels on the other three axles”: This is used when the participant does not attach one wheel to each axle. 5. “Wrong Direction: Slide the container through the other direction”: This is used when the participant slides the container in the wrong direction. 6. “The screw should be assembled bottom-up”: used when the participant uses the screw in the wrong direction. 7. “The container door is slightly smaller than the cabin doors”: used when the participant mistakes the container door for one of the cabin doors. 8. “The cabin door is slightly bigger than the others”: used when the participant mistakes the cabin doors for the cabin ceiling or container door.

Each participant run was recorded and analyzed to ascertain the assembly completion time. The research assistant recorded the types of errors, their causes, and the number of times they were made. The research assistant observed the participant from a distance without impeding their movements. After completing the test, participants were asked to answer a questionnaire to obtain subjective evaluations of the system’s usability.

Evaluation of Feedback in Manual Assembly Assistance Systems 789

RESULTS

This section presents the results of the time spent completing the entire toy truck assembly task, errors in task execution, and participants’ subjective perception of system usage. Individual results for both groups (control group and intervention group) can be seen in Table 1 below.

Table 1. Results from each participant in both groups.

Table available in the original PDF paper.

Group Without System Feedback (Control) No. Age No. errors Error Types Total Time Spent (s) Reached End State?

-

1 20 0 - 401 Yes 2 24 5 1,1,2,6,7 477 Yes 3 23 0 - 443 Yes 4 21 7 2,1,1,1,1,1,1 1085 No 5 22 7 1,1,2,1,1,1,1 1574 No 6 32 2 1,1 575 Yes Mean (std) 23.7 (4.32) 3.5(3.27) - 759,17 (471.57) Group with system feedback (intervention)

N. Age N. errors Error Types Total Time Spent (s)

Reached end state?

-

7 23 1 5 235 Yes 8 25 0 - 240 Yes 9 25 5 4,5,1,6,7 496 No 10 24 4 3,5,6,1 371 Yes 11 27 4 3,1,1,7 741 Yes 12 22 4 4,1,7,1 434 Yes Mean (std) 24.17 (1.94) 3(2) - 419.50 (188.70)

A total of three participants weren’t able to reach the end state and decided to abort. Their time spent was stopped the moment they confirmed their intention to finish. Two belong to the control group, and the last belongs to the intervention group.

Time Spent The total time spent during the assembly was more divergent between both groups. The intervention group was 52.07 seconds faster on average, with a significantly lower standard deviation, meaning that the intervention group was more consistent in time spent. Figure 6 presents a chart to better visualize the variability of these results.

The dispersion of time spent on data in the control group is greater than in the intervention group. However, if we only take into consideration the participants that did reach the end state, the control group has an average time spent of 474 seconds with a standard deviation of 74.16 seconds, and

the intervention group has an average of 404.20 seconds with a standard deviation of 202.77 seconds.

Figure 6: Time spent in complete assembly.
Original PDF Figure 6:

The total time spent is an indirect measure of a feedback system’s impact as it informs the participants of mistakes. In contrast, the control group must rely on their senses and comparison to the final state expected. Therefore, we posit that a feedback system directly reduces time spent due to a reduction in error recovery because error detection stops the user from compounding on errors previously made and avoids scenarios where the user has to disassemble the product to reach the state where the mistake happened.

Error Occurrence The impact of the feedback system on the accuracy of the assembly tasks was evaluated through the average number of errors by each group, as shown in Table 1. The average number of errors in the control group was 3.5, with a 3.27 standard deviation, and the average number of errors in the intervention group was 3, with a standard deviation of 2. This shows that feedback effects are more pronounced after users diverge from the instructions, but their performance was considered similar, given that the participants did not have experience in assembly. Figure 7 shows the dispersion of the number of errors in each group through a box plot.

The results suggest that the dispersion of the control group’s data is greater than that of the intervention group and that there are more errors.

We also found that the participants often made mistakes not predicted by the authors (categorized as 1), and the nature of these mistakes had no apparent pattern.

Evaluation of Feedback in Manual Assembly Assistance Systems 791

Figure 7: Dispersion of errors occurrence.
Original PDF Figure 7:

CONCLUSION

The experiment presented valuable information for assembly assistance systems. The main finding on the effect of feedback on time spent, particularly on error recovery, was important, given the purpose of such systems. Future works can further improve our findings by evaluating the media and language used to give feedback to users and further investigating feedback on users who have previous experience (such as repeating the experiment with the same product or with a different product would generate similar results) as well as conducting more experiments in successively to observe the learning process of skilled and unskilled workers.

ACKNOWLEDGMENT

The authors wish to acknowledge the contribution from the foundation for research support of the Federal District (FAP-DF), the University of Brasilia, and the Mechatronic Systems Postgraduate Program for their support in participating in this event.

REFERENCES

Büttner, S., Sand, O., & Röcker, C. (2017). Exploring Design Opportunities for Intelligent Worker Assistance: A New Approach Using Projetion-Based AR and a Novel Hand-Tracking Algorithm (pp. 33–45). https://doi.org/10.1007/978-3- 319-56997-0_3 Dantas, R. F. Barbalho, S. C. M.. The effect of islands of improvement on the maturity

models for Industry 4.0: the implementation of an inventory management system in a beverage factory. Brazilian Journal of Operations & Production Management, v. 18, p. e20211119-17, 2020.

Fischer, H., Engler, M., & Sauer, S. (2017). A Human-Centered Perspective on Software Quality: Acceptance Criteria for Work 4.0 (pp. 570–583). https://doi. org/10.1007/978-3-319-58634-2_42 Fu, M., Fang, W., Gao, S., Hong, J., & Chen, Y. (2022). Edge computing-driven

scene-aware intelligent augmented reality assembly. The International Journal of Advanced Manufacturing Technology, 119(11–12), 7369–7381. https://doi.org/ 10.1007/s00170-022-08758-4 Funk, M., Dingler, T., Cooper, J., & Schmidt, A. (2015). Stop helping me - I’m bored!: why assembly assistance needs to be adaptive. Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers - UbiComp ‘15, 1269–1273. https://doi.org/10.1145/2800835.2807942 Funk, M., & Schmidt, A. (2015). Cognitive Assistance in the Workplace. IEEE

Pervasive Computing, 14(3), 53–55. https://doi.org/10.1109/MPRV.2015.53 Heine, I., Hellebrandt, T., Huebser, L., & Padrón, M. (2023). Hybrid Intelligence: Augmenting Employees’ Decision-Making with AI-Based Applications. In Handbook of Human-Machine Systems (pp. 321–332). Wiley. https://doi.org/10. 1002/9781119863663.ch27 Kosch, T., Abdelrahman, Y., Funk, M., & Schmidt, A. (2017). One size does

not fit all: challenges of providing interactive worker assistance in industrial settings. Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers, 1006–1011. https://doi.org/ 10.1145/3123024.3124395 Markus, A., Marleen Daling, L., Borowski, E., & Isenhardt, I. (2024). Error Communication in Manual Assembly through a Projection-Based Assistance System. https://doi.org/10.54941/ahfe1005365 Metzmacher, A. I., Hellebrandt, T., Ruessmann, M., Heine, I., & Schmitt, R.

H. (2019). Aligning the Social Perspective with the Technical Vision of the Smart Factory. In Advances in Production Research (pp. 715–729). Springer International Publishing. https://doi.org/10.1007/978-3-030-03451-1_69 Michalos, G., Makris, S., Papakostas, N., Mourtzis, D., & Chryssolouris, G. (2010).

Automotive assembly technologies review: Challenges and outlook for a flexible and adaptive approach. CIRP Journal of Manufacturing Science and Technology, 2(2), 81–91. https://doi.org/10.1016/j.cirpj.2009.12.001 Rodriguez Gasca, M., & Macedo Barbalho, S. (2024). Exploring the Influence of Industry 4.0 Technologies on Workplace Dynamics: A Literature Analysis. https://doi.org/10.54941/ahfe1005149 Rodríguez-Gasca, M., Corvelho, F. M., & Macêdo Barbalho, S. C. (2024,

May 9). A Bibliometric Review on Augmented Intelligence Systems Enabled by Industry 4.0 Technologies. Proceedings of the International Conference on Industrial Engineering and Operations Management. https://doi.org/10.46254/SA 05.20240084 Thamm, S., Huebser, L., Adam, T., Hellebrandt, T., Heine, I., Barbalho, S., Velho, S. K., Becker, M., Bagnato, V. S., & Schmitt, R. H. (2021). Concept for an augmented intelligence-based quality assurance of assembly tasks in global value networks. Procedia CIRP, 97, 423–428. https://doi.org/10.1016/j.procir.2020.05. 262 Xiao, H., Duan, Y., Zhang, Z., & Li, M. (2018). Detection and estimation of mental

fatigue in manual assembly process of complex products. Assembly Automation, 38(2), 239–247. https://doi.org/10.1108/AA-03-2017-040.

Cover of the proceedings volume Human Factors in Design, Engineering, and Computing

Explore the complete digital volume

Human Factors in Design, Engineering, and Computing

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-964867-35-9

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →