Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

A Comparative Evaluation of Assistance Systems for an Instrument Reprocessing Workbench

Chair of Medical Engineering, RWTH Aachen University, Aachen, Germany ↗
Publication year
Digital Object Identifier10.54941/ahfe1004366

Abstract

IntroductionDigital assistant systems (DAS) can support tasks by mediating between complex data and users, promoting continuous learning and on-the-job training (Jwo et al., 2021; Longo et al., 2017; Prinz et al., 2017). Integrating a DAS into surgical instrument reprocessing, where instrument-specific and complex manual tasks need to be performed strictly according to the manufacturer's instructions, can be advantageous. In Germany, reprocessing by trained on-the-job personnel is common. In addition, human factors are often neglected in the instructions for instrument reprocessing (Choi et al., 2017). A cooperative robot can be a valuable addition to mitigate health risks associated with handling contaminated surgical instruments during reprocessing, resulting in a cyber-physical assistance system (Heibeyn et al., 2021). However, it is unclear how the transition from paper-based instructions to either a DAS or a DAS supplemented with a cooperative robot assistance (“cyber-physical assistance system” – CPAS) affects usability for complex and workpiece-specific tasks in instrument reprocessing. Therefore, this study aimed to investigate the differences in usability with different assistance systems for typical tasks in instrument reprocessing for untrained personnel.MethodsWe conducted an interaction-centered user study with 13 participants unfamiliar with the reprocessing tasks. The participants were asked to complete typical reprocessing tasks three times in a random order, using different assistance approaches each time. The reference process consisted of paper-based instructions that required stepping away from the workstation to retrieve information from the storage of guidelines, simulating common set-ups of current reprocessing processes. The first assistance approach presented digital instructions right at the workstation. The second assistant combined a cooperative robot with digital instructions. The robot performed simple processing steps, while the human operator focused on the complex tasks and verified the cleaning success. We measured the required time, counted user errors and rated the criticality, measured the perceived workload using the NASA-TLX questionnaire (NASA, 2020), and documented the remarks of the participants using the thinking-out-loud method for all assistance systems.Results The NASA-TLX did not reveal any significant differences among the three systems, however, the CPAS reduced the number of critical errors. The errors included omitted processing steps and deviations from required times, which could pose risks to patients. The DAS was perceived as a suitable way to check instructions, based on the participants’ comments. However, in the case of the CPAS, some participants missed or ignored messages provided by the user interface.Discussion and ConclusionIn summary, the CPAS improved usability the most, improving effectiveness (number of errors) while maintaining the same efficiency (total duration). Although our study found promising results for integrating a DAS or CPAS into on-the-job training assistance for novice personnel, future studies should compare the results obtained from inexperienced to those of experienced users to fully assess the usability of related approaches. This study contributes to the field of human factors by providing comparative data on usability across different levels of assistance for complex and workpiece-specific tasks in surgical instrument reprocessing.ReferencesChoi, J., Seraphina, S., & Knudsen, K. (2017). The Clean and Dirty of Redesigning Reprocessing Instructions for Use. Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care, 6(1), 150–153. https://doi.org/10.1177/2327857917061032Heibeyn, J., König, N., Domnik, N., Schweizer, M., Kinzius, M., Janß, A., & Radermacher, K. (2021). Design and Evaluation of a Novel Instrument Gripper for Handling of Surgical Instruments. Current Directions in Biomedical Engineering, 7(1), 1–5. https://doi.org/10.1515/cdbme-2021-1001Jwo, J.‑S., Lin, C.‑S., & Lee, C.‑H. (2021). An Interactive Dashboard Using a Virtual Assistant for Visualizing Smart Manufacturing. Mobile Information Systems, 2021, 1–9. https://doi.org/10.1155/2021/5578239Longo, F., Nicoletti, L., & Padovano, A. (2017). Smart operators in industry 4.0: A human-centered approach to enhance operators’ capabilities and competencies within the new smart factory context. Computers & Industrial Engineering, 113, 144–159. https://doi.org/10.1016/j.cie.2017.09.016NASA (Ed.). (2020, December 15). NASA TLX: Task Load Index. https://humansystems.arc.nasa.gov/groups/TLX/Prinz, C., Kreimeier, D., & Kuhlenkötter, B. (2017). Implementation of a Learning Environment for an Industrie 4.0 Assistance System to Improve the Overall Equipment Effectiveness. Procedia Manufacturing, 9, 159–166. https://doi.org/10.1016/j.promfg.2017.04.004

INTRODUCTION

Digital assistant systems (DAS) can support tasks by mediating between com- plex data and users, promoting continuous learning and on-the-job training (Jwo et al., 2021; Longo et al., 2017; Prinz et al., 2017). Integrating a DAS into surgical instrument reprocessing, where instrument-specific and com- plex manual tasks must be performed strictly according to the manufacturer’s instructions, can be advantageous (Jolly et al., 2013). In Germany, reprocess- ing of surgical instruments by trained on-the-job personnel is common. In

© 2023. Published by AHFE Open Access. All rights reserved. 146

A Comparative Evaluation of Assistance Systems 147

addition, human factors are often neglected in the instructions for instru- ment reprocessing (Choi et al., 2017). A cooperative robot can be valuable in mitigating health risks associated with handling contaminated surgical instru- ments during reprocessing, resulting in a cyber-physical assistance system (Heibeyn et al., 2021). However, it is unclear how the transition from paper- based instructions to either a DAS or a DAS supplemented with cooperative robot assistance (“cyber-physical assistance system” – CPAS) affects usability for complex and workpiece-specific tasks in instrument reprocessing. There- fore, this study aimed to investigate the differences in usability with different assistance systems for typical tasks in instrument reprocessing for untrained personnel.

DESIGN OF THE ASSISTANCE SYSTEMS To investigate the influence of different assistance systems, we simulated typi- cal manual activities in the reprocessing of medical devices. Manual activities are mainly used for complex medical devices such as endoscopes and include rinsing, brushing, disassembling, and placing them in a washing basin for a defined time (Jolly et al., 2013). Hildebrand et al. identified three main issues in processing complex medical devices: lack of feedback, high mem- ory demand, and lack of visibility (Hildebrand et al., 2010; Jolly et al., 2013). Lack of visibility can be caused by insufficient visualization of process- ing steps in paper-based instructions. High memory demand arises from the variety of complex instruments in a single reprocessing facility, each requir- ing instrument-specific processing steps and related equipment (brushes of different sizes, connectors). Especially for instruments with cavities, it can be impossible to verify whether the cleaning of all surfaces was success- ful and whether the processing steps were performed in the correct order (Jolly et al., 2013).

Figure 1: Design of the digital step-by-step instruction used for CPAS, showing the cooperative processing: the robot positions the trocar sleeve, and the human operator brushes the respective surfaces.
Original PDF Figure 1:

We have considered these issues in the development of the DAS by allowing the user to select surgical instruments from the set currently being processed to visualize step-by-step instructions (Figure 1). The instruction included large-scale photos of the processing step performed on the current instru- ment. The DAS also pointed out required processing times but required the

user to use a separate timer. A touchscreen allowed intuitive navigation of the DAS. We followed the DIN EN ISO 9241 standard family for information presentation and user interface elements.

While introducing a cooperative robot into a work system can relieve the human operator by taking over dangerous or physically demanding tasks, cooperation with a robot can also require increased attention and concen- tration (Gerst, 2020). Predictability and announcement of robot movements can improve perceived safety (Ikeura et al., 2003). In addition, the coop- erative robot might need the help of a human operator for some tasks, for example, to disassemble an instrument. Therefore, we have expanded the functionality of the DAS when integrating the robot to allow the commu- nication of humans and robot through the same interface used to assist the human operator. The resulting CPAS specifies a fixed sequence of processing steps. Humans can work in parallel as the robot takes over some processing steps. Some more complex steps are performed cooperatively. For example, during brushing, the robot holds the instrument in a suitable position to avoid human contact with contaminated instruments. As in the DAS, pictures and descriptions guide each step. However, the sequence of processes is much more strictly specified, as coordination with the robot is required. In addi- tion to the information provided by the DAS, the CPAS provides information about the current activity of the robot. The robot’s information is color-coded to indicate the activity or waiting position of the robot and match the color of the LEDs on the robot. A warning is displayed before the robot starts mov- ing and needs to be acknowledged in the interface. The robot can also call the human operator for cooperative tasks via the user interface. The human operator can decide when to stop the current task to interact with the robot.

EXPERIMENTAL SETUP To evaluate the usability of different assistance systems for reprocessing com- plex medical devices by untrained personnel, we used three scenarios in which the participants performed the same tasks with three forms of assistance. In addition to the DAS and the CPAS, we used paper-based instructions that required stepping away from the workstation to retrieve information from the storage of guidelines, simulating typical setups of current reprocessing units.

To test the integration of the robot into the work system, we used a two-part task consisting of a trocar and a magnetic structure (Figure 2) by Geomag, Geomagworld, Switzerland. The trocar requires disassembly into two parts, brushing, flushing, and immersion into a cleaning solution. The magnetic structure resembles the cognitively demanding task of disassem- bling and preparing an endoscope, as the task requires a high level of focus on visual and haptic feedback and needs many steps that must be performed in the correct order. The two-part task, consisting of processing the trocar and assembling the magnetic structure, stayed the same for all three levels of assistance. The robot performed part of the tasks on the trocar with the CPAS.

A Comparative Evaluation of Assistance Systems 149

Figure 2: The magnetic structure: On the left, the correct configuration is shown (trian- gular part on top of square part); on the right, the faulty configuration which occurred is shown (triangle over triangle). The color of the surfaces did not matter for the experimental procedure.
Original PDF Figure 2:

The test setup consisted of a workstation containing areas marked to rep- resent a washing basin, a rinsing system, an area for brushing, a robot, and an area for the manual assembly of the magnetic structure (Figure 3). The participants could interact with the two digital assistance systems using a touchscreen placed in the assembly area. The paper-based instructions were placed two meters from the workstation (not visible in Figure 3), similar to the typical setup in conventional instrument processing scenarios.

Figure 3: Setup of the workstation used for the user study as seen by the participants.
Original PDF Figure 3:

We conducted the interaction-centered user study with 13 participants unfamiliar with the reprocessing tasks. All participants were students or researchers, with a mean age of 27. While the sequence of assistance systems was randomly appointed to each participant, we ensured each sequence was performed similarly frequently across all participants. We measured the time required to complete each task, counted user errors and rated the criticality of each error, measured the perceived workload using the NASA-TLX ques- tionnaire (NASA, 2020), and documented the participants’ remarks using the thinking-aloud method.

A previously performed risk analysis determined the criticality of the errors. Before interacting with the digital user interface, all participants received an explanation about how it works. The participants filled out a questionnaire after finishing each of the three tests. The results of the NASA- TLX were investigated for significant differences between the three assistance systems (α = 0.05) using the Friedman Test (Friedman, 1937).

RESULTS OF THE USER STUDY The NASA-TLX revealed no significant differences among the three assis- tance systems (Figure 4). The effect size using Cohen’s d indicates minor effects between the three groups with all d < 0,2 (paper-based – DAS d = 0,02, paper-based – CPAS d = 0,03, DAS – CPAS d = 0,01) (Cohen, 1988). Nevertheless, the boxplot indicates that the CPAS poses the lowest task load.

The participants reported high usefulness and comprehensibility for the DAS and CPAS. Participants mentioned that they preferred the more lin- ear and strict guidance of the CPAS system, which presented them with a fixed sequence of processing steps, while the DAS allowed them to choose the sequence. Especially during the parallel processing steps, e.g., continuing to work while one part is in the washing basin, the participants mentioned that there might be an optimal solution to minimize the total time for the repro- cessing process. Participants mentioned that the hints about frequent errors in the digital guidelines helped them to be more careful about the mentioned steps. Most participants requested an option to start multiple digital timers in the user interface to have all the information in one spot and always in view.

Figure 4: Results of the NASA-TLX task load measurement.
Original PDF Figure 4:

While the task load showed no significant difference, we observed that the CPAS reduced the number of critical errors. The errors included omitted processing steps and deviations from required times. We did only count a too- short processing time as an error while accepting a deviation up to 10% from the required time as acceptable. Due to the strict sequence of processes, the

A Comparative Evaluation of Assistance Systems 151

CPAS limited the deviations from the displayed processing times, and we did not observe any participants disregarding the system’s guidelines (Table 1).

Table 1. Number of observed user errors across all 13 participants.

Table available in the original PDF paper.

Assistance System Paper-based DAS CPAS Deviations from process time 9 14 None No movement in the washing basin 4 3 None No vertical position during flushing 2 2 None Faulty magnetic structure 6 6 6 Trocar not correctly passed to the robot None None 1 Part of the trocar held by the robot was not brushed None None 9

Another error occurred before placing part of the trocar in the washing basin. The guidelines stated that the trocar had to be moved in the basin to allow air to escape from the hollow part of the instrument. Missing this step was counted as a critical error, as it can lead to an incomplete washing, possibly resulting in contamination remaining in the hollow part – which makes successful sterilization impossible and poses a patient risk (Wismer & Zanette, 2020). Two participants did not hold the trocar in a vertical position during the flushing (paper-based and DAS), which can lead to one side of the trocar’s inside not being in contact with the fluid. Therefore, this error is critical – the robot took over the flushing step for the CPAS; therefore, no errors occurred for this system.

While the robot in the CPAS can ensure standardized processing and avoid human errors, it also introduces new error sources into the work system. In our experimental setup, the parts of the trocar had to be handed over to the robot in a specific position and orientation. One participant did not fol- low the orientation guidelines, which led to faulty processing by the robot. When the participants were tasked with brushing parts of the trocar’s surface, while the robot held the trocar in a suitable position to avoid human contact with the contaminated material, part of the surface was not brushed by 9 participants. Some participants asked for a more detailed visualization of the surfaces to brush, highlighting them with different colors in the guideline’s figures. We observed two non-critical errors with the CPAS: some partici- pants were so focused on assembling the magnetic structure that they did not notice the robots call for help to brush the trocar at first. Two participants initially forgot to acknowledge the message to start the cooperative brushing process but noticed their mistakes.

One error occurring for all three assistance systems was the faulty assembly of the magnetic structure (Figure 2). The figure the participants had to build contains stacked square and triangular structures, requiring a high focus on the assembly task. Some participants missed this detail and built a figure that closely resembled the target figure but had a faulty arrangement of structures, despite having step-by-step instructions. The error occurred for six partici- pants, who repeated it with all assistance systems after making the error on their initial execution of the processing task. All participants who started the

experiments using the CPAS did build the magnetic structure correctly. Par- ticipants stated that CPAS made them feel like they could take their time to learn the assembly task.

The participants took a similar amount of time to fulfill the tasks using each assistance system, not considering the participants that did not follow the necessary times or omitted processing steps. The paper-based approach took 9:50, the DAS 9:25, and the CPAS 10 minutes.

DISCUSSION

The reported task load for all three assistance systems is low to medium. While we found no statistically significant difference, the paper-based process tends to present the highest task load and the CPAS the lowest. Therefore, integrating a CPAS system into manually reprocessing tasks for contami- nated instruments could relieve the personnel of infection risk and reduce the task load. The introduction of the robot, which could pose an additional task load on personnel just getting to know a process, did not increase the reported task load. We measured the process times while asking the partici- pants to think aloud, influencing the absolute task performance. While this would be a limitation for evaluating absolute efficiency, the relative durations across the assistance systems, as evaluated in this study, is appropriate, as the participants did report their thoughts for each of the systems.

Our results indicate that the assisting robot can reduce the number of errors occurring during typical reprocessing tasks. Many errors with high criticality are related to necessary processing times needing to be followed. 30 % of participants disregarded at least one processing time. While the auto- mated process steps are done for the recommended time by the robot, the integration of timer functions could help the human operator keep track of multiple parallel processes that require specific durations.

Another critical error is the omission of process steps without perceivable feedback about the completion. 30 % of participants did not move the trocar in the washing basin, a step that ensures the complete removal of trapped air. The remaining air inside the trocar cannot be perceived from the outside but can lead to surfaces not being cleaned by the washing fluid. A similar risk arises from a faulty orientation of the trocar during flushing, which 15 % of participants performed incorrectly. The CPAS successfully prevented these easy-to-miss errors by using the robot.

While the robot helped to brush for the required time, communicating to the participants which surface to brush was only successful in some cases. A method to directly project guidance on the instruments might help to reduce human error. Therefore, future studies should investigate light-based marker systems or augmented reality to provide visual guidance during instrument reprocessing.

One drawback of using the robot to hold the instrument for brushing is its limited flexibility, as the human operator might decide to brush differ- ent surfaces due to unpredicted contamination on the surgical instrument. One possible solution could be to have the robot’s position be adaptable by the operator (Gaz et al., 2018) or to include enhanced video camera-based

A Comparative Evaluation of Assistance Systems 153

inspection of contaminated instruments for automatic robot pose adaptation. Further studies are necessary to investigate the suitability of these approaches for instrument reprocessing.

Participants who made an error while assembling the magnetic structure continued in their first interaction and repeated the same error while using the other assistance systems. The CPAS helped the participants focus on the com- plex assembly task and supported the on-the-job training, as all participants starting with the CPAS, were able to correctly learn the assembly steps and successfully repeat them during the test of the subsequent assistance systems. One helpful addition for the CPAS and the DAS would be an indicator of typi- cal or previous errors, calling more attention to critical and easily overlooked details in the guidance. The assistance system should allow the human oper- ator to leave digital notes for individual processing steps to help colleagues. Medical device manufacturers could also provide notes to correct oversights in their instructions.

The CPAS included a linear sequence of processing steps, while the paper- based system and the DAS allowed the participants to select the sequence of tasks themselves. Participants mentioned that they perceived the linear sequence of tasks as helpful in learning the necessary processing steps, as they had no previous experience deciding about the sequence. Participants suspected an optimal sequence exists, as some processing steps can be done in parallel.

While a predefined sequence of processing steps is helpful for on-the-job training to gain a feeling for the required processing times, experienced per- sonnel might prefer an additional flexible guidance mode, which needs to be investigated in future studies.

Participants did not report feeling threatened by the robot, presumably achieved by the reduced movement speed of 400 mm/s and avoidance of abrupt movements (Arai et al., 2010). Additionally, the participants appre- ciated the announcement of robot movements using the user interface, espe- cially during cooperative process steps, to increase the feeling of safety (Ikeura et al., 2003). Participants pointed out that the announcements of robot movements should stand out visually to clearly distinguish that a message corresponds to the robot’s movement without needing to read the informa- tion. A drawback of acknowledging robot movements for cooperative steps is the increased need for user input. Further studies should investigate how this need for inputs is perceived after prolonged use of the system and investi- gate alternative ways to communicate intended robot movements to the user (e.g., light or sound based).

CONCLUSION

In summary, the CPAS improved usability the most, improving effectiveness (number of errors) while maintaining the same efficiency (total duration). Although our study found promising results for integrating a DAS or CPAS into on-the-job training assistance for novice personnel, future studies should compare the results obtained from inexperienced to experienced users to

assess the usability of related approaches fully. In addition, implement- ing light- or augmented reality-based presentation of information on the instruments during processing should be investigated further.

ACKNOWLEDGMENT

This work has been funded in parts by the German Ministry of Education and Research (BMBF) in the framework of the project “SteriRob” (Grant- No. 16ES0902).

REFERENCES

Arai, T., Kato, R. and Fujita, M. (2010) ‘Assessment of operator stress induced by robot collaboration in assembly’, CIRP Annals, vol. 59, no. 1, pp. 5–8. Choi, J., Seraphina, S. and Knudsen, K. (2017) ‘The Clean and Dirty of Redesigning

Reprocessing Instructions for Use’, Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care, vol. 6, no. 1, pp. 150–153. Cohen, J. (1988) Statistical power analysis for the behavioral sciences, 2nd edn, Hillsdale, NJ, Erlbaum. Friedman, M. (1937) ‘The Use of Ranks to Avoid the Assumption of Normal-

ity Implicit in the Analysis of Variance’, Journal of the American Statistical Association, vol. 32, no. 200, pp. 675–701. Gaz, C., Magrini, E. and Luca, A. de (2018) ‘A model-based residual approach for human-robot collaboration during manual polishing operations’, Mechatronics, vol. 55, pp. 234–247. Gerst, D. (2020) ‘Mensch-Roboter-Kollaboration – Anforderungen an eine humane

Arbeitsgestaltung’, in Buxbaum, H.-J. (ed) Mensch-Roboter-Kollaboration, Wies- baden, Springer Gabler, pp. 145–162. Heibeyn, J., König, N., Domnik, N., Schweizer, M., Kinzius, M., Janß, A. and Radermacher, K. (2021) ‘Design and Evaluation of a Novel Instrument Gripper for Handling of Surgical Instruments’, Current Directions in Biomedical Engineering, vol. 7, no. 1, pp. 1–5. Hildebrand, E. A., Branaghan, R. J., Wu, Q., Jolly, J., Garland, T. B., Taggart, M.,

Nguyen, M. A., Epstein, D. R., Babcock-Parziale, J. and Brown, V. (2010) ‘Explor- ing Human Factors in Endoscope Reprocessing’, Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 54, no. 12, pp. 894–898. Ikeura, R., Hagiwara, A., Kosha, T. and Mizutani, K. (2003) ‘Previous notice method of three dimensional robotic arm motion for suppressing threat to humans’, Pro- ceedings / RO-MAN 2003, the 12th IEEE International Workshop on Robot and Human Interactive Communication: October 31 - November 2, 2003, Millbrae, California, USA. Millbrae, CA, USA, 31 Oct.-2 Nov. 2003. Piscataway, NJ, IEEE Operations Center, pp. 353–357. Jolly, J. D., Hildebrand, E. A. and Branaghan, R. J. (2013) ‘Better instructions for use

to improve reusable medical equipment (RME) sterility’, Human factors, vol. 55, no. 2, pp. 397–410. Jwo, J.-S., Lin, C.-S. and Lee, C.-H. (2021) ‘An Interactive Dashboard Using a Virtual Assistant for Visualizing Smart Manufacturing’, Mobile Information Systems, vol. 2021, pp. 1–9. Longo, F., Nicoletti, L. and Padovano, A. (2017) ‘Smart operators in industry 4.0:

A human-centered approach to enhance operators’ capabilities and competen- cies within the new smart factory context’, Computers & Industrial Engineering, vol. 113, pp. 144–159.

A Comparative Evaluation of Assistance Systems 155

NASA (ed) (2020) NASA TLX: Task Load Index [Online]. Available at https://humansystems.arc.nasa.gov/groups/TLX/ (Accessed 9 May 2023) Prinz, C., Kreimeier, D. and Kuhlenkötter, B. (2017) ‘Implementation of a Learning Environment for an Industrie 4.0 Assistance System to Improve the Overall Equipment Effectiveness’, Procedia Manufacturing, vol. 9, pp. 159–166. Wismer, G. and Zanette, T. (eds) (2020) Handbuch für die sachgerechte Aufbereitung

von Medizinprodukten, Wiesbaden, mhp Verlag GmbH.

Cover of the proceedings volume Emerging Technologies in Healthcare and Medicine

Explore the complete digital volume

Emerging Technologies in Healthcare and Medicine

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-92-6

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →