Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Usability testing of instructions for use in cleaning, disinfection and sterilization of ultrasound probes

Publication year
Digital Object Identifier10.54941/ahfe1004149

Abstract

Infection Prevention and Control (IPC) represents an area of big interest in healthcare. In this context, 'reprocessing' means a cleaning, disinfection or sterilisation process carried out on a used device in order to allow its safe reuse. A typical IPC responsibility assignments matrix includes healthcare facilities, manufacturers of reprocessing agents and equipment, and manufacturers of medical devices. This project aims to analyze and validate the Instructions for Use (IFUs) for reprocessing of ultrasound (US) probes, from the point of view of the usability, in accordance with IEC 62366-1:2015+AMD1:2020 and IEC TR 62366-2:2016.US probes enable a wide range of applications, with the level of infection risk based on the Spaulding Classification, going from noncritical and semi-critical (intact skin and mucous membrane contacts respectively), to critical (intraoperative). In addition to the diverse clinical procedures, the risk of contamination depends also on the variability in the healthcare environment and user knowledge and expertise.Safe reprocessing of an US probe requires that its IFUs, considered as part of the user interface, are clear, legible, and complete. For this reason, to ensure their correct execution and workflow, a usability testing for the reprocessing IFUs is performed. According to the IEC 62366 standards, such test consists of two steps: formative and summative evaluations. The overall aim of the first step is to explore if the instructions are recognizable, understandable and operable by the user and to identify worst-case scenarios and critical tasks which will be further investigated during the second step by a user group, including lay and professional users with different profiles and needs. For the purpose of this project, the usability testing has confirmed a powerful tool to verify that our reprocessing IFUs for aUS probe can be easily and effectively used. The preliminary analysis and the following interviews of diverse users operating in different environments, have provided evidence that such IFUs enable the user to perform a proper and reliable reprocessing for a safe reuse of the probe.

INTRODUCTION

Diagnostic ultrasound (US) imaging is a non-invasive technique widely used worldwide, being non-hazardous, compact, portable, real-time and versa- tile. Dedicated and specialized US probes allow many clinical applications,

© 2023. Published by AHFE Open Access. All rights reserved. 368

Usability Testing of Instructions for Use for Cleaning, Disinfection and Sterilization 369

including abdomen, cardiovascular, musculoskeletal, obstetrics and gynae- cology, interventional and intraoperative procedures. Different users (e.g. sonographers, physicians, radiologists, surgeons, anaesthesiologists, mid- wives, paramedics) may perform an US scan/exam in diverse environments such as emergency room, intensive care unit, operating theatre (De Luca et al. 2021; Andreoni et al. 2015). In this complex context characterized by diverse intended uses and environments, a correct reprocessing of probes is a cru- cial process, to guarantee a reliable and safe use of the probe. Spaulding divided medical instruments and equipment into three categories (critical, semi-critical, and non-critical) on the basis of the risk of infection from contamination on the item:

  • Non-critical: medical device in contact only with the intact skin of the patient;
  • Semi-critical: medical device in contact with intact mucous membranes or nonintact skin of the patient;
  • Critical: medical device introduced directly into the human body, either in contact with the bloodstream or into other normally sterile areas of the body (Spaulding, 1972).

Manufacturers of reusable medical devices have the responsibility to support product label claims of reusability by providing complete and com- prehensive written instructions for the cleaning, disinfection, and sterilization of their products. Manufacturers also have the responsibility to conduct and document any testing necessary to validate the suitability of these instructions (AAMI TIR12:2010). Then, it is important to provide users with suitabil- ity information aimed at the proper reprocessing of probes according to their use, to avoid cross-contamination and consequently possible infec- tion. Usability is an important topic in the risk analysis to minimize use errors that can lead to hazardous scenarios. As defined in the IEC 62366- 1:2015 + AMD1:2020, usability is the characteristic of the user interface that facilitates use and thereby establishes effectiveness, efficiency, and user satis- faction in the intended use environment (IEC 62366-1:2015 + AMD1:2020). The use of medical devices could be a normal or abnormal use. An abnor- mal use is an intentional improper use of the medical device, which is not managed by the standards. In particular, the standards help to manage the normal use of the medical device, that can be divided in correct use and use error. The use error is a user action or lack of user action while using the medical device that leads to a different result than that intended by the man- ufacturer or expected by the user. The relationship between different types of use of medical device is shown in Figure 1. In this scenario, the main role is played by the user interface, which is the means by which the user and the medical device interact (IEC 62366-1:2015 + AMD1:2020). The analysis of user errors and of critical errors aims to either approve the user interface or to identify non-acceptable use risks that would lead to device interface re-design (Ravizza et al. 2019; Zhang et al. 2003).

Accompanying documentation is considered part of the medical device and its user interface. When information for safety is used as a risk control mea- sure, the medical device manufacturer shall subject this information to the

usability engineering process with the following aim: to determine that the information is perceivable by, is understandable to, and supports correct use of the medical device by users of the intended user profiles in the context of the intended use environment (IEC 62366-1:2015 + AMD1:2020). The pur- pose of this paper is to evaluate the usability of the reprocessing IFUs defined according to the probe category.

Figure 1: Relationship of the types of use (IEC 62366-1:2015 + AMD1:2020).
Original PDF Figure 1:

USABILITY ENGINEERING PROCESS As reported in the usability standards, the usability engineering process con- sists of two main steps: formative and summative sessions. The formative session is made of iterative tests performed by different types of user and by the design team. It aims to improve the user interface before the summative session, performed by a significant sample of final users.

In the specific study, the usability test has been performed following the steps described by the IEC 62366 standards (Stuppia et al. 2022).

Formative and Summative Assessments To perform the formative test, the following techniques, recommended in the IEC TR 62366-2:2016 standard, have been implemented: focus group, cog- nitive walkthrough, brainstorming, task analysis and standard review. The focus group is defined as an effective technique for understanding the per- ceptions, opinions, beliefs and attitude of medical device users. The focus group of the first formative session was composed by the design team and two experts of the product, which collected feedback from the field. Then, the cognitive walkthrough was conducted, to predict what we expect from the end user. Thereafter the task analysis techniques were carried out to study the interaction between the user and the manual and to understand what actions hinder or facilitate the user performance (IEC TR 62366-2). In the second formative session, other three experts were involved to brainstorming to identify other possible hazard related use scenarios. In this study, the risk assessment started with a high-level risk analysis, depending on the improper reprocessing, and continued with a deeper and detailed level of risk analy- sis including an erroneous understanding of the instructions to reprocess the

Usability Testing of Instructions for Use for Cleaning, Disinfection and Sterilization 371

probe following the provided documentation (IFUs). According to the stan- dards, the information provided to the final user, shall be easy to read, easy to find and easy to understand. The risk analysis of the correct probes repro- cessing instructions was based on these three fundamentals (IEC TR 62366-2; ISO 14971:2019 + A11 2021; ISO 15233-1:2021).

The study through formative sessions, aimed to identify if some instruc- tions contained in the manual could lead the user to conduct an improper reprocessing of the probe and therefore to define the related tasks. The task analysis was carried out to produce detailed descriptions of the activities of personnel performing the reprocessing (IEC TR 62366-2:2016).

The output of the formative sessions was the identification of the primary operating functions (POFs) that are functions that involve user interaction that is related to the safety of the medical device, and their related tasks, reported in Table 1 (IEC TR 62366-1:2015 + AMD:2020).

Table 1. POFs and critical tasks.

Table available in the original PDF paper.

POFs Tasks POF1 Understanding of probe classification and clinical procedure

Choose of reprocessing methods to be used following the instructions

Choose of clinical procedure with probe classification POF2 Identification of reprocessing methods to be used with respective instructions POF3 Association of the products to be used to perform the appropriate probe reprocessing Choose of appropriate products for each probe and reprocessing method POF4 Follow the instructions for performing reprocessing for non-critical, semi-critical, critical probes, biopsy application

Preliminary steps: probe disconnection, cover and accessories removal Performing reprocessing within the specified time frame Performing reprocessing according to the immersion level Performing cleaning before disinfection/sterilization

Finally, a standards review was conducted with an internal usability expert, to evaluate the user interface according to established usability engineering practices. Though the feedback from the field regarding the manuals, and the two formal formative sessions with a group of experts, primary operat- ing functions and tasks were listed. For this study, 15 specialists with different types of expertise, were chosen to perform the tasks and fill out an associ- ated questionnaire. Such questionnaire required to associate an agreement or disagreement score to each question related to the task list. Hereafter the list of questions is reported in the Table 2.

The different colours have the scope to link each POF to one or more tasks, listed in the Table 1, that is linked to one or more questions (Table 2).

Table 2. Questionnaire.

Table available in the original PDF paper.

Questions POF1 Is it easy to identify the classification of the probe? Is it easy to identify the type of procedure? Is the search for such information within the manual sufficiently intuitive? POF2 Is it easy to identify the reprocessing methods associated with probe classification? Is it easy to identify the reprocessing methods associated with the biopsy procedure? Depending on the classification and access and the tissues it comes into contact with, is it clear which reprocessing method to use? Is the difference between LLD, MLD and HLD clear? Is the difference between manual liquid and wiping methods clear? POF3 Is it easy to identify the reprocessing agent to be used for each method? Is the table containing product information is easy to consult? POF4 Before starting reprocessing, is clear that the probe must be disconnected from the system? Before reprocessing can begin, is clear that the cover must be removed? Is it clear that any accessories must be removed before reprocessing can begin? Are the times given in the reprocessing instructions understandable? Is it clear that the process has a certain duration, which is mandatory? Is it clear where to find, within the manual, the inherent indication of immersion level? Is it sufficiently clear when to perform the cleaning step within a disinfection and/or sterilization procedure?

For each question an agreement or disagreement score was associated as reported in the Table 3.

Table 3. Agreement/disagreement scores.

Table available in the original PDF paper.

Agreement/Disagreement Related score

3

I like it, no problem 0 I might need some time to get comfortable on how to use it 1 It is hard to use. It makes me nervous 2 The use and/or the information for use are too complicated; I can’t understand it Don’t use it! Possible risk 4

Severity rating scale was used to identify the critical points within the man- ual referred to the reprocessing (J. Zhang et al. 2003). There is a wide variety of end users involved in the use of an ultrasound probe. As reported in the annex K of the standard 62366-2, a representative sample needs to be cho- sen in the summative session. Fifteen participants are a practical minimum number of participants for human factors validation testing (Health et al. 2019). User group involved in the summative session, was composed by dif- ferent specialists with different age comprised between 28 and 65 years old

Usability Testing of Instructions for Use for Cleaning, Disinfection and Sterilization 373

and different levels of expertise and experience, by entry level to senior. The areas of expertise and applications included urology, breast care, obstetrics, gynaecology, interventional, cardiology, service, and usability.

The standard shows the decrease in cumulative for a number of users greater than 10. However, the factors that cause cumulative decreases are number of users, probability of a usability error occurring, formative iterations.

Added to the formative iterations, there is the important statistic feature of the presence of the manual in the field and the feedback reported by the internal experts.

Summative Evaluation Results To analyze the IFUs understanding, identification and execution, the over- all score associated to each task was calculated as the average of the scores assigned to the relative questions. No critical error (corresponding to totally unsatisfied in the bar chart) was observed during the simulated use indicat- ing the absence of patient-related risk linked to intended user/user-interface interaction. Therefore, the risk of cross-contamination and, consequently, infection is effectively mitigated.

Figure 2: Task analysis: average of the scores assigned to the questions related to each task.
Original PDF Figure 2:

The task including preliminary steps (probe disconnection, cover and accessory removal), achieved the 90% of totally satisfied score, with a not significant percentage of partial unsatisfied and neutral. A generic suggestion collected during the summative test regarded the possibility to have a leaflet with the IFUs described by pictograms to have an easier access and under- standing to the information for an appropriate reprocessing. Such suggestion

does not affect the current IFUs, but it represents a potential additional information to be provided to the user and therefore an improvement of the manual. The question regarding the probe immersion level limit (“Is it clear where to find, within the manual, the inherent indication of immersion level?”, POF4, Table 2), registered almost the 30% of neutrals, however this result does not correspond to a potential user error. It is considered a useful input to improve the manual section showing the probe immersion limit. A remarkable result is on performing cleaning before sterilization and disinfec- tion. It means that the information is clearly expressed and that there is the confidence that this step is performed as indicated. It is an important result because it is a crucial step to begin the correct reprocessing.

Heuristic summary is hereby proposed in Figure 3 through a radar-graph with all the participants’ responses. The underlying area of the graph is approximately proportional to the perceived risk (larger area, major per- ceived risk) and therefore inversely proportional to the degree of under- standing and implementation of the provided reprocessing IFUs (small area, overwhelming positive response). Since all the responses are centred in the middle, it is confirmed that the reprocessing IFUs were well-received by the participants.

Figure 3: Heuristic analysis: participants scores assigned to each question.
Original PDF Figure 3:

The following questions:

  • Is the search for such information within the manual sufficiently intuitive? (POF 1, Table 2),
  • Before starting reprocessing, is clear that the probe must be disconnected from the system? (POF 4, Table 2)

Usability Testing of Instructions for Use for Cleaning, Disinfection and Sterilization 375

registered the highest scores (higher score, lower favourableness) that came from entry-level users, who are less familiar with manuals and workflow. Note that this feedback regards the capability to easily find the correct infor- mation in the manual, that is operator-dependent, and it does not affect the correct understanding of a proper probe reprocessing execution. A training dedicated to the information flow showed in the manual represents a suitable mean to aid the manual readability.

The polarized graph in Figure 4 shows the average of the scores (refer to Table 3) assigned by the fifteen participants to each question. This graph allows to identify the most critical aspects under investigations, correspond- ing to the highest score. Note that all the obtained results are lower than 1; this means that the overall summative evaluation result proved that the provided IFUs minimize the risk of improper reprocessing. Therefore, the summative evaluation has confirmed that essential information for the effec- tive use of the IFUs represents a powerful mean directed to the user to perform an appropriate probe reprocessing to optimize infection prevention and control in healthcare environments.

Figure 4: Average of the scores assigned by the participants to each question.
Original PDF Figure 4:

CONCLUSIONS

Safe reprocessing of an US probe requires that its IFUs, considered as part of the user interface, are clear, legible, and complete. According to the IEC 62366 standards, formative and summative evaluations have been per- formed, to evaluate the reprocessing IFUs from the usability point of view. For the purpose of this project, the usability testing has confirmed a powerful tool to verify that our reprocessing IFUs for a US probe may be easily and effec- tively used. The preliminary analysis and the following interviews of diverse users operating in different environments, have provided evidence that such IFUs enable the user to perform a proper and reliable reprocessing for a safe reuse of the probe.

REFERENCES

AAMI TIR12:2010, Designing, testing, and labeling reusable medical devices for reprocessing in health care. Andreoni G., Mazzola M., Matteoli S., D’Onofrio S., Forzoni L. (2015), Ultrasound

system typologies, user interfaces and probes design: a review, Procedia Manuf Elsevier, 3: 112–119, doi: 10.1016/ j.promfg.2015.07.115. De Luca R., Dattoma T., Forzoni L., Bamber J., Palchetti P., Gubbini A. (2018), Diagnostic ultrasound probes: a typology and overview of technologies, Current Directions in Biomedical Engineering, 4(1): 49–53, doi: 10.1515/cdbme-2018- 0013. De Luca R., Forzoni L., Gelli F., Bamber J., An Educational Overview of Ultrasound

Probe Types and Their Fields of Application, Archives of Acoustics – Volume 46, Number 1, 2021. EN ISO 14971:2019 + A11 2021 Medical devices. Application of risk management to medical devices. EN ISO 17664:2021 Processing of health care products—Information to be provided

by the medical device manufacturer for the processing of medical devices — Part 1: Critical and semi-critical medical devices. Health, C. for D. and R. (2019, September 2). Applying Human Factors and Usability Engineering to Medical Devices. U.S. Food and Drug Administration; FDA. https://www.fda.gov/regulatory-information/searchfda-guidance-document s/applying-human-factors-andusability-engineering-medical-devices IEC 62366-1:2015 + AMD1:2020, Medical devices - Part 1: Application of usability

engineering to medical devices. IEC TR 62366-2:2016, Medical devices - Part 2: Guidance on the application of usability engineering to medical devices. ISO-TR 15223-1:2021 Medical devices – Symbols to be used with information to be

supplied by the manufacturer – Part 1: General requirements. ISO-TR-24971:2020 Medical devices—Guidance on the application of ISO 14971. Ravizza, A., Lantada, A. D., Sánchez, L. I. B., Sternini, F., & Bignardi, C. (2019).

Techniques for Usability Risk Assessment during Medical Device Design. Proceed- ings of the 12th International Joint Conference on Biomedical Engineering Systems and Technologies, 207–214. https://doi.org/10.5220/0007483102070214 Spaulding EH. Chemical disinfection and antisepsis in the hospital. J Hosp Res, 9:5-31, 1972. Zhang, J., Johnson, T. R., Patel, V. L., Paige, D. L., & Kubose, T. (2003). Using usabil-

ity heuristics to evaluate patient safety of medical devices. Journal of Biomedical Informatics, 36(1), 23–30. https://doi.org/10.1016/S1532-0464(03)00060-1

Cover of the proceedings volume Human Systems Engineering and Design (IHSED 2023): Future Trends and Applications

Explore the complete digital volume

Human Systems Engineering and Design (IHSED 2023): Future Trends and Applications

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-88-9

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →