Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

sWafety: A Complementary Low-threshold Safety Management Process

1 University of Applied Sciences and Arts Northwestern Switzerland (FHNW), School of Applied Psychology, 4600 Olten, Switzerland ↗
2 Maennedorf Hospital Ltd., 8780 Maennedorf, Switzerland ↗
Publication year
Digital Object Identifier10.54941/ahfe1003063

Abstract

Effective safety management requires a reliable information basis. At the same time, established safety management systems (SMS) and surveys are known to have certain limitations, such as low participation rates or data bias due to the influence of situational factors (e.g., Pfeiffer, Manser & Wehner, 2010; Sujan, 2015). In collaboration with Swiss companies from the aviation, nuclear and healthcare industries, sWafety was developed and tested as a complementary process concept to address these shortcomings. sWafety provides a low-threshold process design and digital tool that aims to further engage and motivate employees to participate in data collection, analysis, and feedback to further improve data quality.This paper describes the design of the sWafety process, which is based on a user-centered approach as well as insights from motivation theory, gamification, and practical expertise. Key process elements are presented that support motivated employee participation through short survey cycles, low respondent burden, direct feedback processes and active user involvement in data interpretation and safety measure development. Based on a use case with a Swiss hospital, a minimum viable app-prototype was applied to test the extent to which key process elements can be transferred to operational practice and how they are evaluated by users. An eight-day application scenario of sWafety was conducted with a team of anesthesia professionals (n = 7). Prototyped process elements included daily app-based data collection and alternating feedback presentation. A formative evaluation was conducted by means of a workshop.The results of a qualitative content analysis indicate that the key process elements are transferable to operational practice. Concisely worded questions, tasks, or feedback that encourage reflection and learning, were found to be more motivating than gamification. Timely feedback, particularly on work-related information, and full transparency regarding the purposes and outcomes of the surveys also appear to be very important factors motivating participation in providing safety-related data. However, users called for more opportunities to provide bottom-up information and a more active role in discussing findings and developing safety measures.Effective implementation of sWafety into operational practice requires that the general process design is adapted to a company’s specifics. Particular attention should be paid to how local, decentralized processes can be implemented to enable timely feedback and appropriate employee involvement in discussing results and developing safety measures. Regarding low-threshold process design, the results imply that elements which trigger intrinsic motivation (e.g., learning or reflection) should be preferred over gamification. Also, further development of technical solutions for flexible interaction with the app and for data analysis can further increase the potential of sWafety.

INTRODUCTION

Effective safety management requires reliable information. However, infor- mation provided by current safety management systems (SMS), or safety surveys is often limited because of low participation rates or data being biased (Pfeiffer, Manser and Wehner, 2010; Sujan, 2015). To address these shor- tcomings, partners from industry and academia cooperated closely in the sWafety project to develop a digitally supported process, which complements current SMS. Mainly two objectives were pursued: (i) Negative influences of situational factors on data quality should be kept as low as possible. There- fore, sWafety offers the possibility to ask employees a few questions every day, which they can answer within seconds. Through this continuous involvement of employees, the sWafety process allows for a longitudinal data collection which is more robust against random situational factors. (ii) Motivation to participate should be kept as high as possible. Hence, sWafety is desi- gned as a low-threshold process, which relies on process elements that meet psychological needs (Peters, Calvo and Ryan, 2018), like giving task-related

© 2023. Published by AHFE Open Access. All rights reserved. 1

feedback (Kluger and DeNisi, 1996) or promoting a sense of competence (Deci and Ryan, 2012). Increasing employee participation through the deli- berate integration of such elements is critical to promote learning (Lukic et al., 2013) and improve safety at the workplace as employees have signifi- cant explicit and tacit safety-related knowledge (Pedler, 2002). Consequently, sWafety involves employees in data collection, interpretation, and feedback, which has a positive impact on behavior and attitude (Mayo, 2003; Leana, Ahlbrandt and Murell, 1992).

After a brief overview of the sWafety process concept, this paper presents a pilot study that was carried out to learn how the concept can be transferred into practice, how it is evaluated by participants, and what the implications are for its further development.

THE DIGITALLY SUPPORTED SWAFETY PROCESS DESIGN Applying a human-centered design approach (Norman, 2013) and integra- ting insights from motivation theory (Deci and Ryan, 2012) and gamification (Zagel and Bodendorf, 2014), a low-threshold process design, and an app prototype for implementing sWafety in organizations were developed.

The sWafety process is designed as a circular structure that ensures closed- loop feedback, which is important for maintaining participation (Sujan, 2015). It comprises five process loops that involve different stakeholders in various stages of safety management (see Table 1). In this study, mainly process loop 2 “data collection and immediate feedback” was tested and evaluated (see below for details).

The sWafety app enables continuous data collection regardless of time and place, and provides immediate feedback to participants. Data collection and feedback presentation are supported by an editable format of questi- ons, selectable responses, and various feedback formats. The latter take into account that feedback is most motivating when it reacts to a specific input or offers learning opportunities (Kluger and DeNisi, 1996), e.g., by providing cues to improve safety. As it is not always possible to provide immediate, content-related feedback, gamification elements such as quizzes can be used and combined with tasks or questions (Tolks et al., 2020). To further support

Table 1. Overview of the five sWafety process loops.

Table available in the original PDF paper.

sWafety process Description Identification of safety indicators Definition, operationalisation, and validation of safety indicators Data collection and immediate feedback Continuous, short-cycled assessment of safety indicators, followed by instant feedback Feedback and identifications of areas for improvement

Presentation of detailed results followed by an assessment of where action is needed Development and mutual coordination of safety measures

Development of specific safety measures and implication Evaluation of safety measures Systematic evaluation of the effectiveness of the safety measures implemented sWafety: A Complementary Low-Threshold Safety Management Process 3 employee participation, the app also provides the possibility for participants to submit comments or notices.

PILOT AND CONCEPT EVALUATION Since sWafety is a generic process concept, it needs to be adaptable to the specific contexts and safety objectives. We piloted an initial use case in a Swiss hospital to evaluate to what extent the concept is transferable into practice, how the process elements for low threshold are perceived by participants, and which implications can be derived for further improvement of sWafety.

Application Scenario Together with a subject-matter expert, an application scenario for the work environment of anesthesia was developed, which considered sWafety ele- ments for low-threshold design, participant involvement, and continuous data collection. These are described below.

Transfer Into Practice The process of data collection and immediate feedback was adapted to ane- sthesia staff working in an operating room as follows: During an eight-day period seven anesthesia physicians and nurses answered the same 12 questi- ons about patient safety at the end of their shifts. These questions referred to safety indicators such as availability of resources, adaptability, team support, self-confidence, and subjective wellbeing and were selected, and adapted in wording and content to the specific context. An example of these questions is “Today’s circumstances require to violate my personal safety principles”. Answers could be given on a 7-point Likert scale. Since the focus was on piloting sWafety, the psychometric quality of the questions was not exami- ned. Finally, data was extracted from the sWafety database and transferred to SPSS for data analysis.

For immediate feedback, various elements supported by the app were selected. Every day, alternate feedback was provided, either instantly after the participants had responded to a particular question or after they had answered all questions. At irregular intervals, questions were followed by a comment field where participants could provide additional information or observations. The questions were randomized for daily presentation.

Implemented Low-Threshold Design To support sustained participation during the pilot, the questions were short and concisely formulated. Also, only one question at the time was presen- ted. As the app did not yet provide the ability to present immediate results (e.g., means, trends) related to specific questions, immediate feedback was designed using elements of gamification. These were namely reference tags on participation rates (see Figure 1, No. 1), informative posts on speaking up for safety or similar error prevention practices (see Figure 1, No. 2), and text or images expressing appreciation (see Figure 1, No. 3). The random arrangement of the questions and the alternating presentation of immediate feedback in terms of content and timing created variety for participants, even though the same questions were asked every day.

Figure 1: Examples of instant app-based feedback used in pilot (adapted from sWafety app).
Original PDF Figure 1:

Implemented Involvement Channels In order to involve the participants even more in the process, for some questi- ons the participants were not given direct feedback but the possibility to enter information in the form of an open text. The opportunities for participants to engage differed in their purpose. It was specified whether the participants had the option to provide additional information on a specific question (“Did you notice anything special today related to this question (positive, negative, neutral)?”) or the option to give their perception of patient safety in gene- ral (“Did you observe anything else you’d like to share about today’s patient safety (positive, negative, neutral)?”). To further increase participants’ invo- lvement, they received detailed feedback on the assessed aspects of patient safety at the end of the pilot when the findings were discussed in a facilitated workshop.

Continuous Data Collection To simulate continuous data collection through the app during the relatively short pilot, participants were asked the same 12 questions every day. This was to test the functionality of sWafety to collect time series data, which serves to account for daily contextual factors as well as to identify changes or trends over time.

FORMATIVE PROCESS EVALUATION The sWafety pilot was evaluated in a workshop with some participants (n = 5). The aim was to evaluate the extent to which the core concepts of sWafety: A Complementary Low-Threshold Safety Management Process 5 sWafety are transferable to practice, how implemented sWafety process ele- ments are perceived and what implications derive for further improvement. Group discussions along the starfish-retrospective (e.g., Rice-Khouri and Yer- bury, 2015) focused on questions related to defined evaluation criteria as described below:

  • Transfer: Successfully implemented elements for continuous data colle- ction, feedback, and involvement as well as the perceived integration into practice.
  • Low-threshold design: Perception of implemented elements, participation rate, and subjective motivation to participate.
  • Involvement: Noticed opportunities for participation (commentary functi- ons to provide additional information), actual use of opportunities, perceived adequacy.
  • Continuous data collection: Subjective judgments, perceived adequacy.

A brief quantitative survey including items from the Intrinsic Motivation Inventory (Ostrow and Heffernan, 2018) was used to assess motivation for integrating sWafety into daily work routines. The workshop results were transferred into MAXQDA for qualitative content analysis (Rädiker and Kuckartz, 2019) following the evaluation criteria.

RESULTS

The formative evaluation showed that the prototypical elements of low- threshold process design and continuous data collection can be transferred to organizational practice. In general, participants accepted using the app on their smartphone. Potential was seen especially in the stimulation of refle- ction and the support of learning (e.g., due to the content of questions) as well as for providing an information basis for safety initiatives. Based on subjective assessments, information was obtained on the potential of the pro- totypical low-threshold elements to promote motivation for participation in ongoing data collection processes.

Transfer Into Practice The application scenario with a randomized presentation of questions and immediate, app-based feedback was implemented as planned and ran stably, with no major technical problems. It was only reported that an accidental logout or an interruption of the answering process blocked any further par- ticipation and thus the answering of all questions. Nevertheless, participants found it quite easy to integrate sWafety into their daily workflow.

Low-Threshold Design Most participants took about five minutes per day to answer all safety questi- ons, which was considered reasonable. However, participants missed variety in the questions and suggested fewer questions (e.g., 1 to 2 per day) for the benefit of more focus and precision. They also pointed out that a clear rela- tion of the content (question items, tasks, information, feedback) to their work is essential for motivation. However, a decline in daily participation

rate was observed. Shift work could be a possible explanation for this. Parti- cipants also reported forgetting to report observations or events when this did not happen immediately (e.g., when they didn’t have their smartph- one with them). Thus, it was emphasized that allowing more flexible data entry as close as possible to a happening would lower thresholds. Questions or informative feedback that prompted reflection (e.g., “Are there guideli- nes for this situation?”) or learning were rated as particularly motivating. However, participants missed transparency about data processing (e.g., pur- pose of the survey, participation activity) or specific outcomes (e.g., lessons learned resulting from sWafety). Such transparency would foster motivation to continuously provide safety-related data. On the other hand, the evalu- ation showed that instant feedback using generic images (e.g., to express appreciation, comics for a positive user experience) should be avoided. Most participants perceived no benefit in these elements, as they offered no infor- mational value. Participants particularly discouraged the use of humor as a gamification feature because it is perceived as devaluing their work. There was general agreement that gamification elements should be used judiciously so as not to distract from the actual purpose of sWafety as a professional safety tool.

Involvement In general, the elements for involvement (opportunity to submit input) were positively evaluated. Participants requested more such options, especially for communicating observations and suggestions for improvement. Howe- ver, usability needs to be improved. This is also reflected by the fact that opportunities for involvement were noticed by participants but only sporadi- cally used. Furthermore, participants wish to be more involved in interpreting data, evaluating results, and developing safety measures. Physical settings that allow for face-to-face discussion were mentioned as appropriate methods for this purpose.

Continuous Data Collection It was possible to collect continuous data on patient safety. Although data analysis was limited to descriptive methods (frequency, mean) because of the small sample size and short application period. Even though there is need for further technical improvement, as the configuration of the app and the data analysis still require considerable effort, it can be assumed, that the sWafety process provides meaningful data, which supports appropriate methodological requirements.

CONCLUSION

The sWafety pilot showed, that the generic sWafety process is adaptable to the specifics of organizations. Concise questions and feedback with refere- nce to work contexts and target groups support participation. Feedback has high potential to increase motivation to participate. Therefore, companies should implement feedback into their communication. Immediate app-based sWafety: A Complementary Low-Threshold Safety Management Process 7 feedback seems to be effective if it triggers reflection and learning, while pro- viding transparency of the sWafety process itself (e.g., aim of assessment, data processing status, resulting consequences) further increases motivation. Gamification, on the other hand, should be used sparingly. Furthermore, employees wish to be involved in discussing results of assessments and deve- loping safety measures. Settings such as workshops or focus groups should be established, since the need for face to face exchange is emphasized. For involving employees via the app, appropriate formats must be designed. Tech- nical improvements must focus on enabling more flexible data collection and interaction regardless of time and place, as well as efficient data analysis to shorten feedback loops. Nevertheless, the results imply that the sWafety concept for low-threshold involvement and continuous data collection is ada- ptable to the conditions of professional practice and provides a sound basis for improving established SMS.

ACKNOWLEDGMENT

This work was funded by Innosuisse, the Swiss Federal Innovation Agency.

REFERENCES

Deci, E. L. and Ryan, R. M. (2012) ‘Motivation, personality, and development within embedded social contexts: An overview of self-determination theory’, in R. M. Ryan (ed.) The Oxford handbook of human motivation. New York: Oxford University Press, pp. 85–107. Kluger, A. N. and DeNisi, A. (1996) ‘The Effects of Feedback Interventions on Per-

formance: A Historical Review, a Meta-Analysis, and a Preliminary Feedback Intervention Theory’, Psychological Bulletin, 119(2), pp. 254–284. Available at: https://psycnet.apa.org/doi/10.1037/0033-2909.119.2.254 (Accessed: 30 January 2023). Leana, C. R., Ahlbrandt, R. S. and Murrell, A. J. (1992) ‘The effects of employee involvement programs on unionized workers’ attitudes, perceptions, and preferences in decision making’, The Academy of Management Journal, 35(4), pp. 861–873. Lukic, D., Margaryan, A. and Littlejohn, A. (2013) ‘Individual agency in lear-

ning from incidents’, Human Resource Development International, 16(4), pp. 409–425. Available at: https://doi.org/10.1080/13678868.2013.792490 Mayo, E. (2003) The human problems of an industrial civilization. London: Routledge. Norman, D. (2013) The design of everyday things. Revised and expanded edition.

New York: Basic Books. Ostrow, K. S. and Heffernan, N. T. (2018) ‘Testing the Validity and Reliability of Intrinsic Motivation Inventory Subscales within ASSISTments’, in Artificial Intel- ligence in Education (ed.) Lecture Notes in Computer Science, 10947. Cham: Springer, pp. 381-394. Available at: https://doi.org/10.1007/978-3-319-93843- 1_28. Pedler, M (2002) ‘Accessing local knowledge: action learning and organizational lear-

ning in Walsall’, Human Resource Development International, 5(4), pp. 523–540. Available at: https://dx.doi.org/10.1080/13678860110061732. Peters, D., Calvo, R. A. and Ryan, R. M. (2018) ‘Designing for Motivation, Engagement and Wellbeing in Digital Experience’, Frontiers in Psychology, 9, pp. 1–15. Available at: https://doi.org/10.3389/fpsyg.2018.00797.

Pfeiffer, Y., Manser, T. and Wehner, T. (2010) ‘Conceptualising barriers to incident reporting: a psychological framework’, Quality and Safety in Health Care, 19(6), pp. 1–10. Available at: https://doi.org/10.1136/qshc.2008.030445. Rädiker, S. and Kuckartz, U. (2019) Analyse qualitativer Daten mit MAXQDA.

Wiesbaden: Springer VS. Rice-Khouri, A. and Yerbury. J. (2015) ‘Starfish behavioral Design’, in S. Stein, (ed.) Creative techniques handbook 2015 Digital Futures OCAD University, 1, pp. 50–69. Available at: https://core.ac.uk/download/54850037.pdf (Accessed: 31 January 2023). Sujan, M. (2015) ‘An organisation without a memory: A qualitative study of hospi-

tal staff perceptions on reporting and organisational learning for patient safety’, Reliability Engineering and System Safety, 144, pp. 45–52. Available at: https: //doi.org/10.1016/j.ress.2015.07.011. Tolks, D., Lampert, C., Dadaczynski, K., Maslon, E., Paulus, P. and Sailer, M.

(2020) ‘Spielerische Ansätze in Prävention und Gesundheitsförderung: Serious Games und Gamification’, Bundesgesundheitsblatt, 63(6), pp. 698–707. Available at: https://doi.org/10.1007/s00103-020-03156-1. Zagel, C. and Bodendorf, F. (2014) ‘Gamification: Auswirkungen auf Usability, Datenqualität und Motivation’ in M. Koch, A. Butz and J. Schlichter (eds.) Mensch und Computer 2014 -Tagungsband. München: Oldenbourg Wissenschaftsverlag, 2014, pp. 15–24. Avaliable at: https://dl.gi.de/bitstream/handle/20.500.12116/ 7546/Zagel_Bodendorf_2014.pdf?sequence=2&isAllowed=n (Accessed: 31 Janu- ary 2023).

Cover of the proceedings volume Safety Management and Human Factors

Explore the complete digital volume

Safety Management and Human Factors

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-81-0

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →