Jonas Brüngger, Cornelia Kleindienst, Julia Koch, Frank Ritz (2020). PUMA - Development and Application of a Tool for Supporting Nuclear Power Plant Operating Teams in Unexpected and Unknown Situations. In: Advances in Social and Organizational Factors. AHFE Open Access, vol. 9. https://doi.org/10.54941/ahfe100325
School of Applied Psychology Institute Humans in Complex Systems University of Applied Sciences Northwestern Switzerland Riggenbachstrasse 16, 4600 Olten, Switzerland ↗
The method PUMA was developed to help operating teams coping with potentially critical situations that are unexpected and unknown, that means, situations that cannot be handled by standardized procedures. The method was developed with the participation of the operating teams of a nuclear power plant in Switzerland within the project TeamSafe (see also: Kleindienst, Brüngger, Koch & Ritz, 2014). By participation of the operating teams good practices were collected and condensed to the method PUMA, which consists of two essential components: "structure problem-solving process" and "coordination/communication during team discussion". Especially the encouragement of reflexivity and inclusion of all team members for information collection, situation assessment and planning of measures is supported by the tool. The method PUMA has been introduced to 10 operating teams (n=53) in the nuclear power plant. During the annually held simulator trainings in 2012 and in 2013, all control room teams of the plant have been trained theoretically and practically in applying PUMA for two simulation scenarios. After both training units all participants have been asked to evaluate the PUMA tool in a questionnaire. Analyses of the results clearly show that the tool is seen as a useful supplement to the existing standardized procedures.
Operating teams in the control room of a nuclear power plant (NPP) use standardized procedures to handle in a safe way potentially critical situations that are unexpected and unknown. On the one hand, reliable processing of standardized procedures (manuals, checklists, etc.) helps to handle a wide range of potentially critical situations. On the other hand, the future is not fully predictable, so it is not possible to develop appropriate action plans for all future critical situations in advance (Ritz, 2012). Standardized procedures might not be safe in unexpected and unknown situations that have not been anticipated while designing the procedures.
The basic idea of the PUMA method is to facilitate safety in a nuclear power plant by facilitating the operating teams' ability to adapt to unexpected and unknown situations. Safety and reliability are often considered the same. It is assumed that safety can be established by reliably following pre-defined action plans (standardized procedures, operation manual, checklists, etc.) that have proven to be safe in the past. By taking a closer look, this assumption can be proven as incomplete. The reliable functioning of a system and the following of pre-defined action plans can only guarantee safety if the given situation is known or was anticipated by the creation of the plans. In unexpected and unknown situations the reliable execution of pre-defined action plans will not ensure safety (Leveson, 2011). Such situations can happen when multiple faults in a system combine and the different pre-defined action plans demand different or even conflicting measures. Due to the many systems involved in the processes of a NPP, interactions can be very complex when faults occur. It is impossible to anticipate all possible interactions of the systems and pre-define action plans for all these cases. Only by flexibility and adaption a team can manage such unexpected and unknown situations in a control room. A team has to make use of the knowledge and experience of all its members to elaborate a solution to cope with an unexpected and unknown situation in a safe way. The PUMA method was developed to support control room teams in a NPP in these situations.
The method PUMA was developed to encourage operating teams in a nuclear power plant for coping with unknown situations that have not been anticipated in the available procedures by giving guidance for team processes. It was developed in the project TeamSafe in cooperation with a nuclear power plant in Switzerland (see also: Kleindienst, Brüngger, Koch & Ritz, 2014). There are two core elements included in the PUMA tool and its application. First, the acronym PUMA represents the steps of a problem-solving process: Problem (problem) - Ursache (cause) - Massnahme (measure) - Ausführung (execution). Each of these steps in the PUMA structure comprises four to five questions that should be answered by the team in order to adapt to an unknown situation in a safe way. Second, PUMA includes recommendations for a coordinated discussion of the PUMA steps within the team. The tool especially encourages to involve all team members in the processes of information collection, situation assessment and planning of measures as well as to act in a flexible manner.
Usually the Operations Manual (OM) serves as a guide and facilitates collectively shared situation awareness (Team Situation Awareness, Endsley, Jones, Schneider, McNeese and Endsley 2001) in NPP operations teams. This allows a team to act in a coordinated way. When a team is confronted with an unknown situation while applying the OM this is no longer the case. The PUMA method helps a team independent of the situation to create team situation awareness. It gives all team members the orientation how information should be gathered and interpreted as well as how to collect, select and execute measures to deal with an unknown situation.
DEVELOPMENT OF THE PUMA METHOD For the development of PUMA we have used two main pillars. On the one hand, we used existing successful practices of control room teams in unexpected and unknown situations, on the other hand we based the method on existing psychological knowledge. PUMA therefore connects existing good practices with existing theoretic knowledge.
A basic assumption for the chosen procedure is that the knowledge and the skills to cope with the potentially critical unknown situation are in principle available in the operating teams. Teams and their members have different strengths for coping with difficult situations in the control room. When dealing with unknown situations some teams are more effective and efficient than others. The PUMA method strives to make strategies of those teams available for all teams. The focus is on recognizing one's own strengths and learning from the strengths of others.
The development of PUMA took place in a three year process. The collecting of successful practices of the available operating teams was conducted by video during special training units. The aim of the developing process was to create a tool to cover the specific needs of the operating teams in the participating power plant. During several iterative loops it was reviewed by training instructors, operating teams, engineers and psychologists until a final version was released.
The elements of PUMA (steps of a structured problem solving process and coordination of the collaboration in the team) have been derived from the observations of behavior in control room teams during simulated potentially critical unexpected and unknown situations. During these observations the structuring of the situations as well as the coordination of the problem solving processes of the most successful teams have been analyzed. After identification, these practices have been condensed to the PUMA method that is presented in this paper.
The purpose for the active inclusion of all the operating teams in the development process was twofold. First, it was necessary to involve subject matter experts for the correct understanding of the processes involved in managing unexpected and unknown situations in a NPP. Second, giving the teams the opportunity to express worries and discuss problems at an early stage was considered being important for the later acceptance of the developed method.
The PUMA method is also based on theoretical concepts. Situation awareness respectively Team Situation Awareness (Endsley et al., 2001) being the most important of these concepts. The structure of the PUMA steps was influenced by the situation awareness concept. In unexpected and unknown situations the team has to build up a new construct of the situational conditions as part of an adaptation process (Waller, Gupta and Gianbatista 2004). The individual knowledge and information each team member holds about the state of the specific systems of the plant can be valuable for building up an accurate image or map of the whole situation. Interaction and communication processes are in the center of this re-construction process within the team. Coordination of the interaction process is one of the main tasks for the team leader (shift supervisor). It should be ensured that all the relevant information is gathered and social inhibition processes like groupthink (Turner and Pratkanis 1998) will be controlled. The gathering of information from the operators needs an atmosphere of openness, so that operators also have the courage for speaking-up (Kolbe et al. 2012). Formal interventions can help to structure this process and to open a "window of opportunity" for information collection, discussion and reflexivity within the team (Okhuysen 2001; Okhuysen and Eisenhardt 2002). PUMA fulfills the function of giving the operating team and the team leader an opportunity of withdrawing attention from the task execution to the opportunity of discussing and reflecting the actual situation and serves as a "formal intervention" for the reflecting process within the team.
By means of several iterative loops the method was reviewed by training instructors, operating teams, engineers and psychologists until a final version was released. During the development, emphasis was laid on the use of (verbal) expressions reflecting actual used language of the operating teams that are to apply the method in practice.
The PUMA method is now a part of the NPP control room in which it was developed. Is has become part of the regular training for all members of the control room teams.
The two elements of the PUMA method PUMA regulates two different elements of the teamwork during unexpected and unknown situations in the control room of the NPP. First, the steps of a structured problem solving process. Second, the coordination of the collaboration in the team.
The acronym PUMA represents the steps of a problem-solving process: Problem (problem) - Ursache (cause) - Massnahme (measure) - Ausführung (execution). Each of these steps in the PUMA structure comprises four to five questions that should be answered by the team in order to adapt to an unknown situation in a safe way (see Figure 1).
The coordination of the problem solving (or decision) process is assisted by recommendations for the application of the PUMA steps in team discussions. The intention is to give all team members the opportunity to participate in the process of answering the questions of the above mentioned structuring steps. Especially the encouragement of reflexivity and inclusion of all team members for information collection, situation assessment and planning of measures is supported by the tool.
APPLICATION OF THE PUMA METHOD For the use of PUMA in the operating room a short manual (11 pages) has been written. It answers the most important user questions and gives application instructions as well as a detailed description for all questions of the four PUMA steps (problem, cause, measure and execution). The following section of this paper is structured according to those user questions and application instructions.
One of the first questions for the application of PUMA is; when should it be used? In short, when there is the feeling that it might be helpful. Of course there is also a long answer to this question witch is given and specified in the following. As already mentioned above, the PUMA method is made to be used in unexpected and unknown situations. For an operation team in a NPP it can be very difficult to realize that it is in such a situation. There are Social and Organizational Factors (2020) standard operations procedures for most unexpected situations in the operations manual and on several checklists. Training is also heavily focused on using standard operations procedures in unexpected situations, since most of them can be processed by such in a safe way. This leads to a very high trust in the given standard operations procedures. Also is it very difficult for team members to spot the deviations in NPP system behaviours from what was planned in the standard operation procedures. The possibility of such inconsistencies and how to interpret them is rarely mentioned in the standard operation procedures. It also is reasonable to expect unexpected and unknown situations to show similar symptoms like known situations (that are covered in the operations manual). Recognition of incongruities from the expected (symptoms, values, system behaviour, inappropriate suggestions in the operations manual, etc.) therefore becomes a signal to make use of the PUMA method. Every member of the operating control room team can propose the use of PUMA to get rid of ambiguities. A trigger for this can also be a "gut feeling" that something is just not right (see Klein, 2004).
Figure 1. The PUMA method (translation from German version)
Figure available in the original PDF paper.
Other important application questions are who should start PUMA and how to do it? In the control room the shift supervisor usually has the task to coordinate the team and to make decisions about how to proceed. The PUMA method also acknowledges the shift supervisor in this role. The shift supervisor can initiate the use of PUMA at any time. If the proposition to use PUMA comes from a team member the supervisor has to decide if PUMA actually is going to be used.
To proceed according to the method, all team members should be called together. It is best if the team members already know they are going to perform PUMA, so they can take out their own PUMA documents. This can be achieved by including this information in the call: e.g. "everybody come to me, we're going to do the PUMA method."
The shift supervisor should seek a place for this so the operators that are occupied with urgent tasks (e.g. regulating the cooling water feed of the reactor core) can still participate in the discussion or at least listen what gets discussed while working off the PUMA method.
Too much information to be handled in time by a single person comes together in the control room of a NPP. Therefore, it is important that the whole control room team works together and shares the important information so team situation awareness (Endsley and Jones 2001) can be formed. It is recommended that while working off the steps of the problem solving process with PUMA the questions should be clearly formulated for all to understand. As far as known, answers, doubts, contradictions and uncertainties or even "gut feelings" should also be given. All team members are called up to supplement the answers, point out doubts and contradictions.
The discussion of the questions in their respective steps (see Figure 1) follows specific purposes. At the first step of the structured problem solving process ("problem") the observations of all team members are collected and the reason for the use of PUMA is stated. Coherent and conflicting observations and symptoms should be discussed. If there are any not already applied urgent measures (e.g. evacuation of the containment) that have to be taken immediately (e.g. due to quick situation escalation) they should be taken here.
At the second step ("cause") the operating team tries to understand the cause of the collected observations and symptoms. If assumptions can be made the team should try to verify these by additional information.
At the third step ("measures") the possible measures to cope with the situation are discussed with consideration of the available time. The discussed measures are evaluated for safety as well as time and resource requirements.
At the last step ("execution") tasks are assigned, priorities and monitoring parameters are set.
Shift supervisors are advised to note the collected information, possible risks, causes and measures as well as the allotted tasks. Such notes can later on be particularly useful as a help against forgetting something of importance. If the assessment of situation allows enough time for taking notes it is recommended doing so.
In most of the cases it makes sense to process the questions on the given order of the PUMA method structure. In the practical application there will be situations where the discussion of one question also answers another question in advance at the same time. The team will then be able to skip this question later on. In other cases the developments in a situation might lead to new information concerning an already discussed question. If this happens, the team should go back to the concerned question and supplement the answers according to the new information. For the practical application a team can expect that some of the PUMA questions will have to be answered in several iterations.
It is recommended to process all questions of the PUMA method in the team. Nevertheless, there is no obligation for a team to answer all the questions. If questions are skipped or omitted it is recommended to inform the team about this and also give a reason as to why they have been omitted. E.g. "I skip the question about the urgent measures because we have already diagnosed that we have enough time to discuss further measures first".
The PUMA method can be aborted at any time. The PUMA method should support a team in handling an unexpected and unknown situation by allowing a maximum of flexibility and adaption. To be able to handle an unknown situation adaptively, the PUMA method itself should not have a limiting effect on the operating control room team. It can make sense to abort PUMA when a team finds out that the situation they are confronted with is actually covered and can be safely conducted with the standard operations procedures.
For the processing of PUMA there is no defined endpoint. In principle the decision when to end PUMA is with the shift supervisor. The method can be stopped or cancelled at any time. The processing time for all questions of the PUMA method in the recommended way in several different simulated scenarios and with different teams was on average about 10 to 20 minutes.
PUMA is not intended to be used as an alternative to pre-existing tools in the control room. The PUMA method is intended to be used for unexpected and unknown situations when the pre-existing tools do not support the operating team. The decision if a given situation is such a situation lies in the competence of the shift supervisor.
Material Several PUMA documents for the control room have been created for facilitating the application of the method:
(1) A size A4 paper, (2) four pocket sized riveted together cards (one for each step),
(3) a big writing pad size A3 also containing information about procedures used in accidents, communication and important telephone numbers,
(4) a desk blotter with the same information as the writing pad. All these individual documents contain the four PUMA steps, the corresponding questions and information about the recommended coordination of the cooperation in the team.
These PUMA documents have been created in close coordination with the control room teams, engineers and training instructors. Wishes of the teams on what kind of documents should be created have been considered and implemented. The documents are now resident in the control room to enable further and constant familiarizing with the method.
EVALUATION RESULTS The method PUMA has been tested and introduced to 10 operating teams (each in two different configurations) (n=53) in the nuclear power plant. During the annually held simulator trainings in 2012 and in 2013, all control room teams of the plant have been trained theoretically and practically in applying PUMA for two simulation scenarios.
After both training units all participants have been asked to evaluate the PUMA tool in a questionnaire. All items in the evaluation questionnaire have been presented with a five level discrete visual analog scale (DVAS). The poles of the scale are 1 "do not agree at all" to 5 "I agree completely". The usefulness of the PUMA method had a mean score of 3.88 (SD = .88). The usefulness of the PUMA documents for the control room (cards, page A4, big writing pad A3, desk blotter) had a mean score of 4.17 (SD = .70). The mean score to the question if the team members will use PUMA in the future was 3.97 (SD = .79).
CONCLUSIONS
The available time of three years for the project and the development of the PUMA method as well as the mixed participative and theoretical approach of its development have led to success. The inclusion of all the persons concerned (operating teams) during the development process is considered to be essential for later application in a non-simulated environment. It is seen as a result of the participative way of development that there was almost no resistance regarding application from the operating teams at the end of the development process. It is important to point out that the participative way used for the development process of the method is probably a necessity for its successful implementation. Analyses of the evaluation clearly show that the tool is seen as useful supplement to the existing standardized procedures. The fact that PUMA has become a fix part in the NPP control room in which it was developed and of the regular training for all members of the control room teams shows need, trust and usefulness of the method. The authors presume that a need for such a tool also exists in other NPP's.
REFERENCES
Endsley, M. R., Jones, W. B., Schneider, K., McNeese, M., Endsley, M. (2001). "A model of inter- and intrateam situational awareness: Implications for design, training, and measurement", in: New Trends in Cooperative Activities Understanding System Dynamics in Complex Environments, McNeese, M., Salas, E., Endsley, M. R. (Eds.), pp. 46-67. Human Factors and Ergonomics Society. Kathleen, A. O., Eisenhardt, M. (2002). Integrating in groups: How formal knowledge interventions enable flexibility.
Organization Science, 13(4), 370-386. Klein, G., (2004). The Power of Intuition: How to Use Your Gut Feelings to Make Better Decisions at Work. New York: Currency. Kleindienst, C., Brüngger, J., Koch, J., & Ritz, F. (2014). Adaptive team behaviours for coping with unexpected and unknown
situations - an observational study with nuclear power plant operating teams. Proceedings of the 5th International Conference on Applied Human Factors and Ergonomics (AHFE), Kraków (Poland), July 19th-23rd 2014. Kolbe, M., Burtscher, M. J., Wacker, J., Grande, B., Nohynkova, R., Manser, T., Spahn, D., Grote, G. (2012). Speaking-up is related to better team performance in simulated anesthesia inductions: An observational study. Anesthesia & Analgesia, 115, 1099-1108. Leveson, N. G. (2011). Applying systems thinking to analyze and learn from events. Safety Science, 49(1), 55-64. Okhuysen, G. A. (2001). Structuring change: Familiarity and formal interventions in problem-solving groups. Academy of
Management Journal, 44, 794-808. Ritz, F. (2012). Organizational trust and trust in automated systems as predictors for safety related team performance - Results from a cross- cultural study. Proceedings of the 4th International Conference on Applied Human Factors and Ergonomics (AHFE), San Francisco (USA), July 21th-25th 2012, pp. 7249-7258. Turner, M., Pratkanis, A. (1998). Twenty-Five Years of Groupthink Theory and Research: Lessons from the Evaluation of a
Theory. Organizational behavior and human decision processes, 73(2/3), 105-15. Waller, M. J., Gupta, N., Giambatista, R. C. (2004). Effects of Adaptive Behaviors and Shared Mental Models on Control Crew Performance. Management Science, 50(11), 1534-1544.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Advances in Social and Organizational Factors
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-4951-2102-9
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional