Claire Blackett (2023). Evaluation of a Basic Principle SMR Simulator for Experimental Human Performance Research Studies. In: Human Factors and Simulation. AHFE Open Access, vol. 83, pp. 78–88. https://doi.org/10.54941/ahfe1003564
Simulator studies are important to understanding and collecting data on human performance, especially for first-of-a-kind technologies such as Small Modular Reactors (SMR) and/or in cases where the role of the human operator is expected to change, such as in multi-unit operations. But not all simulators are the same, and the level of complexity and fidelity of the simulator can significantly affect the possibilities for data collection. As a researcher, how can you evaluate whether the simulator you are using is suitable for the studies that you wish to run? In 2020, the researchers of the Halden Reactor Project (HRP) activity on operation of multiple small modular reactors had a unique opportunity to explore this question. A multi-unit basic principle integral pressurised water reactor (iPWR) simulator was installed in the FutureLab facility in Halden, Norway in early 2019, and a first, small study was conducted in late 2019 to test the simulator environment and study design. The simulator was provided to the HRP free of charge by the International Atomic Energy Agency (IAEA). It is important to note that the simulator was not designed for performance of research studies, but rather as an education tool to demonstrate the basic principles and concepts of SMR operation, and as such is limited in scope by design. The goal for the small study in 2019 was to determine whether the basic principle simulator could enable investigation of predefined topics relevant to SMR operations research, such as monitoring strategies, prioritization of taskwork and staffing requirements in multi-unit environments. The study involved two experienced former control room operators, who were tested individually in a series of scenarios of increasing complexity in a multi-unit control room setup, observed by experienced experimental researchers. Further studies with licensed control room operators were planned for 2020, but due to the COVID-19 pandemic these plans had to be postponed. Instead, the research team took the opportunity to reflect on the experience of the 2019 small study, to perform a more detailed analysis of the study results and to substantiate the feasibility of the test environment for future experimental data collection. The detailed analysis was performed in a workshop format with the research project team using a set of questions related to specific aspects of the iPWR 3-unit control room setup. The team used a “traffic light” rating system to evaluate each question, where red indicated that that the test environment item under discussion is not currently feasible and would require redesign and extensive changes in order to use it; yellow indicated that the item is not currently feasible and would require moderate changes, and green indicated that the item is currently feasible, and some minor changes could be required. This paper describes the evaluation process in more detail, including the criteria for assessing the usefulness of the basic principle simulator for conducting experimental human performance studies in the future, and the results of the evaluation. While the list of criteria identified during this workshop is not considered exhaustive, these results are still expected to be useful to other researchers considering experimental control room studies to determine the level of complexity and fidelity that can be reasonably achieved in a control room simulator.
Simulator studies are important to understanding and collecting data on human performance, especially for first-of-a-kind technologies such as Small Modular Reactors (SMR) and/or in cases where the role of the human ope- rator is expected to change, such as in multi-unit operations (Blackett et al., 2022a). But not all simulators are the same, and the level of complexity and fidelity of the simulator can significantly affect the possibilities for data col- lection. As a researcher, how can you evaluate whether the simulator you are using is suitable for the studies that you wish to run?
A basic principles integral pressurised water reactor (iPWR) simulator was installed in the Futurelab facility in Halden, Norway in early 2019, and a first, small study was conducted in late 2019 to test the simulator environment and study design. The simulator was provided to the HRP free of charge by the International Atomic Energy Agency (IAEA). We acknowledge that the simulator was not designed for performance of research studies, but rather as an educational tool to demonstrate the basic principles and concepts of SMR operation, and as such is limited in scope by design.
The goal for the small study was to determine whether the basic princi- ples simulator could enable investigation of predefined topics relevant to
SMR operations research, such as monitoring strategies, prioritization of taskwork and staffing requirements in multi-unit environments. The study involved two experienced former control room operators, who were tested individually in a series of scenarios of increasing complexity in a 3-unit con- trol room setup, observed by experienced experimental researchers. Further studies with licensed control room operators were planned for 2020, but due to the COVID-19 pandemic these plans had to be postponed. Instead, the research team took the opportunity to reflect on the experience of the first small study, to perform a more detailed analysis of the study results and to substantiate the feasibility of the test environment for future experimental data collection.
This paper describes the evaluation process in more detail, including the criteria for assessing the suitability of the basic principles simulator for con- ducting experimental human performance studies in the future, and the results of the evaluation. While the list of criteria is not considered exhaustive, these results can be useful to other researchers considering experimental con- trol room studies to determine the level of complexity and fidelity that can be reasonably achieved in a control room simulator.
USE OF SIMULATORS FOR HUMAN FACTORS RESEARCH Boring (2010, pg. 3) states that a simulator is “a physical device that repli- cates the operations of an actual device used in the workplace or other environments”. In the nuclear industry, simulators are typically used to train operators in the performance of their usual tasks, and in response to abnor- mal or emergency scenarios. Simulators are also utilised in human factors research to study and evaluate human performance, especially (although not exclusively) in control room settings. As noted in Boring (ibid.), the IAEA defines four different types of plant simulator:
Basic principles simulator, which illustrates general concepts and demon- strates fundamental physical processes of a plant.
Full-scope simulator, which includes detailed modelling of the plant systems in an actual control room environment.
Other-than-full-scope simulator, which is a simulator that does not pro- vide the same human-machine interface (HMI) as the plant to which it is referenced.
Part-task simulator, which incorporates detailed modelling of a referenced plant, but does not simulate all systems.
Boring emphasizes the usefulness of research simulators for the study of human performance in “a realistic embodied cognitive environment” that allows for the study of “operators in the wild – operator actions embodied in the full context of the control room environment” (ibid., pg. 6). The use of full-scope simulators for human factors research has been at the heart of the Halden Reactor Project since the early 1980s (Skjerve and Bye, 2011).
The HRP/HTO Simulator Facilities and Experience in Halden The OECD Halden Reactor Project (HRP) was established in 1958 as a joint research programme covering topics related to (i) fuels and materials (F&M)
80 Blackett
and, in 1967, (ii) Man-Technology-Organisation (MTO). The HRP F&M programme continues today, but the HRP MTO programme was completed in 2020. In its stead, the OECD NEA Halden Human Technology Organisa- tion (HTO) project was established in 2021 and is a direct continuation of the HRP MTO research programme.
The interest in human factors as part of the MTO programme fuelled the need for access to a full-scope nuclear control room simulator, and in 1983 the Halden Man-Machine Laboratory (HAMMLAB) was established (Skjerve and Bye, 2011). Over the years, and as the scope of the HRP MTO programme and now the Halden HTO programme has grown, additional simulator facilities have been added to the Halden research laboratories, including virtual reality and cybersecurity. The HAMMLAB now comprises three full-scope nuclear power plant control room simulators, a control room where licensed operating crews operate the simulator in experimental studies, and a gallery from where the experimental team controls the scenarios, obse- rves crew behaviour, and evaluates human performance and human-machine interfaces (HMI).
Human Factors Research Needs for Small Modular Reactors There are over 70 designs for small modular reactors (SMRs) currently in development around the world, according to the IAEA (2020). In 2022, the United States Nuclear Regulatory Commission (US NRC) announced that it will issue the final rule that certifies NuScale’s SMR design for use in the USA1, marking a significant milestone for the nuclear industry. Literature searches performed as part of the Halden HTO activity on SMRs identified that there is a lack of information available on human factors and human performance aspects of SMR operation (Blackett et al., 2022b). This may be explained by the low maturity levels of the majority of SMR designs, and the fact that there are currently no SMR plants in operation, as well as the highly competitive nature of the SMR industry meaning that a lot of information about design and operation of SMRs is proprietary and not available to the public.
Despite the limited information available, we can make some inferences about how the role of the operator may change in SMR operations. Some of the design characteristics of SMRs, which are expected to have an impact on SMR operations, are outlined in Blackett et al. (2022c) as follows:
SMRs tend to have smaller, simpler designs which will make them easier to operate than conventional nuclear power reactors.
SMRs are likely to place greater emphasis on the use of passive safety systems for reactivity control, meaning that there is less/no reliance on operator intervention.
SMRs are expected to utilise higher levels of automation than at conven- tional plants, which will replace and/or change many previously manually performed tasks.
1News Release-22-029: NRC To Issue Rule Certifying NuScale Small Modular Reactor
Evaluation of a Basic Principles SMR Simulator 81
SMRs have the capability for multi-unit operation, meaning that several reactor units could be monitored and managed in parallel from a single control room.
SMRs are expected to create greater flexibility for deployment applicati- ons, including the ability for remote operation, as well as for non-nuclear applications, unlike conventional plants today that are primarily used for baseload electricity production.
From this information, we can infer that the role of the operator is likely to change from one of active participant/controller of the plant and its systems, to that of a supervisor or monitor of highly automated or even autonomous plant systems. We also expect that the nature of the operational tasks may change, and thus the knowledge and competence requirements for operators may change, due to e.g., multi-unit, remote and/or non-nuclear applications (ibid.). There is uncertainty about the human factors challenges and effects on human performance of these changes, especially as there is no operatio- nal experience available to support research and learning. As such, simulator studies on SMR operations are highly valuable to give insights into the poten- tial effects of new concepts of operation on human performance, risk, and safety. The Halden HTO research activity on “Operation of SMRs” attem- pts to identify some of these uncertainties through experimental simulator studies.
OVERVIEW OF THE HALDEN IPWR SMR SIMULATOR In 2019, the HRP MTO project obtained a basic principles iPWR simula- tor from IAEA. The design is based on the Idaho National Engineering and Environmental Laboratory’s Multi-Application Small Light Water Reactor (MASLWR), which has been slightly modified to make the design more gene- ralisable to several different reactor design variants in development around the world (Eitrheim et al., 2020). As per the IAEA classification of simu- lator types, this simulator was not developed as a research tool, but rather is intended for educational purposes to demonstrate the basic principles of SMR operation. Thus, it comes with a simplified graphical user interface and with limited configurability and scope.
The simulator is described as follows in Eitrheim et al. (2020). The primary circuit components of the reactor (steam generator, pressuriser and control rod drive mechanism) are integrated in the reactor pressure vessel (RPV). Core cooling is achieved either by forced or natural circulation of light water within the RPV. As with many new SMR designs, the iPWR utilises seve- ral passive safety features, including an Automatic Depressurisation system (ADS), a Pressure Injection system (PIS), a Gravity Injection system (GIS) and Passive decay heat removal (PDHR).
The simulator has two possible plant operating modes (turbine leading or reactor leading) and two core cooling options (natural circulation with no reactor coolant pumps (RCP) or forced circulation with four RCPs). The HMI supplied with the simulator has 11 displays, all of which have the same con- figuration menu and alarm information at the top, and key plant parameters
82 Blackett
Figure 1: The HMI from the iPWR simulator.
and a navigation area to the left of the display window. An example is shown in Figure 1. The displays can be opened as separate windows and shown side- by-side across multiple display screens. It is possible to open trends in separate windows, and to display several curves in each trend area. The configura- tion menu has a dropdown list of functionalities to enable loading of initial simulator conditions, modification of specific parameters and activation of malfunctions. It is not possible to remotely access configurations; these must be performed on the display screen in which the simulator is running.
There are 16 alarms which are indicated by tiles along the top of all of the display windows, covering e.g., reactor trip, low reactor coolant system pressure, and feedwater isolation. The iPWR simulator does not include an audible alarm sound. The HRP research staff developed a simplified alarm system for use in the 2019 small study that alarm sounds and additional alarms that did not come with the simulator. It was not integrated with the simulator and had to be operated manually from a separate computer in real- time during the experiment scenarios.
Simulator Setup for the 2019 Small Study on Operation of SMRS Three instances of the iPWR simulator were installed in the MTO Futurelab, as an example of a multi-unit control room setup. For each reactor unit, one large overview display screen was set up showing the human-machine inter- face (HMI) for that reactor, with two smaller operator workstation screens on a desk in front of the overview screen. The HMI on the overview screen was fixed, but the operators could change the displays on the workstation screens by using the navigation menu in each display window. The set-up per unit was spatially fixed, meaning that the operators had to physically move to the specific unit workstation in order to interact with that reactor via the workstation displays. They could not, for example, open the display
Evaluation of a Basic Principles SMR Simulator 83
Figure 2: 3-unit test environment setup for the 2019 small study, with single unit setup on the right.
screens for Reactor 1 at the workstation for Reactor 3. Figure 2 shows the test environment set-up.
The findings from the small study are documented in Eitrheim et al. (2020); this is a technical report available to members of the Halden HRP and Hal- den HTO projects only. The report documents that participants were able to successfully interact with and operate the 3 units, that unit confusion was not verified, and that multi-tasking across units did appear to increase operators’ workload but did not appear to result in errors or failure to complete tasks.
EVALUATION OF THE SIMULATOR FOR RESEARCH STUDIES As noted in the introduction, further studies with licensed operators were planned for 2020, but these had to be cancelled due to travel restrictions as a result of the COVID-19 pandemic. Instead, the research team decided to utilise the opportunity to perform an evaluation of the basic principles iPWR simulator, to determine the capabilities and limitations for future research studies. The full findings from the small study are documented in Blackett and Lunde-Hanssen (2020). Note that this is a technical report available to mem- bers of the Halden HRP and Halden HTO projects only. The main findings from the workshop are summarised in this paper.
The evaluation was performed in a workshop format, as two half-day video meetings. The iPWR simulator was set up in the Futurelab, and displayed to the workshop participants via webcam, to support discussions as necessary. The participants comprised three cognitive and experimental psychologists with significant experience of conducting experimental studies in HAMMLAB, a human factors expert who chaired the workshop, and two process experts with extensive nuclear operating experience. The primary goal of the workshop was to evaluate the feasibility of the test environ- ment used in the 2019 small study, to determine whether it would match the long-term research and experimental needs of the HRP activity on SMRs. A secondary goal was to evaluate an experimental workload measurement tool that was used in the small study; the results of this are not relevant to the
84 Blackett
discussion of the simulator capabilities or limitations and are not reported in this paper.
Summary of Evaluation Methodology To discuss the feasibility of the test environment, the workshop participants discussed the following questions in relation to specific aspects of the iPWR 3-unit control room environment that had been used in the 2019 small study:
1. Does the [simulator control room] layout fulfil minimum requirements for allowing one operator to monitor three units? 2. Does the alarm system fulfil minimum requirements for allowing one operator to monitor three units? 3. Does the human-machine interface (HMI) fulfil minimum requirements for allowing one operator to monitor three units? 4. Can we assume that the iPWR simulator represents a highly automated plant? 5. Can we create the scenarios we want?
The following additional keywords were used to guide the discussion for each question listed above:
Control room layout: size, viewing angles, readability, flexibility/configu- rability, consistency.
Alarm system: tiles, sounds, lists, discriminate urgency vs. units (systems).
HMI: situation awareness for multiple units, overview displays, trends, intervention at one unit, navigation.
Degree of automation: expected automated features, operator role.
Scenario design and instructor system: flexibility and timing of events and malfunctions, systems not simulated or not operable, number of units, common cause failures, differences between units.
In addition to discussion of the above questions, the participants rated the specific aspect using a “traffic light” system as follows.
RED – this aspect is not feasible for research studies, as redesign and extensive changes would be needed. The advice is to not proceed with the current solution, or else to try to modify this.
YELLOW – this aspect not feasible, or feasibility is disputable. Moderate changes would be needed.
GREEN – this aspect is feasible, meaning that the current solution meets the minimum requirements and/or minor changes would be needed.
Summary of Evaluation Findings Control room layout: the layout of the control room was considered good enough for operation of a 3-unit plant, and the spatially fixed setup was con- sidered useful to combat unit confusion by preventing operators from being able to open displays for any unit from any workstation. Some issues were identified with the overview display: (i) level trends shown on the overview display were difficult to read and should ideally be shown on the operator
Evaluation of a Basic Principles SMR Simulator 85
workstation displays; (ii) the overview display does not show the full rea- ctor and turbine plant, which should ideally be visible at all times; and (iii) it would be useful to show alarm logs on the overview display so that operators can easily see when alarms are initiated.
The team noted a limitation of the simulator, which is that it does not include, for example, electrical systems. This could limit the complexity of future experimental scenarios which might require operators to check and/or manipulate the electrical systems. The team noted that, although the spatially dedicated is useful to distinguish between units, it could be helpful to have a fixed screen in the centre displaying values across all units for easier and quicker comparison.
Overall, the layout of the control room and displays was rated as GREEN, meaning that with only minor changes (i.e., moving the level trends to the operator workstations and adding an alarm log to the overview display), the layout is conducive to future research studies.
Alarm system: the alarm system supplied with the simulator consisted of alarm tiles that blink orange when initiated, without any audible sound. A process expert developed a supplementary system that presented a separate window with a short information message about the alarm, as shown in Figure 3. In addition, a verbal alarm was added indicating which unit was in alarm.
In the discussion, the research team noted that while this simple alarm system was sufficient for the small study in 2019, it is not representative of how alarms would be presented at a plant. Furthermore, it does not ena- ble simulation of multiple alarms from multiple units at the same time, thus restricting exploration of how operators would handle multi-unit disturba- nces. This is a limitation of the simulator for the types of research studies that we wish to conduct in the future, where we would expect to have more complex scenarios with a greater number of alarms, to mimic a realistic SMR control room environment more closely.
Overall, the alarm system of the iPWR was rated as RED, meaning that extensive redesign would be needed for use in future research studies.
HMI: the team agreed that the HMI supplied with the simulator was suffi- cient for the 2019 small study. As noted earlier, the basic principles simulator does not include systems that would normally be included in a full-scope
Figure 3: The iPWR alarm system with activated alarm tiles in orange (left), and the later-developed alarm popup (right).
86 Blackett
simulator, such as the electrical systems, cleaning systems for reactor and tur- bine, charging letdown system, condensate polishing on turbine, spent fuel cleaning and cooling, and health systems. This limits the complexity of the scenarios that can be run on the iPWR simulator.
The research team noted that the iPWR HMI shows numerical values only and does not display any mini trends that would be more commonly seen in a modern control room. This affects the operator’s ability to quickly get a sense of the situation from the overview display.
In summary, the team rated the HMI as GREEN, noting that the HMI itself was not problematic but rather it is the limitations of the simulator itself that prevents running more complex scenarios to explore the human factors challenges of SMR operation.
Degree of automation: as noted earlier in the paper, it is difficult to know what constitutes a representative SMR design or level of automation due to the lack of publicly available information and experience on SMR opera- tional concepts. Thus, the discussion amongst the research team on degree of automation was speculative, based on the expectation that SMRs will be highly automated (US NRC, 2021). The team discussed possibilities for a highly automated SMR which included more automation on reactor prote- ction systems, such as the ability to automatically isolate containment in the event of a leak, or to isolate feedwater or steam pipes if a leakage was dete- cted on these systems. The team also noted that modern conventional nuclear power plants can already be considered as “highly automated” and conclu- ded that the iPWR simulator could not be considered highly automated since it does not appear to have more automated systems than the PWR simulator also used in HAMMLAB.
Overall, the team rated the degree of automation of the iPWR simulator as YELLOW, because the degree of automation is disputable, and noted that the degree of automation in the simulator may be considered representative of this type of SMR.
Scenario design and instructor system: the iPWR simulator did not come with an instructor system, meaning that plant malfunctions had to be pre- programmed into the experimental scenarios at specific times. This runs the risk of high variability in how the scenarios unfold because not all opera- tors will respond in the same way or according to the same timescales in a scenario.
Furthermore, the lack of an instructor system meant it was not possible to “hide” the intentions of the simulator instructor; when we wished to intro- duce a malfunction, the instructor had to do this at the operator workstation in front of the participant. This may reduce the level of realism of the scenario for the participant and may also cue them on the direction of the scenario. The simulator also lacked the ability to save trends, did not include a logging system, and it was not possible to save process states to “jump into” a scena- rio at a specific point to reduce lead time. These are all highly useful features of a research simulator.
In summary, the team rated the scenario design and instructor system as RED due to the difficulty experienced in designing and running scenarios of the type that we would typically utilise in our HTO research studies.
Evaluation of a Basic Principles SMR Simulator 87
CONCLUSION
The goal of the evaluation workshop was to evaluate the features of the basic principles iPWR simulator, and from this determine what kinds of experimental research studies would be possible with such a simulator. The workshop participants agreed that the basic principles simulator would ena- ble small-scale exploration of human performance issues such as: (i) operator monitoring strategies and unit confusion; (ii) reactivity control across mul- tiple units; and (iii) prioritisation of multi-unit disturbances. However, the team noted that such studies would be exploratory in nature and may require a hybrid approach using the simple simulator with static images, micro task- style data collection, workshops, or other methods to collect more detailed information about the issues.
It is important to state that the evaluation conclusions, and indeed this paper, are not intended as a dismissal of the IAEA’s iPWR simulator. It is developed as a basic principles simulator and as such it correctly fulfils that function. The small study performed using the iPWR simulator gave us inva- luable insights into potential concepts and conducts of operation for a generic multi-unit SMR control room. In 2021, the Institute for Energy Technology commissioned the development of a full-scope multi-unit SMR simulator, based on knowledge gained from our experience of using the basic principles simulator and experience from other full-scope simulators. The new full- scope simulator was installed in a six-unit setup in HAMMLAB in July 2022 and a first small study was performed within the Halden HTO programme with licensed operators in August 2022.
ACKNOWLEDGMENT
This work was supported by the OECD Nuclear Energy Agency (NEA) Halden Human-Technology-Organisation (HTO) Project. Thank you to the members of the Programme Review Group of the Halden HTO Project for review and constructive comments. The author would like to acknowledge the other members of the HRP/HTO SMR research team: Maren Eitrh- eim, Rob McDonald, Gyrd Skråning, Alexandra Fernandes, Håkon Jokstad, Daniel Odéen and Marten Bloch.
REFERENCES
Blackett, C., & Lunde-Hanssen, LS., (2020). Operation of Small Modular Reactors: Findings from Two Workshops in 2020. Report No. HWR-1297. OECD Halden Reactor Project, Halden, Norway. Blackett, C., Eitrheim, M. H. R., & Bye, A. (2022a) The Challenge of Assessing
Human Performance and Human Reliability for First-of-a-Kind Technologies. In Proceedings of the 16th International Conference on Probabilistic Safety Assessment and Management (PSAM 16), Honolulu, Hawaii, 26 June-1 July 2022. Blackett, C., Bloch, M., Pichancourt, I., Poret, C., & Jeffroy, F. (2022b). Small Modular Reactors – State of the Art Review 2021. Report No. HTOR-011. OECD NEA Halden Human Technology Organisation Project, Halden, Norway.
88 Blackett
Blackett, C., Eitrheim, M. H. R., McDonald, R., and Bloch, M. (2022c). Human Performance in Operation of Small Modular Reactors. In Proceedings of the 16th International Conference on Probabilistic Safety Assessment and Management (PSAM 16), Honolulu, Hawaii, 26 June-1 July 2022. Boring, R. L. (2010). The Use of Simulators in Human Factors Studies Within the
Nuclear Industry. In: Skjerve, A., Bye, A. (eds) Simulator-based Human Factors Studies Across 25 Years. London, England: Springer. Eitrheim, M. H. R, Skraaning, G., & Fernandes, A. (2020), Operation of Small Modular Reactors: Results from the 2019 SMR study in FutureLab. Report No. HWR-1292. OECD Halden Reactor Project, Halden, Norway. International Atomic Energy Agency (IAEA). “Advances in Small Modular Reactor
Technology Developments”, International Atomic Energy Agency, 2020, Vienna, Austria. Skjerve A. B., Bye A. (Eds.). (2011). Simulator-based human factors studies across 25 years. London, England: Springer. US NRC. (2021) “Risk-Informed and Performance-Based Human-System Consi-
derations for Advanced Reactors (Draft white paper)” United States Nuclear Regulatory Commission. Washington D. C., USA.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors and Simulation
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-59-9
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional