Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Studying Control Room Operations on a Shoestring Budget - Reflections on the Rancor Microworld

Publication year
Digital Object Identifier10.54941/ahfe1001488

Abstract

As the U.S. continues to develop and mature advanced reactor designs, the nuclear industry is becoming increasingly aware of the need for good human factors are to ensure safe, reliable, effective, and economical concept of operations. Advanced reactor designs aim to reduce staffing, and significant operational costs, by adopting high levels of automation. The highly automated control system designs must be informed with human factors and human reliability data. The proposed concepts of operations are unlike the current, largely manual, concept of operations found in operating nuclear power plants. Human performance data collection has proven difficult to obtain for existing nuclear power plants. Human factors researchers working on advanced reactor designs will encounter these same fundamental challenges and more. The novel concept of operations and accompanying human-system interfaces are novel and require human performance data for validation and licensing. Methods to evaluate novel concepts of operations for diverse advanced reactor designs must be identified to aid vendors in their system design activities. The Rancor microworld is a simulation platform that is currently used to support advanced reactor vendors in developing their control room concepts. The rationale and historical use of the Rancor microworld demonstrates a unique and complimentary approach to traditional full-scope simulator data collection methods that rely on expert licensed operators. The Rancor microworld is a reduced-order model of a small modular reactor conceived and developed to support human performance research on nuclear operations topics. The microworld represents the core elements of a nuclear power plant sans the complexity associated with full-scope simulators that are typically used to support human factors and human reliability research. The impetus for the microworld as an alternative method to acquire human performance data stems from the challenges in performing full-scope simulator studies. Full-scope simulators are expensive to build and maintain. Furthermore, they require extensive expertise to develop scenarios to support specific hypothesis testing. Operations data is historically difficult to obtain since even large research organizations that can afford a full-scope simulator facility encounter sample size issues. Licensed operators are expensive and fully time committed to their employing nuclear power plant. As such, it is very difficult to perform research on nuclear control room operations with sufficient sample sizes to approach statistical significance and draw generalizable conclusions applicable to different designs. Therefore, an alternative population using a simplified simulator offers an approach to evaluate human factors issues. Through numerous studies, the Rancor microworld has demonstrated an effective means to leverage inexpensive and ubiquitous student participants to expand the data collection capability and build a corpus of human performance data to inform advanced reactor control system designs and human reliability modeling. This paper provides an overview of the Rancor microworld studies and describes the benefits and disadvantages of using novice participants in simplified simulator environments in contrast with licensed operators in full-scope simulator environments.

INTRODUCTION

The Rancor Microworld is a reduced order model (ROM) nuclear process control simulator designed to support human factors and human reliability research (Ulrich, 2017; Ulrich, Lew, Werner, & Boring 2017). Rancor was developed with a simplified thermohydraulic and control system model inten- ded to support control room research outside of the traditional full-scope, full-scale simulator environments. Furthermore, the simplicity also supports the use of naïve participants in lieu of expert nuclear personnel primarily com- prised of licensed or formerly licensed nuclear operators. This paper describes the Rancor Microworld and its complementary role to support nuclear pro- cess control research and industry efforts that traditionally rely on full-scope full scale simulators.

To understand the unique capabilities and limitations Rancor provides, it is useful to understand how and why main control room (MCR) simula- tors were initially conceived and are currently maintained as exact replicates to any given nuclear power plant’s (NPP’s) actual MCR. MCR simulators arose as a response to a series of high-profile accidents that revealed the need for standardized and comprehensive operator training and qualification for severe accident response scenarios.

© 2022. Published by AHFE Open Access. All rights reserved. 27

28 Ulrich

In the wake of several nuclear accidents, such as Three-Mile-Island and Chernobyl, the Nuclear Waste Policy Act of 1982 mandated the U.S. Nuclear Regulator Commission to establish simulator facility, training program, and operator qualification testing regulations to address the personnel perfor- mance issues identified in these accidents. The requirements for a simulator facility, specifically those for a plant reference-simulator, are defined under §10 CFR 55.46. To comply, each licensee is required to maintain a refere- nce plant simulator to support operator testing and training. These reference simulators are identical copies to the actual MCR, which for the existing fleet of light water reactors consist of control boards with analog indica- tors and controls wrapping around the perimeter of the room. To the best of the author’s knowledge, control room operations research in the U.S. is still largely restricted to researchers with access to training simulators or access to a handful of unique one-off facilities purpose built to conduct the resea- rch, such as the Human Systems Simulation Laboratory (HSSL) at the Idaho National Laboratory (Boring et al., 2012).

Advances in monitors and computing technologies, via high resolution displays with accurate touch input, have led to the development of reconfi- gurable control room simulators that can support human factors engineering activities in addition to training. These new simulators use glass top bays to represent each physical control board panel on “glass” monitors. These were first introduced as classroom trainers and gained popularity through the work by Boring et al. to pioneer their use with rapid prototyping in full- scale studies using 15 bays to represent the major systems of an MCR. This configuration has been used to support several control room modernization efforts to upgrade the existing analog turbine control systems to digital vari- ants (Boring et al., 2012). The human factors process followed the guidance contained within NUREG-0711 (O’Hara et al., 2012), which is the primary guidance outlining prudent human factors activities necessary to design and evaluate control room modifications. The Rancor Microworld was conceived and developed within the context of the glass top simulator as it was used to support control room modernization. Rancor was purpose built to support human performance research in nuclear process control without the heavy cost or expertise associated with the full scope simulation.

RANCOR MICROWORLD ORIGIN Rancor represents the core elements of an NPP without the high-fidelity models of full-scope simulator typically used to support human factors and human reliability research. The impetus for Rancor as an alternative method to acquire human performance data stems directly from challenges to develop and execute a full-scope simulator study. Full-scope simulators are expensive to build and maintain. Furthermore, they require extensive expertise to deve- lop scenarios to support specific evaluation objectives. Operations data is historically difficult to obtain since even large research organizations that can afford a full-scope simulator facility encounter sample size issues. Licen- sed operators are expensive and fully time committed to their duties at their respective NPP. As such, it is very difficult to perform research on MCR

Studying Control Room Operations on a Shoestring Budget 29

operations with sufficient sample sizes to approach statistical significance and draw generalizable conclusions. Therefore, an alternative population using a simplified simulator offers an approach to evaluate human factors and human reliability issues. Through numerous studies, the Rancor Micro- world has demonstrated an effective means to leverage inexpensive and ubiquitous student participants to expand the data collection capability and build a corpus of human performance data to inform advanced reactor control system designs and human reliability modeling. The remaining secti- ons describe four independent research efforts to provide an overview of Rancor applications and account for the advantages and disadvantages of using novice participants in simplified simulator environments contrasted with the use of licensed nuclear operators in traditional full-scope simulator studies.

Rancor was developed by the same team that also performs full-scope simulator experiments using the HSSL. This perspective benefited the simu- lator development since it was designed by researchers as an experimental tool to support research. Specifically, the team possessed expertise in human factors, experimental psychology, cognitive science, neuroscience, nuclear process control, and computer science. Rancor was developed in support of a doctoral dissertation with the specific goal of creating a research plat- form that affords human performance and cognitive psychology investigation without the reliance on the full scope simulator. Prior to the decision to develop Rancor, other simplified process control platforms such as the dual reservoir system simulation (DURESS II; Vicente & Pawlak, 1994) were ini- tially explored. A customizable variant of DURESS was developed and used in one pilot study. Ultimately this platform was abandoned for a fixed system design that was easier to maintain; however, the process behind this initial development was informative to the Rancor conceptualization.

INITIAL RANCOR VALIDATION STUDIES The focus for the first series of experiments was on the validation and via- bility of the platform to serve as a research tool (Ulrich, 2017; Ulrich, Lew, Werner, & Boring 2017)). First, as psychology students represent one of the largest participant pools, it was deemed necessary for the platform to be rela- tively easy to learn without a nuclear or engineering background. Second, it was evaluated in terms of its ability to generate variability in nuclear process control performance and cognitive measures. Third, the platform was evalua- ted in terms of performance and cognitive measure variability as they differed between naïve students and experts with process control experience. Lastly, through training and experience using Rancor, the learning time course for students to demonstrate performance resembling experts was evaluated. For any results to be generalizable to nuclear process control domain, the simu- lator should generate variable effects between and within experts and naïve students. It is challenging but nonetheless necessary to ensure the simulator is sufficiently complex to capture variability among experts but also suffi- ciently simple that it can simultaneously generate variability among naïve participants.

30 Ulrich

During this initial set of experiments, participants received an hour of training including background on the systems, indicators, and controls for Rancor. Participants completed an experimenter guided practice session prior to four experimental trials. Each trial required the participant to manipulate the system from a shutdown state, i.e., reactor core with control rods fully inserted, to online power producing mode of operation, i.e., full reactivity with steam generators producing steam for the turbine with the generator synced to the grid. This represents a compressed set of normal operations.

Faults are often used to evaluate operator diagnostic and action respon- ses to a faulted component. Perturbations comprised of a spurious reactor or turbine trip were included in each trial, and these forced participants to diagnose and then execute a series of control actions specific to the trip type to restore normal operations. These scenarios are perturbations and not true faults since no components, sensors, or controls were compromised and all functionality was available. An actual fault entails a component casualty that forces operators to address the issue with varying levels of reduced system functionality. Future versions of Rancor addressed this shortcoming and will be described in subsequent sections.

Student performance and learning effects were compared to a sample of steam plant and nuclear power plant operators. There was evidence for an initial effect of expertise on primary task performance, but the relationship was more complicated due to interactions for the cognitive factors of situ- ation awareness, workload, and attention. This initial set of experiments established basic viability for Rancor to support nuclear process control research.

SOUTH KOREAN RANCOR VALIDATION STUDY The second major Rancor-based research activity aimed to gather human per- formance data to support human reliability analysis (HRA). HRA typically uses full-scope simulator data with operating crews (Massaiu, et al., 2010); however, this proves to be an expensive and time-consuming process. Furth- ermore, it is difficult to acquire sufficient data and effect sizes to properly quantify human error classification categories and rates. The suitability of Rancor as a platform to evaluate human performance was the central que- stion for this study as it could potentially expand data collection capabilities if the results are representative for a traditional full-scope simulator study. To validate the use of Rancor for HRA data collection, it was necessary to compare metrics of human error in Rancor using an existing framework to determine if the metric assessments were representative to human error measured in a full-scope simulator.

To perform the validation against existing full-scope simulator data, a collaboration with a South Korean University, the Korean Atomic Energy Research Institute, and a Northwestern U.S. university was performed to test Rancor’s ability to evaluate human errors using the HuREX HRA method (Park et al., 2022). Twenty nuclear engineering students and twenty licen- sed reactor operators from South Korea completed ten scenarios requiring the detection, diagnosis, and response to plant faults. Rancor was updated

Studying Control Room Operations on a Shoestring Budget 31

to support actual component faults beyond the initial perturbation scenarios used in the original validation experiment. Furthermore, procedures were drafted to support normal operations and emergency responses, such as a steam generator tube rupture response procedure.

The results of the study demonstrated promise for Rancor to generalize, at least with respect to HRA. There were some notable differences. The ratio of errors of commission to errors of omission was larger in Rancor than what was observed in previous full-scope data collection studies. The participants committed more errors than they omitted correct actions respectively. Furth- ermore, the rate of errors overall was significantly higher in Rancor. Rancor appeared to overestimate the human error probability in comparison to a full scope simulator based on this result. Additional research is needed to further understand these differences, but the ability to capture these errors is a fruitful outcome. It is unreasonable to assume the error rates would match, but the amount of variability should be predictable. Future work is planned to further explore the use of Rancor to gather HRA data and examine how to extrapolate the results systematically to augment existing human error data sets.

THERMAL DISPATCH SYSTEM DESIGN AND EVALUATION The third research activity Rancor support focused on using student parti- cipants to evaluate usability issues for a prototype thermal dispatch system human-machine interface (HMI) intended for integration with existing U.S. commercial light water reactors. This research demonstrates the use of Rancor in tandem with a full-scope simulator. The thermal dispatch system extracts steam from the main steam before the turbine and diverts it to a nearby chemical process, such as hydrogen production plant. The rese- arch aimed to develop an HMI prototype display to control the thermal dispatch system and demonstrate this with a full-scope full-scale simulator running GSE System’s Generic Pressurized Water Reactor model (gPWR; Ulrich et al., 2021). The prototype HMI was developed with input from licensed operators during a quasi-dynamic evaluation. After this evalua- tion, Rancor was modified with a model of the thermal dispatch system and the prototype HMI, which in itself is relatively simple for a nuclear process control system. Students performed the same test scenarios presen- ted to the operators in the quasi-dynamic evaluation, and their feedback was collected. Though students lack the nuclear expertise to comment on operational impacts of the thermal dispatch functionality, their perceptual capabilities are similar to licensed operators, and therefore human factors issues were expected to be reported. Strikingly, the students and operators reported similar types of usability issues, since the execution of the ope- rations and the HMI itself was nearly identical. Therefore, the students augmented the small sample of operators and afforded a more compreh- ensive evaluation with greater confidence in successfully detecting all per- tinent human factors issues. The university collaborators then refined the HMI design based on the outcomes of the student evaluation. This refi- ned prototype was then tested again with licensed operators across fifteen

32 Ulrich

different trials including normal and fault scenarios using the full scope simulator.

The simplified plant model highlight both the advantage and disadvantage for Rancor to support control room operations research. The appropriate use is critical to ensure that the results translate back to the nuclear process con- trol domain. Within the context of this thermal dispatch system development, it was necessary to first have a viable but still immature design and concept of operations before modifying the Rancor models and testing the general usability of the HMI and operations within the simplified environment with naïve participants. Experts must provide input and verify the design before it is translated to the reduced order model and tested with students; otherwise there is no basis for the validity of the testing, and it yields little benefit. Subsequently, the usability issues identified by the students must then be validated, which is already ensured in control room HMI development activi- ties through NUREG-0711, human factors program review model (O’Hara, 2012). Rancor is best suited to serve as a formative phase simulator platform performed as an intermediate evaluation. The primary benefit for employing Rancor to address usability issues is the cost savings avoided by not using the full-scope simulator to vet the design for general usability issues. The full- scope simulator can then be used for high-fidelity scenario testing in which the plant responses and operator feedback are of more concern. The scenarios can avoid becoming bogged down in usability issues and the focus can remain on evaluating the HMIs ability to support the primary task for controlling the thermal dispatch system as it was the case for the second operator-in-the-loop study of the thermal dispatch system development (Ulrich et al., 2021).

ADVANCED REACTOR CONTROL ROOM DEVELOPMENT Numerous advanced reactor vendors are faced with developing the control room concept that affords efficient operations through high levels of auto- mation. Unlike existing commercial plants, fully developed thermal models are typically not available to support designing and testing the control room concept. Deferring the control room design and evaluation until the reactor design is finalized is unwise, as this would push the operational considerati- ons to the tail end of the overall system design process and leave little room to address any deficiencies. This is especially prescient for advanced reactor vendors, since reduced staffing is a requirement to remain cost competitive during operations. Rancor provides an agnostic simplified model that can be coupled to control room design concepts to support evaluation. Through a collaboration with an advanced reactor vendor, Rancor was used to per- form user preference and performance evaluations for some basic HMI design concepts, representing the initial efforts to develop a control room concept. As mentioned in the previous section, Rancor is not well suited to test the robustness of the HMI to support specific operations in comparison to a full- scope simulator. Testing the specific implementation of control schemes for specific systems is not useful in the absence of fully matured system design. However, Rancor can and has been used to develop the reactor vendors style guide comprised of the graphical elements, nomenclature, navigation, and

Studying Control Room Operations on a Shoestring Budget 33

layout of the HMIs for the control room design concept. To date one study has been performed and at the time of this writing, another is in progress. The first study illustrates how Rancor can be used to support advanced reactor control room design.

In the study, the Rancor HMI graphics were stylistically altered to repre- sent competing industrial process control design styles. Four different styles were implemented including the basic Rancor design, neumorphic, high per- formance HMI,and three-dimensional graphics. Static images were generated for each design with several different Rancor states. An online survey was developed to assess user preference and capture limited performance data with questions prompting participants to identify states or values within each display style. The collaborating vender provided engineering and operational personnel to complete the survey. Preferences, accuracy, and performance were captured and analyzed. The results identified a preference for the basic Rancor style. The three-dimensional style exhibited the greatest accuracy and shortest response times to identify states or values. These results were contrary to expectations that predicted the neumorphic style to be the most preferred and the three-dimensional to demonstrate the poorest performa- nce. Rancor follows a simple industrial design aesthetic and therefore its preference is not surprising. Familiarity may also have driven performance as participants working in this domain encounter three-dimensional graph- ics routinely despite their typically poor human factors. Familiarity may have augmented participants’ ability to interpret the display states and improve their accuracy and response times. Regardless of the study outcome, Rancor was an effective platform in identifying preference and performance across different styles. Furthermore, since Rancor is system design agnostic, the style evaluation alleviated vendor reported issues in which participants became distracted by the system engineering design as opposed to the style. The follow-up study moves beyond the static image survey and tasks participants with controlling Rancor across small scenarios.

CONCLUSION

Rancor was developed to support existing research on control room conce- pts by offering a simplified nuclear process control environment to evaluate human performance constructs using naïve student participants. It has been used to evaluate human factors issues and HRA across several different research efforts. Full-scope simulators remain the gold-standard for these activities, but Rancor provides unique capabilities that complement existing control room research.

ACKNOWLEDGEMENT

This work of authorship was prepared as an account of work sponsored by Idaho National Laboratory (under Contract DE-AC07-05ID14517), an age- ncy of the U.S. Government. Neither the U.S. Government, nor any agency thereof, nor any of their employees makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness,

34 Ulrich

or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights.

REFERENCES

Boring, R. L., Agarwal, V., Joe, J. C., & Persensky, J. J. (2012). Digital full-scope mockup of a conventional nuclear power plant control room, Phase 1: installation of a utility simulator at the Idaho national laboratory. US Department of Energy National Laboratory. Park, J., Boring, R. L., Ulrich, T. A., Lew, R., Lee, S., Park, B., & Kim, J. (2022). A

Framework to Collect Human Reliability Analysis Data for Nuclear Power Plants Using a Simplified Simulator and Student Operators. Reliability Engineering & System Safety, 108326. Massaiu, S., Bye, A., Braarud, P. Ø., Broberg, H., Hildebrandt, M., Dang, V. N., ... & Forester, J. A. (2010). International HRA Empirical Study, Overall Methodology and HAMMLAB Results. In Simulator-based Human Factors Studies Across 25 Years (pp. 253–269). Springer, London. O’Hara, J., Higgins, J., & Fleger, S. (2012). Human factors engineering pro-

gram review model (NUREG-0711) revision 3: update methodology and key revisions. In proceedings of 8th international topical meeting on nuclear plant instrumentation and control and human-machine interface technologies. Vicente, K., & Pawlak, W. (1994). Cognitive work analysis for the DURESS II system. Cognitive Engineering Laboratory, Department of Industrial Engineering, University of Toronto, Toronto, Canada CEL, 94–03. Ulrich, T. A., Lew, R., Werner, S., & Boring, R. L. (2017, September). Rancor: a

gamified microworld nuclear power plant simulation for engineering psychology research and process control applications. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 61, No. 1, pp. 398–402). Sage CA: Los Angeles, CA: SAGE Publications. Ulrich, T. A. (2017). The Development and Evaluation of Attention and Situation Awareness Measures in Nuclear Process Control Using the Rancor Microworld Environment. University of Idaho. Ulrich, T. A., Lew, R., Mortenson, T., Medema, H., Boring, R. L., Werner, S. &

Minard, N. (2021, July). A Dual Full-Scope and Reduced-Scope Microworld Simulator Approach to Evaluate the Human Factors of a Coupled Hydrogen Pro- duction Concept of Operations. In International Conference on Applied Human Factors and Ergonomics (pp. 179–186). Springer, Cham.

Cover of the proceedings volume Human Factors and Simulation

Explore the complete digital volume

Human Factors and Simulation

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-06-3

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →