Thomas Ulrich, Ronald Boring, Roger Lew, Jisuk Kim, Dylan Jurski, Olugbenga Gideon, Kelly Dickerson (2025). Rancor-HUNTER - data collection and virtual operator modeling tool. In: Human Error, Reliability, Resilience, and Performance. AHFE Open Access, vol. 173, pp. 129–138. https://doi.org/10.54941/ahfe1006287
Historically, human reliability analysis (HRA) methods require analysts to develop static models of human error within a predetermined human failure event through expert estimation. Objective quantitative models of human performance in the form of a virtual operator offer an avenue to perform dynamic HRA through Monte Carlo simulation. Idaho National Laboratory developed the Human Unimodel for Nuclear Technology to Enhance Reliability (HUNTER) as a dynamic HRA framework and has demonstrated success in modeling existing scenarios. More data is needed to create generalizable virtual human models to explore undefined human failure event space. The Rancor Microworld Simulator is a simplified nuclear process control simulator that students can easily train to use and that can be modified to support the development of concepts of operation for advanced reactors. Rancor-HUNTER fills the data collection niche by integrating the Rancor Microworld Simulator with the HUNTER software. Rancor automates human performance data collection with a digital procedure system compatible with HUNTER’s procedure database through their integration. This digital procedure system provides implicit task level goals associated with each step that affords automatic HRA coded data collection. Therefore, high-resolution sequences of operator behaviors can now be automatically recorded with error rates and time distributions, which collectively represent a virtual human. A thermal power dispatch concept of operations development use case highlights benefits of Rancor-HUNTER to augment human factors evaluation through automated human performance data collection, which can then be used to also inform HRA models.
Advances in distributed control system technologies and manufacturing capabilities have eliminated many of the uncertainties present in complex process control systems. Current materials science and advanced manufacturing capabilities afford reliable components for use in advanced reactors. Digital control systems add robust control schemes to protect
equipment during operation and enable online condition monitoring to further enhance systems resilience through strategic maintenance upon performance degradation detection. As these traditional sources of potential system failures are eliminated, the remaining uncertainty associated with system failure resides within the human operation of the system. Human reliability analysis (HRA) aims to account for the human actions on the system as it relates to the system’s reliability. HRA accounts for, or credits, human actions that add to system safety and resiliency by leveraging the innate human problem-solving capabilities for novel and unplanned events. HRA also accounts for failure events in which operators take unsafe actions that degrade the safety and resiliency of the system in specific form of human failure events.
Traditional probabilistic risk analysis (PRA) and HRA methods emerged primarily as retrospective analysis of as-built nuclear power plant (NPP) failure events after they occurred. Analysts gathered operational information about the event to then build a fault tree to trace the source of the failure to its initiating cause. This analysis can then be generalized to other NPPs and other systems that possess similar characteristics to screen for the likelihood of this event occurring in these other NPPs and systems. Results of the retrospective analyses applied to a specific NPP or system that yield a suitably low human error value require no further mitigation. Results yielding high estimated failure probabilities that then support risk- informed decision- making to modify relevant systems or operations to add protections reduce the likelihood of the event occurring again. Proposed mitigation can be reevaluated with the model to identify which individually or collectively provide the best human failure probability reduction. Based on the cost and human failure probability reduction values, the organization can then have an informed decision toward what is implemented. Risk models are also constructed prospectively to evaluate failures for systems that have not experienced a substantial failure. The methodology is the same, but it is simply not guided by the historic failure context. In either type of analysis, the models and the quality of their results are dependent on the methods used to estimate failures and the data used to construct these models. Recently, PRA models have advanced substantially, and HRA must advance similarly to support new methods, such as dynamic or simulation-based PRA.
Prospective PRA and HRA methods are used to support new deployments such as those required for license amendments for existing plants or initial licensing of new plants. Such approaches benefit from dynamic modeling that anticipates different risk outcomes. Dynamic PRA and HRA are used in support of prospective PRA. Prospective PRA, like its retrospective counterpart, makes use of historic data collected by the utilities and vendors on the failure rates of equipment. The nature of equipment failures and the perception that these failures are expected to be increasingly over the operational time of the equipment lends itself to building databases characterizing the equipment failure. Unfortunately, the very nature of human failure and the perception of lower tolerance toward human failure has made it more challenging to build databases of human failure characteristics. First, human failures are less constrained than equipment
Rancor-HUNTER - Data Collection and Virtual Operator Modeling Tool 131
failures in that they can manifest in an infinite space of possibilities. Second, those committing the human error as well as those observing the human error have perceptions of the error that may lead to failures in recording those errors. At the organization level, U.S. utilities have historically been reticent to record, let alone share, human failure data at resolution suitable to support human failure database development. A common, and well founded, perception is that any human failure is negatively viewed as a failure of the individual or the operational oversight, such as training, and therefore, it represents a potential risk to the licensee operating the NPP. As such, utilities have mostly adopted an approach to capture primarily regulatory-required human performance metrics for U.S. Nuclear Regulatory Commission (NRC) reporting purposes. Indeed, prior efforts to create equivalent databases experienced some success, such as Scenario Authoring, Characterization, and Debriefing Application (SACADA) Chang et al. (2014). Although well- conceived, to date these efforts have fallen short of the overall data needed by the HRA research community for developing dynamic HRA methods and models compatible with prospective PRA.
The data dearth issue is not new, and there is little consensus on an agreed-upon model of human failure. A unified theory of human failure is beyond the scope of this paper. However, a central theme of this work is to collect data that is suitable for deriving human failure models, which ultimately aids the progress toward identifying a base model of human error suitable for nuclear process control. This paper presents the synthesis of two intersecting lines of research from Idaho National Laboratory focusing on (1) dynamic HRA data requirements and collection strategy and (2) model construction, simulation, and outputs for dynamic HRA. Rancor-HUNTER (Human Unimodel for Nuclear Technology to Enhance Reliability) Boring et al. (2024b) is a dynamic HRA software solution using a virtual operator to simulate operator behaviors across various plant state contexts across normal and abnormal scenarios. Rancor-HUNTER is comprised of two formerly separate software solutions, the Rancor Microworld Simulator Ulrich et al. (2017) and HUNTER Boring et al. (2021), integrated together to function as a single platform or ecosystem. A common digital procedure system structure implemented through this integration eliminates much of the workflow encountered when using these tools in isolation. The digital procedure system serves as a bridge to the use cases for Rancor and HUNTER; thus, this is the logical place to start in an explanation of the integrated concept that is Rancor-HUNTER. This first presents each of the two formerly separate tools and then describes the workflow and the efficiencies gained in leveraging the digital procedure system in both.
HUNTER HUNTER Boring et al. (2021) started as a framework to perform dynamic HRA and has since been developed into a software tool under the same name. Other dynamic HRA methods exist Boring et al. (2015), most notably ADS-IDAC Coyne and Mosleh (2014). The existing dynamic HRA methods have great utility; however, they are complicated tools that historically
have not led to widespread adoption within the research community or among practitioners at nuclear power plants. HUNTER adheres to a philosophical perspective that any viable solution should be accessible and usable. A scheduler module coordinates the Monte Carlo simulations of the user- defined simulation comprised of three modules, which are the individual, task, and environment. The environment is defined as a scenario comprised of the dynamic plant model, initial starting conditions, and malfunctions that induce scenarios that require operator response.
Possible operator responses are defined in the task module using procedures adapted from sampled operating procedures from existing U.S. light-water reactors. The procedures contain logic that determines their suitability based on component states and actions that can be taken on components to maneuver the plant toward a desired state.
Last, the individual module defines contextual factors in the form of SPAR- H performance shaping factors (PSFs) Gertman et al. (2005) that govern the timing and failure rates for completing procedure steps contained within the task module. The PSFs can be configured by the analyst as a static multiplier value, event-based trigger using specified parameter combinations, or dynamically calculated to adjust to the evolving scenario parameters based several available functions. While executing the procedure logic, the virtual operator is emulated by applying the PSF multipliers to the sampled time distribution over multiple instances using a Monte Carlo simulation.
As differential time is accumulated to achieve a sequence of steps and procedures for a scenario, the success or failure of the virtual operator to respond to the scenario can be determined in several different ways. First, meeting explicit failure conditions such as key parameters exceeding preconfigured limits can terminate the scenario as a failure, i.e., reaching a high core temperature that may lead to core damage. The allowed time for the response can be exceeded though the goal state may be achieved. Lastly, sufficient elapsed time can result in an inability to achieve the goal state when logic of the procedure has been exhausted without arriving at the goal plant state. In all cases, additional metrics beyond what is typically calculated in a traditional static model are captured by the analysis.
RANCOR The Rancor Microworld Simulator was developed to provide an alternative platform to evaluate nuclear process control issues outside of full-scope training simulators that represent the gold standard in HRA and human factors studies in the nuclear domain Ulrich et al. (2017). The infeasibility of current full-scope approaches preclude performing the number of studies needed to collect the large volume of data required for building quantitative models of human error. The need for an alternative arises from the cost and complexity of full-scope simulator studies. Both cost and complexity severely restrict these studies to large research institutions. The laboratory infrastructure to support a full-scope study is a substantial, initial capital cost. A dedicated team of information technology, computer science, and domain-specific experts are required to manage the simulator hardware and
Rancor-HUNTER - Data Collection and Virtual Operator Modeling Tool 133
software models. Experiments seldom focus purely on human performance but rather evaluate new technology integration to the main control room, and as such, simulator models or human-machine interfaces represented by virtual panels or digital control systems require substantial expertise. Last, developing scenarios to perform the evaluations requires a high level of knowledge of nuclear operations. As such, any human performance research leads pursing full-scope studies must have knowledge across several domains, which further restricts the pool of HRA researchers that would even attempt such an endeavor. Due to these challenges, there are only a handful of facilities scattered across the globe that actively perform full-scope simulator studies to support HRA and participate in international efforts to build databases of human performance. A new method to collect data is needed if the field hopes to expand research beyond these expensive and complex facilities.
The Rancor Microworld Simulator is a simplified simulator developed to functionally emulate nuclear power plant main control room operations of a pressurized-water reactor. Rancor is proposed as part of a solution to reduce the barriers of entry for researchers to collect meaningful human performance data in a representative nuclear process control context. Rancor possesses several features that afford a much larger data collection than can be achieved in traditional full-scope settings. The simplicity of the models and gamification ensure that naïve users can undergo short, 30-minute, training and be capable of generating useful human performance data. Training can be extended to close the performance and behaviors gap between naïve and experienced operator participants, depending on the objectives of the study. Once trained, participants can perform many more scenarios due to the accelerated simulation speed to present scenarios in a 5–10 minute duration as opposed to full-scope scenarios ranging anywhere from 10–240 minutes. Lastly, the data logging within Rancor supports automatic recording of plant process parameters, simulation events, and operator actions, which serve as high resolution record of each participant’s scenario completion. More recently, a digital procedure system was added to Rancor to expand the research utility of rancor further.
DIGITAL PROCEDURE SYSTEM Procedures act as decision aids, structuring situational context with logic to help decision-makers interpret, diagnose, and identify appropriate actions. They guide decision-making and response execution to achieve desired outcomes while preventing undesired ones.
Procedures achieve two key goals: consistency and improved performance. They serve as administrative controls, ensuring tasks are performed as intended and approved by management and experts. This reduces human error and operational risk. Consequently, the NRC requires procedures for reactor operation and maintenance, as outlined in 10 CFR 50.73 Appendix C, which describes procedures as “a predetermined set of decisions and actions.”
While rote execution of procedures can ensure administrative control, but their true utility can be achieved in combination with proper implementation and training. Procedures enhance human performance by providing
structured context and logic, serving as memory aids and diagnostic guides. This positively impacts operators’ cognitive processes, leading to better system reliability than mere rote execution.
However, procedures alone are not sufficient to prevent undesired outcomes. Operators, with their unique problem-solving capabilities, add resilience by applying their knowledge and skills. They can recognize when procedural objectives misalign with unforeseen operational circumstances and adjust accordingly. This human element is crucial for achieving the desired outcomes and maintaining system reliability.
As stated previously, existing light-water reactors use paper-based procedures and maintain these procedures with a change management system. The Final Safety Analysis Report requires initial procedures, but licensee procedure changes after initial approval are not subject to additional NRC approval outside of yearly inspections. This treatment of paper-based procedures has far reaching and significant implications for digital procedure systems implanted in nuclear facilities.
Computerized operating procedure systems, more modernly referred to as digital procedure systems (DPS), present procedures within an electronic visual display as defined in IEEE-Std-1786 IEE (2022). This basic definition aims to differentiate the digital procedure representation from that of traditional paper procedures. IEEE-Std-1786 further characterizes DPS within a three-level hierarchical structure as follows. Type 1 procedures recreate the basic paper procedure in digital form. Type 2 procedures include live process parameter values (i.e., indicators) embedded within the step instruction and can include logical evaluations to denote whether the current state of the parameter upholds the logic of the step. Type 3 procedures extend the functionality of Type 2 with the addition of embedded soft controls such that the participant can perform the prescribed actions without having to use the typical control interface. NUREG/CR- 6634 provides a more comprehensive characterization scheme which defines several types of procedure functions across four main categories of monitoring and detection, situation assessment, response planning, and response implementation as can be seen depicted below O’Hara et al. (2000):
Monitoring and Detection
Process Parameter Values
Operator actions
Situation Assessment
Procedure entry conditions
Resolution of procedure step logic
Procedure history
Context sensitive step presentation
Assessment of continuous, time, and parameter steps
Assessment of cautions
High-level goal attainment and procedure exist conditions
Rancor-HUNTER - Data Collection and Virtual Operator Modeling Tool 135
Response Planning
Selection of next step or procedure
Procedure modification based on current situation
Response Implementation
Transition from one step to the next
Transition to other parts of procedure or other procedures
Control plant equipment
Figure 1 provides an example of how these elements manifest in aggregate with screen captures from the Rancor Microworld Simulator experimental digital procedure system developed by INL to support human-in-the-loop concepts of operations testing Ulrich et al. (2023), Park et al. (2023).
A DPS was developed for Rancor Boring et al. (2025, in press) to enable human factors research on computerized-operating procedures while also bolstering the simulators usability for naïve participants with less experience in nuclear process control. The implementation of the DPS has been mapped to the NUREG/CR-6634 characterizations in Ulrich et al. (2023). The details of the actual implementation cannot fit within this paper, but an example was collected from simulator studies and coded as seen in Figure 1.
Figure 1: Digital procedure system implemented in the rancor microworld with the ability to customize functionality and record operator actions and timing data.
One key feature of the DPS worth noting is the automatic analysis of task time and human error probabilities (HEPs) of operator actions when completing the procedures. Specifically, the DPS collects data that can be used to update the HUNTER generic GOMS primitives Boring and Rasmussen (2016). Task level primitives represent the most fundamental types of human activities, while procedure level primitives are calibrated to the procedure step level in which one procedure step may require multiple human actions. The GOMS task level primitives serve as a dictionary of basic human activities that can be chained together to describe a procedure step. Each GOMS primitive has a time distribution and HEP derived from several full-scope studies by analyzing manually logged procedure step times and outcomes mapped to types of GOMS primitives to generalize a time distribution for each GOMS primitive. These can be updated with new data
and should be calibrated to specific simulators based on a sample using that specific simulator to ensure system specifics are captured. This tuning requires modifying the GOMS primitives with respect to the smaller sampled timing from the specific simulator. The DPS is instrumental in this process going forward because it records and analyzes operator actions mapped to GOMS primitives, which greatly simplifies the process of using this data to update the generic GOMS primitives for a specific simulator, in this case the small module power plant model within Rancor. Furthermore, human error probabilities across the standard errors of commission or omission, in addition to several additional metrics, are automatically output by the DPS. It is through the implementation of the DPS that a natural coupling between the data collection capabilities with Rancor and virtual operator task module scripting needed for HUNTER emerged.
RANCOR-HUNTER HUNTER does not require a specific simulator for the NPP in the environment module. However, the simulator must have an application program interface to run the simulation incrementally, get and set parameter values, and facilitate plant status updates and operator actions. Components referenced in the task module procedure steps must be mapped to simulator point names to execute procedures effectively.
To support Rancor data collection, paper procedures were converted into digital structures containing point names corresponding to simulated power plant components. This work by Rancor’s DPS facilitates the use of DPS digital structures within HUNTER. A module was developed to categorize procedures in the DPS with GOMS task-level primitives, providing HUNTER the context needed to calculate task duration and associated human error probabilities.
The Rancor simulator environment recently transitioned to a server- client architecture, enabling HUNTER integration. This allows HUNTER to access plant models as a client and perform Monte Carlo Simulations Boring et al. (2024b).
CONCEPT OF OPERATIONS DEVELOPMENT USE CASE Recent research on the thermal power dispatch (TPD) concept illustrates how Rancor-HUNTER can support system development and evaluation. TPD involves diverting steam from electrical production during low demand periods to support industrial processes like high-temperature steam electrolysis for hydrogen production. This novel system for the U.S. commercial nuclear industry requires empirical evidence of safe operation to secure license amendments for TPD capabilities.
In the study, paper procedures were used in full-scope simulator scenarios, with a large team of observers manually logging operator actions. This data was later compiled and analyzed, which was time-consuming. The DPS system could have automated much of this data capture, requiring only the mapping of simulator point names, many of which were already known from
Rancor-HUNTER - Data Collection and Virtual Operator Modeling Tool 137
TPD controls development. This would have streamlined the analysis and reduced costs. Analysts could then focus on understanding complex operator activities, reported experiences, and biometric measures like eye tracking.
Finally, the data could calibrate GOMS primitives in HUNTER, allowing it to act as a virtual operator. This would help evaluate and quantify constraints of TPD operations for future procedures.
CONCLUSION
The combination of the Rancor Microworld Simulator and HUNTER as Rancor-HUNTER integrates the complementary functionality into a single platform. Specifically, bundling HUNTER with a simplified power plant model simulator has promise for accelerating research on HRA and human factors in the nuclear domain, since it provides a single and relatively accessible platform, at least when compared to full-scope training simulators. This integrated platform enables more research on human performance within nuclear process control. As advanced reactor deployment swiftly approaches in the next 5–10 years, it is crucial to address concept of operations as the operations Boring et al. (2024a) and maintenance costs will make or break the economic viability of adopting advanced reactors in the United States.
DISCLAIMER This information was prepared as an account of work sponsored by an agency of the U.S. Government. Neither the U.S. Government nor any agency thereof, nor any of their employees, makes any warranty, expressed or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness, of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. References herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by the U.S. Government or any agency thereof. The views and opinions of authors expressed herein do not necessarily state or reflect those of the U.S. Government or any agency thereof.
REFERENCES
David Gertman, Harold Blackman, Julie Marble, James Byers, Curtis Smith, et al. The spar-H human reliability analysis method. US Nuclear Regulatory Commission, 230(4):35, 2005. IEEE guide for human factors applications of computerized operating procedure
systems (COPS) at nuclear power generating stations and other nuclear facilities. Technical report, New York, 2022. Jinkyung Park, Taeryong Yang, Ronald L Boring, Thomas A Ulrich, and Janghoon Kim. Analysis of human performance differences between students and operators when using the rancor microworld simulator. Annals of Nuclear Energy, 180: 109502, 2023.
John O’Hara, Julian C Higgins, John Stubler, and Jay Kramer. Computer-based procedure systems: Technical basis and human factors review guidance. Technical report, Technical Report No. NUREG/CR-6634, 2000. Kevin Coyne and Ali Mosleh. Nuclear plant control room operator modeling within
the ads-idac, version 2, dynamic pra environment: Part 1-general description and cognitive foundations. International Journal of Performability Engineering, 10 (7), 2014. Ronald Boring, Roger Lew, and Thomas Ulrich. Rancor integrated procedure system (RIPS): A computer-based procedure development platform for advanced reactors. In Proceedings of the American Nuclear Society Nuclear Power, Instrumentation, and Controls and Human-Machine Interface Technology Conference (ANS NPICHMIT), 2025, in press. Ronald Boring, Thomas Ulrich, and Roger Lew. Early demonstration of human
factors concepts of operations for advanced reactors. In Transactions of the American Nuclear Society, volume 131, pages 372–375, 2024a. Ronald L Boring, Ulrich Thomas Lew, Roger, Jooyoung Park, Jisuk Kim, and Olugbenga Gideon. Human unimodel for nuclear technology to enhance reliability (hunter 3.0) user guide. Technical report, Technical Report No. INL/ RPT-24-80110, 2024b. Ronald L Boring, Ulrich Thomas, Jooyoung Park, Torrey Mortenson, Jeeyea Ahn, and Roger Lew. An adaptable software toolkit for dynamic human reliability analysis: Progress toward hunter 2. Technical report, Technical Report No. INL/ EXT-21-64525, 2021. Ronald Laurids Boring and Martin Rasmussen. Goms-hra: A method for treating
subtasks in dynamic human reliability analysis. In Risk, Reliability and Safety: Innovating Theory and Practice, Proceedings of the European Safety and Reliability Conference, pages 956–963, 2016. Ronald Laurids Boring, Jeffrey Clark Joe, and Diego Mandelli. Human performance modeling for dynamic human reliability analysis. In Vincent G. Duffy, editor, Digital Human Modeling. Applications in Health, Safety, Ergonomics and Risk Management: Human Modeling, pages 223–234, Cham, 2015. Springer International Publishing. Thomas A Ulrich, Roger Lew, Stephen Werner, and Ronald L Boring. Rancor: A
gamified microworld nuclear power plant simulation for engineering psychology research and process control applications. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, volume 61, pages 398–402. Sage CA: Los Angeles, CA: SAGE Publications, 2017. Thomas A Ulrich, Ronald L Boring, Roger Lew, and Timothy A Whiting. Rancor computer-based procedures–a framework for task level human performance data collection. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, volume 67, pages 1023–1028. SAGE Publications Sage CA: Los Angeles, CA, 2023. Yung-Jin Chang, Dennis Bley, Lauren Criscione, Barry Kirwan, Ali Mosleh, Tod
Madary, and Alex Zoulis. The sacada database for human reliability and human performance. Reliability Engineering & System Safety, 125:117–133, 2014.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Error, Reliability, Resilience, and Performance
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-49-6
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional