Alexandra Fernandes, Alf Ove Braseth, Robert McDonald (2024). Measuring nuclear control room operators’ performance in the context of overview displays. In: Human Factors in Software and Systems Engineering. AHFE Open Access, vol. 140, pp. 135–143. https://doi.org/10.54941/ahfe1005046
Overview Displays have a potential to provide a high-level, comprehensive view of complex data, systems, or processes. They are used within control room settings to facilitate a quick understanding of the overall context. There is, however, scarce documentation of the effects of the use of overview displays on human performance. We ran two studies with two different overview displays. The two studies were intended to mimic different concepts and implementations of overview displays. In Study 1 the large screen overview display used a design concept consistent with the workstation displays design. In Study 2, an alternative overview display was presented, composed of several smaller/tiled screens, being graphically inconsistent with the workstation display design (e.g. different colour codes). The results indicate that there might be a cost on performance linked to the implementation of overview displays when the different interfaces available in the control room are incongruent. These findings are discussed attending to design and human factors approaches. We conclude with a reflection over the limitations of the study and possible recommendations on the use and implementation of overview displays in new and already existing control rooms.
Nuclear control rooms are highly complex and safety critical environments, where operators are responsible for monitoring and controlling plant status and managing several complex processes, assuring the achievement of plant goals while maintaining it on a safe state.
The introduction of Overview Displays (ODs) was motivated by a sec- ondary effect of digitalization processes in nuclear control rooms (Braseth et al., 2019; Vicente et al., 2001). In conventional analogue control rooms, all the available information about the plant was visible at all times in large wall mounted boards or panels, which were often operated through mechanical knobs and buttons. However, the migration toward hybrid and fully dig- ital control rooms implied that operators worked more time in their own individual computerized workstations, where operating several systems from the same screen was possible. This transition led to human performance and reliability concerns due to lack of visibility of the work performed by other members of the crew, which might impact the team’s situation awareness
and communication processes (e.g. Salo et al., 2006; Vicente et al., 2001). In this context the concept of ODs was proposed as a mitigation strategy to these challenges (Fernandes & Braseth, 2023). ODs intended to revolutionize the way the operators interacted with digital control systems by providing a high-level, comprehensive visualization of data and processes (Laarni et al., 2009). Previous work on ODs in control rooms has had a strong design focus, with few examples of empirical data collections to support the design assumptions for the integration of ODs. According to literature, large screen overview displays might not have the anticipated effects on situation awareness, com- munication, or diagnostic of plant status in nuclear control rooms (Kortchot et al., 2018). The authors argue that the configurational aspects are less rele- vant than the contents and design principles of the displays. Other empirical studies, revealed that participants were quicker to answer specific questions on process parameters when a (large screen) overview display was present (Fernandes et al., 2020). These findings are also congruent with a hypothe- sis raised by Hildebrandt (2022) whereby ODs could present a performance advantage simply by allowing all relevant information for a specific situation to be immediately and continuously available without the need for navigation to complete tasks.
There is still a need to understand the implications of the use of ODs in dig- ital and hybrid control room contexts. Grasping the properties of ODs that can make them usable, useful, and a valid mitigation strategy for potential losses of situational awareness in digital control rooms is crucial to inform the development and implementation of these systems. As such, this work systematically explores two primary topics: 1) the overall effect of ODs on human performance; and 2) the differential impacts of distinct OD designs on operator performance.
METHOD
Participants A total of 25 licensed nuclear control room operators participated in the studies, 12 in study 1, and 13 in study 2. The data collections followed exist- ing human participant protection protocols at the institution, complying with national and international laws and regulations regarding research ethics and data protection. The participants were informed about the purpose of the study as well as the overall procedures and their individual rights. A consent form was then presented before the data collection took place. While in study 1 all participants had a background on Pressurized Water Reactors (PWRs) and were thus familiar with the process aspects of the simulator, in study 2 we had a mixed sample of PWR and Boiling Water Reactor (BWR) backgrounds.
Procedure The studies were conducted at a research facility where a full scale generic pressurized water reactor (gPWR) simulator is installed in a digital control room. There are three to four seated positions in the control room: two at
Measuring Nuclear Control Room Operators’ Performance in the Context 137
the front, facing the large screen OD, where the reactor (left) and turbine (right) systems are controlled from; a third position in the back of the room for a shift supervisor; and a fourth position on the lateral for a shift technical advisor role (see Figure 1).
We used a within-subject approach for the studies, where each participant experienced all of the three interface conditions: workstation displays only, OD only, or both interfaces available. Each participant performed the task individually. All the necessary interfaces were pre-selected in the screens and thus the participants were instructed to not interact with the interface during the task. The conditions were counter-balanced for each crew (set of three to four participants that conducted the study simultaneously). The participants had a maximum of ten minutes to complete each of the trials and the task was interrupted after that time.
The participants were asked to answer as quickly and accurately as possible to a set of 30 sequential questions regarding process parameters (randomly chosen from a database of 50 questions). The questions were pre- sented in a tablet app developed for this purpose (see example in Figure 2). The questions were either yes/no type of questions (e.g. Has the Safety Injec- tion signal been reset? Yes/No) or required the typing of numerical values (e.g. What is the total Auxiliary Feed Water flow?). There was a “don’t know” button available in the bottom right of the tablet screen. The par- ticipants were instructed to use it, not only when they did not know the answer, but also when they felt like they were taking too long to answer a question. After the participant selected/inserted their answer and swiped to the following question, it was not possible to return and change previous responses.
Figure 2: Example screens from the microtask app.
Interfaces The interfaces at the research facility consist of an advanced information system, built with principles from the Information Rich Design approach (Braseth & Ørisland, 2013; details in McDonald & Braseth, 2019). The over- all size of the two OD screens was equivalent and both were projected in the same area in the front wall of the control room, visible to all operators.
The large screen OD used in Study 1 is shown in Figure 3. This OD is consistent with the design concept on the workstation displays as shown in Figure 1. The OD was conceptualized based on the identification of key plant parameters required in emergency situations, as well as allowing shared awareness of plant’s status.
Figure 3: OD used in study 1.
In study 2 we used the same workstation displays interface concept (Figure 1). However, there were significant changes both on the concept and on the implementation of the OD design. In this study, we used a prototype interface, designed as a solution for a control room that need an overview display that could be presented across a set of eight smaller screens (see Figure 4).
This design represented a tiled or fractioned representation of different plant systems across screens, all presented together in a larger projection. There were also differences on the colour coding used in this OD, as the plant wished these to correspond to their own internally used codes of red and green, which were not congruent with the implemented workstation displays at the research simulator.
Measuring Nuclear Control Room Operators’ Performance in the Context 139
Figure 4: OD used in study 2.
RESULTS & DISCUSSION Study 1
The results in Study 1 reveal a very high overall accuracy rate (M = 0.90, SD = 0.05) that was statistically equivalent in the three tested interfaces condi- tions (OD only, workstation displays (WSD) only, and when both interfaces were available).
Figure 5: Average accuracy (left) and response times (right) in study 1.
Regarding response time, the operators took an average of 8.96 seconds to answer each question (SD = 1.31). Also here, we found no differences on response times across the three interface conditions, with just a slightly quicker response time in the condition where only the OD interface was avail- able. Overall the specific interface condition did not have a significant impact in the results.
Study 2 In study 2 we had a mixed sample with operators with both PWR and BWR process expertise operating a PWR simulator. A preliminary analysis of the results showed no significant differences derived to background on response accuracy (U = 128.5, z = −1.70, p = 0.09). However, there was a difference on both the number of responded questions per trial (U = 61.0, z = −3.82, p < 0.001) and response time (U = 41.0, z = 4.16, p < 0.001), with the participants with a BWR background answering less questions (about five
less questions per trial) and taking longer to respond (on average 10 more seconds, twice the time the PWR background participants took).
The overall accuracy was also high in study 2 (M = 0.86, SD = 0.14) and the average response time was of 15.17 seconds (SD = 14.97). As is shown in Figure 6, the condition where only the OD was presented had the higher accuracy rate, followed by the workstation displays condition. Notably, the condition where both interfaces were available was the one with the low- est accuracy rate. For the response time, the condition with the workstation displays only was the one that took longer to answer, followed by the OD and then both interfaces. However, there was no statistical difference of the interface condition for accuracy, nor response time.
Figure 6: Average accuracy (left) and response times (right) in study 2.
When comparing the results in both studies we see that the response accu- racy was higher in study 1, however not statistically significant. The opposite pattern was verified for the response time, with the large screen congruent design OD having an average response time about 6 seconds quicker than the tiled OD (Figure 7).
Figure 7: Average accuracy (left) and response times (right) according to the type of OD.
Measuring Nuclear Control Room Operators’ Performance in the Context 141
These results seem to indicate a clear trend in the participants’ responses, where the design congruent OD seemed to result in less variation in the response patterns and in better performance measures both as accuracy and response time.
There is however a relevant potential confounding factor in the current study: are the differences attributable to the configuration/format of the ODs, or are they a reflection of the congruency variable that was targeted? Approx- imately the same information was presented in both ODs, however it was less detailed and more condensed for the large screen OD than the tiled OD, where each system had a dedicated tile. Both ODs had approximately the same size and were positioned in the same space in the control room. How- ever, the congruent OD was designed for the specific control room, while the tiled OD was used only in an interface assessment study and was meant to be implemented in another control room. We are of the opinion that, more than the configurational aspects (i.e., the size and shape of the screens), it is the content, concept, and integration of the design concept across dis- plays and control room systems that can have the most significant impact on the operators’ performance. We argue that, given the similarities on the display content and overall graphical design approach (with the same type of grey background and core drawings for plant components), the congruency between WSD and ODs was crucial in these studies.
Our findings highlight the potential criticality of design consistency and holistic approaches to the assessment of new technology in the control room. There seems to be potential cognitive costs associated with integrating addi- tional displays into established environments. These insights underscore the necessity of a human-centred approach to design, emphasizing the need for systems that align with the cognitive processes and limitations of operators, attending to safety aspects and plant goals. In light of these results, we recom- mend that future implementations of ODs in control rooms should prioritize design consistency. It is crucial that such displays are developed in close col- laboration with end-users, incorporating iterative feedback to ensure that they enhance, rather than degrade, performance.
The two ODs tested in this work are thought to be representative of the types of ODs that can be currently found in nuclear and other industries. Thus we argue that this study can be relevant for a better understanding of the relevant features in ODs in relation to plant safety and overall perfor- mance. Furthermore, this study acknowledges its limitations, including the simulation-based setting and the lack of familiarity with the tested interfaces, which may not capture the full complexity of an operational control room. Future research should aim to replicate these studies in operational settings where ODs are an integrated part of the work processes, to validate and extend these findings including other measures of performance in the control room such as teamwork, communication, workload and situation awareness.
CONCLUSION
This experimental study on the effects of ODs on nuclear control room oper- ator performance resulted in interesting insights and the verification of the
link between interface design and human factors. Our studies, carried out in a high-fidelity research simulator, have provided empirical evidence that illustrates the potential benefits and drawbacks of ODs in nuclear control rooms.
In Study 1, where a large screen OD, graphically consistent with the work- station interface was presented, we observed high accuracy levels with no significant difference in performance across the various interface conditions (workstation, OD, and both). This suggests that when ODs are compatibly designed and well-integrated with other systems and displays in the con- trol room, they can support effective performance without compromising response times. Conversely, in Study 2, where a tiled OD was presented, which was different from the workstation displays interface, we observed a detriment to performance. Specifically, when both the OD and workstation displays were available, we noted a decrease in accuracy, indicating a possible cognitive overload or divided attention. However, response times were still faster when both interfaces were available, which might reflect the fact that operators relied on whichever interface they felt more familiar with to find the information.
In conclusion, while ODs can potentially result in improvements for human performance, careful consideration must be given to their design and specific implementations, as the existence of an OD cannot assure, per se, a better human-system performance and might even have a negative effect on human performance. It is only through detailed human factors analysis and user-centred design processes that the full potential of these displays can be realized, contributing to improved safety, efficiency, and reliability in nuclear control room operations.
ACKNOWLEDGEMENT
This work was supported by the OECD Nuclear Energy Agency (NEA) Halden Human-Technology-Organization (HTO) Project. Thanks to the members of the Programme Review Group of the Halden HTO Project for review and constructive comments.
REFERENCES
Braseth A. O., and Øritsland T. A., (2013), ‘Visualizing complex processes on large screen displays: Design principles based on the Information Rich Design concept’, Displays, vol. 34, pp. 215–222. Braseth, A. O., & Eitrheim, M. H. R., Fernandes, A. (2019). A theoretical frame-
work for assessment of Large-Screen Displays in nuclear control rooms. Technical Report HWR-1245. OECD Halden Reactor Project, Halden, Norway. Fernandes A., Braseth A. O., McDonald R., Eitrheim M. (2020) Exploring the Effects of Large Screen Overview Displays in a Nuclear Control Room Setting. In: Harris D., Li WC. (eds) Engineering Psychology and Cognitive Ergonomics. Cognition and Design. HCII 2020. Lecture Notes in Computer Science, vol. 12187. Springer, Cham. https://doi.org/10.1007/978-3-030-49183-3_2 Fernandes, A., Braseth, A. O. (2023). Overview displays in Nuclear Control Rooms:
A state of practice review. HTOR-030. OECD Human, Technology, Organization Project (HTO). Halden, Norway.
Measuring Nuclear Control Room Operators’ Performance in the Context 143
Hildebrandt, M. (2022). Micro Task Evaluation of Digital Process Control Displays.
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 66(1), 998–1002. https://doi.org/10.1177/1071181322661429 Kortschot, S., Jamieson, G.A., & Wheeler, C.(2018). An Evaluation of the Efficacy of Group View Displays in Nuclear Control Rooms. IEEE Transactions on Human- Machine Systems, Volume: 48, Issue: 4, Aug. Laarni, J., Norros, L., Salo, L., Koskinen, H., Savioja, P., Aaltonen, I., & Liinasuo, M.
(2009). Designing large-screen overview displays for nuclear power plant control rooms. Presentation at SAFIR2010 Interim Seminar. McDonald R., Braseth A. O., (2019), ‘Report for 2.2.1 Task 3: Develop and Document an Advanced Human System Interface for the Generic Pressurized Water Reactor Simulator’, Idaho National Lab (INL), Idaho Falls, ID (United States). Salo, L., Laarni, J., & Savioja, P. (2006). Operator experiences on Working in Screen-
Based Control Rooms, in Proceedings, NPIC&HMIT, Albuquerque USA. Vicente, K. J., Roth, E. M., & Mumaw, R. J. (2001). How do operators monitor a complex, dynamic work domain? The impact of control room technology, Int. Journal. Hum. Comp. Stud., pp. 831–856.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors in Software and Systems Engineering
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-16-8
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional