Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Using Computer Vision to Reduce Human Errors of Operating on the Wrong Control Valves in Nuclear Power Plants

1 Carnegie Mellon University, Pittsburgh, Pennsylvania, USA ↗
2 The Ohio State University, Columbus, Ohio, USA ↗
3 Arizona State University, Tempe, Arizona, USA ↗
Publication year
Digital Object Identifier10.54941/ahfe1002217

Abstract

Nuclear power plants are complex systems that have many modules for errors to occur. Each year in the United States, an average of approximately 80 accidents happen, of which 50 (or 62%) are related to human errors (Nuclear Energy Agency, 2020). These errors reduce the efficiency of plants and have cost, safety, and environmental consequences. Nuclear operators manipulate control objects, like valves, to complete maintenance procedures involved in power generation by directing water around a plant (U.S. Bureau of Labor Statistics, 2021). However, there are many identical valves in a small area which can lead to errors handling the wrong valve. In practice, valves are identified using lockout tags that can contain as little as its normal position, or large blocks of information on its use, position, or contents. (Occupational Health and Safety Administration, 2011). This method of differentiating valves is insufficient because tags may be damaged and hard to read, or contain inadequate information to be useful in an emergency. Current efforts to reduce valve operation errors use AI tools to diagnose faults, like valve damage, and suggest best practice procedures. However, no existing solution identifies valves to prevent operational errors due to misidentification. Computer vision techniques that use machine learning, like object detection algorithms, can provide a solution to this problem. This paper explores the integration of computer vision and real-time sensors monitoring water systems to reduce errors operating on the wrong valves. This approach analyzes sensor data and uses object detection algorithms to identify control valves as workers are walking through a power plant to minimize the possibility of mixing them up. The sensor log analysis algorithm identifies critical valves that require action, then the computer vision algorithm, the YOLO version 3 object detection algorithm, highlights them in real-time. While developing the sensor data algorithm, we created a simulation that models valves controlling water flow between tanks. Within a nuclear power plant the simulation represents the long cycle cleanup operation where boiled feedwater cannot contaminate cooled feedwater. It captures random noise and is a virtual environment for testing operators on how they react to system errors like tank overflows or sudden influxes of water. For example, one valve oscillates around its input value to represent the uncertainty from imperfect or leaky valves. Other noise from large influxes of water is created by an oscillation of the inflow with a pulse function. Testing the developed algorithms on data from a mechanical room in Posner Hall at Carnegie Mellon University indicates their potential for reducing real-time operation errors. Specifically, testing results confirmed that control object operation is an issue and that computer vision provides a promising solution to this problem. Although this project is limited to merely recognizing control valves due to the method of object detection, it can have significance in reducing emission leakage, improving efficiency and reliability, and advancing technology for safety. Such limitations necessitate future work using other object detection algorithms to compare results as well as integrating spatial data to differentiate between identical valves.ReferencesNuclear Energy Agency, and International Atomic Energy Agency. “Nuclear Power Plant Operating Experience from the IAEA/NEA International Reporting System for Operating Experience 2015-2017.” Nuclear Power Plant Operating Experience, 2020, doi:10.1787/2bdd0383-en.“1910.147 - The control of hazardous energy (lockout/tagout).” (2011). Occupational Safety and Health Administration, U.S. Department of Energy, <https://www.osha.gov/laws-regs/regulations/standardnumber/1910/1910.147> (Nov. 3, 2021). “51-8011 Nuclear Power Reactor Operators.” U.S. Bureau of Labor Statistics, U.S. Bureau of Labor Statistics, 31 Mar. 2021, www.bls.gov/oes/current/oes518011.htm.

INTRODUCTION

NPPs are complex systems that have many modules for errors to occur. Each year in the United States, an average of approximately 80 accidents happen, of which 50 (or 62%) are due to human errors (Nuclear Energy Agency, 2020). These errors reduce the efficiency of plants and can cause emissi- ons leakage. Nuclear operators manipulate control objects, like valves, to complete maintenance procedures involved in power generation by directing water around a plant (U.S. Bureau of Labor Statistics, 2021). However, there are many identical valves in a small area leading to errors in handling the wrong valve. In practice, valves are identified using information tags contai- ning as little as their normal positions or large blocks of information on their use, position, or contents. (Occupational Health and Safety Administration, 2011). This method of differentiating valves is insufficient because tags may be damaged and hard to read or contain inadequate information to be use- ful in an emergency. Current efforts to reduce valve operation errors involve

© 2022. Published by AHFE Open Access. All rights reserved. 23

self-verification and peer checking, sending a second worker to verify valve positions after the first worker completes the changes. However, this is high cost and still error-prone because the second worker could misidentify a valve or its position (Lochbaum et. al, 2017). On top of this, the committed changes remain in place while waiting for the verification.

Computer vision techniques are promising at recognizing nearby valves, effectively reducing operational errors. For example, one study demonstra- ted the potential of computer vision by using an object detection algorithm to classify workers in an NPP by recognizing the safety uniforms they were wearing (Sun et. al, 2020). Automatic object recognition in the field helped reduce plant shutdowns by analyzing human behavior to improve workflow efficiency. Another study used the YOLO version 3 object detection algori- thm in an electrical plant to detect loss of function behaviors by recognizing objects like phones or cigarettes that cause workers to be distracted (Cao et. al, 2021). Minimizing these behaviors will improve the efficiency of the plant and reduce the possibility of errors. Although these studies illustrate the potential to use computer vision to prevent misidentification errors, a com- puter vision model for reliably detecting and distinguishing different control valves in NPPs remains an area that needs algorithm development and testing. Another challenge is the limited studies on incorporating real-time sensor rea- dings in an NPP to identify critical valves given their live states measured by the sensors.

This paper explores integrating computer vision and real-time sensors monitoring water systems to reduce errors operating on the wrong valves. The following sections are: 1). a motivation case to demonstrate the neces- sity of an automated tool to prevent NPP workers’ valve operating errors; 2). a literature review to establish our work within the field; 3). the meth- odology establishing the object detection algorithm used for NPP operation monitoring; 4). a case study to analyze its accuracy; and 5). a concluding section to describe the significance of our work. This approach analyzes sen- sor data and uses object detection algorithms to identify control valves as workers walk through a power plant to minimize the possibility of mixing valves up. The sensor log analysis algorithm identifies critical valves that require action, then the computer vision algorithm, the YOLO version 3 object detection algorithm, highlights them in real-time. While developing the sensor data algorithm, we created a simulation that models valves con- trolling water flow between tanks. Within an NPP, the simulation represents the long cycle cleanup operation where boiled feedwater cannot contaminate cooled feedwater. It captures random noise and is a virtual environment for testing operators on how they react to system errors like tank overflows or sudden influxes of water. For example, one valve oscillates around its input value to represent the uncertainty from imperfect or leaky valves. Some noi- ses from large influxes of water are created by oscillating the inflow with a pulse function. Testing the developed algorithms on data from a mech- anical room in Posner Hall at Carnegie Mellon University indicates their potential for reducing real-time operation errors. Specifically, testing results confirmed that control object operation is an issue, and that computer vision provides a promising solution to this problem. Although this project focuses

Using Computer Vision to Reduce Human Errors of Operating 25

Figure 1: Baseline case of the dynamic flow simulation.
Original PDF Figure 1:

on recognizing control valves using object detection algorithms, integrating object detection methods with NPP simulation and fault detection methods can advance technology for nuclear safety. Such limitations necessitate future work using other object detection algorithms to compare results and integrate spatial data to differentiate between identical valves.

DEVELOPING A SIMULATION TO MOTIVATE THE STUDY OF HUMAN ERRORS There are a variety of possible human operational errors in an NPP that each have different effects. To demonstrate how dangerous mistakes can be and study operators’ roles in causing or preventing them, we created a model that simulates the long cycle cleanup process in NPP operations. The long cycle cleanup operation within the condensate system regulates the circulation of fresh feedwater and used feedwater (or condensate). At the same time, the reactor is shut down, ensuring that the two do not mix (Kozal, 2017). The model was built using Simulink, a MATLAB extension, which allows it to be dynamic and log sensor data while the simulation is running. The model consists of four tanks and five valves (1 automatic con- trol valve and four manual control valves). It simulates how water flows between four tanks. As shown in Figure 1, water enters through Tank 1, is decontaminated as it cycles through Tanks 2 and 3 as needed and exits the system as purified water in Tank 4. An overflow in Tanks 1-3 could contaminate the water in Tank 4. Each tank has a sensor that outputs the water level inside it with a warning light for when it exceeds a set overflow value. Each valve also has a warning light activated by the valve position exiting a 30-70 range to create multiple objectives within the simulation. As shown in Figure 2, the model alerts operators when they have made an error.

As shown in Table 1, valves can be automatically or manually controlled, or users can switch between the modes in real-time. Automatic valves respond to the inputted value of the previous tank with a sine wave that simulates random environmental noise. The function oscillates with an amplitude of 10. Additionally, the inflow rate automatically follows a sine function added to a pulse function to represent large influxes of water that might occur during

Figure 2: Error case of the dynamic flow simulation.
Original PDF Figure 2:
Table 1. Valve types in the simulation.

Table available in the original PDF paper.

Valve Number Control Type

1 Manual 2 Manual 3 Automatic 4 Switch 5 Manual

Figure 3: Simulated sensor log of the water level in each tank during an iteration of the simulation.
Original PDF Figure 3:

weather events. Figure 3 demonstrates the recorded sensor data of the water levels in each tank. This log depicts how the system reacts to changes made by operators while capturing environmental noise. This data can be used to assess how successful operators are at manipulating the system to prevent

Using Computer Vision to Reduce Human Errors of Operating 27

or respond to errors. A similar log can be generated based on the sensor warnings to provide an additional evaluation method. Overall, the simulation motivates this research by demonstrating the consequences of operational errors in a model of a closed loop system in an NPP.

LITERATURE REVIEW

Many NPPs have begun searching for ways to use technology to respond to errors made by plant operators operating on the wrong valve to increase safety and efficiency. The current practice uses information tags to identify valves, but there is no standardization around what information must be on the tags. Therefore, some tags have too much information, making it hard to identify the valve, and some have too little which means they are not helpful in emergencies (Occupational Health and Safety Administration, 2011). One solution is using neural networks that take in signal data from the plant to diagnose and correct human valve errors after a fault has occurred, reducing the number of factors operators need to consider. An example of this is a study that used a neural network that put sensor alarm and gauge data through a Markov chain in a matrix to determine the most critical alarm, diagnose the fault, and propose the best anomaly handling procedure, but left the final decision up to the operator (Hsieh et al, 2012). A similar study followed the same basic structure. However, it categorized errors, such as loss of coolant or steam line rupture, into different neural networks that could memorize similar patterns for more accurate results (Mo et al, 2007). Both solutions are retroactive for emergencies; they respond to valve errors rather than predict and prevent them.

However, many faults occur during repetitive tasks like operating valves which is prone to errors operating on the wrong valve because it is mun- dane and boring for operators. Taking human error out of valve operating would be a more effective solution because it prevents faults in the first place. Recently, surveillance data has been combined with fault diagnosis systems in a preliminary attempt at plant monitoring (Wang et al, 2016). Another study used passive resistive sensors to monitor valve position automatically and display it to a central panel (Agarwal et al, 2016). A different solution used in the oil and gas industry is the VIKINGS robot that detects a valve by matching its image to SIFT descriptors and uses a color-based method and a neural network to determine the position of the valve (Merriuax et. al, 2019). This technique is designed for butterfly valves used in oil and gas plants. However, the research in this paper can support similar robots to identify gate valves used in NPPs to increase safety and efficiency around the plant.

Computer vision techniques have great potential to eliminate human errors in power plants through real-time detection of safety-critical objects for field workers. For example, one study used a human joint detection algorithm to track workers as they completed tasks to analyze human behavior to improve the efficiency of the plant by avoiding shutdowns (Sun et. al, 2020). Another study used an object tracking algorithm to analyze the behavior of human operators around a crane to increase worker safety during power

plant outages (Zhang et. al, 2016). However, computer vision has rarely been used to help workers recognize control valves. The challenge in applying computer vision for valve detection lies in rapidly and reliably identifying the difference between many identical valves (Redmond et. al, 2018). The discus- sion above leads to the formal research question of this paper - How can we diminish emissions leakage by reducing errors operating on control valves incorrectly in NPPs?

BUILDING AN OBJECT DETECTION ALGORITHM The proposed solution to the research question is to use computer vision by training a custom object detection algorithm. To do this, we needed to gather data in the form of images, label our dataset, train the algorithm, and test its accuracy. We used the website makesense.ai to label our ima- ges by bounding what were predetermined to be critical control objects and giving each a class name. Then we built a YOLO version 3 algorithm in Google Colab because of its cloud connection and free GPU (Google Colab, 2018). YOLO algorithms are beneficial because they run significantly faster than other detection methods while obtaining similar results in terms of the mean average precision and intersection over union values (Redmond et. al, 2018). They are also pre-trained using open-source data to support many classes, so they are useful in a wide variety of situations, and their functionality can be tested beforehand. YOLO version 3 is a Region-Based Convolutional Neural Network-based algorithm that uses DarkNet53 to extract features. The training process of this algorithm is to show a sam- ple of labeled images to initialize its weights. Then the network tries to predict the bounding boxes of the manually labeled dataset through a sele- ctive search algorithm and calculates the intersection over union ratio of the ground truth object box to the predicted one to update its weights using a regression model. Version 3 is smaller than more recent versions while still training to a comparable accuracy. After the training, we focused on the algorithm’s percentage loss and confidence in predictions as markers for its accuracy.

In general, object detection is a promising solution because algorithms use existing data to improve their accuracy with each iteration. They can also be adapted to different formats as required for various industrial plants and different interfaces such as computers, phones, or even wearable glasses. Since their structure is basic, object detection algorithms are explainable and can be expanded upon as technology improves. Finally, although the bias from the programmer is included, object detection algorithms minimize bias from the user.

CASE STUDY We applied this methodology to a mechanical room in Posner Hall at Car- negie Mellon University to test the algorithm. The dataset was constructed using data obtained by videotaping a walkthrough of the room and extra- cting every tenth frame for 868 images. Then, the control valves, sensors,

Using Computer Vision to Reduce Human Errors of Operating 29

Figure 4: Percentage loss of the YOLO object detection algorithm after each iteration.
Original PDF Figure 4:
Figure 5: Objects labeled by the algorithm with its degree of confidence in the predictions.
Original PDF Figure 5:

signal lights, electrical carbines, and chemical tanks in each frame were manu- ally bound and labeled before being exported as text files. After converting the annotations to the YOLO format, we configured them using batches of 64 and subdivisions of 16. We set the maximum number of batches to 10,000 for five classes with 30 filters. Once the algorithm was trained at an average learning rate of 0.001, we tested the accuracy by plotting the percentage loss after each iteration as shown in Figure 4. The training resul- ted in a percent loss of about 0.3% after 2,700 iterations, well below the standard of less than 2% loss, indicating an accurate algorithm. The results of this case study are an accurate, fully functional object detection algori- thm trained to support the classes identified as control objects in an NPP, as demonstrated in Figure 5. This algorithm can bound and label each class in real-time as it is fed in live video and output the degree of confidence in each prediction.

CONCLUSION

During this project, we identified that control object operation is an issue, and that computer vision provides a promising solution. As part of this, we trained a custom object detection model to recognize parts in NPPs and desi- gned a simulation that collects environmental noise to test how operators respond to errors. Our solution has environmental benefits by reducing the number of accidents which minimizes radioactive leakage. It also leads to more efficient operating by limiting how often a plant must be shut down, generating more energy, and increasing profits. This makes NPPs a more viable alternative to fossil fuels by increasing their reliability and reducing emissions (Hitchin, 2016). Our algorithm can create a safer workspace for NPP operators and support technological advancements made by future pro- jects. It can be used in a variety of applications. For example, it can be programmed into robots that are used for disaster relief in situations that are too dangerous for humans.

However, this solution is limited because object detection algorithms can only identify parts, not recognize their location within a plant or which are important to the work order. Further, it cannot recognize partially expo- sed objects. Also, our algorithm was only trained with one dataset, and we did not explore other object detection algorithms, which adds potential for error. Similarly, we iterated through the algorithm 2,700 times instead of the recommended 10,000 due to time constraints, but the percentage loss was under 2%, so the algorithm is still highly accurate. More broadly, compu- ter vision cannot actively prevent errors. It is only successful if operators use it. It is also not intuitive or compassionate and can only operate how it was programmed, making decisions based on inputting specific data. Com- puter vision techniques also amass data, which causes privacy concerns over how data is collected, stored, and used. Although, the emergence of fede- rated learning algorithms which can be transferred between users without sharing the data used for training creates a potential solution to this (Cheng et al, 2020). Finally, it necessitates future work comparing the accuracy of different object detection algorithms and fitting our algorithm to different interfaces and applications.

ACKNOWLEDGMENT

This material is based on work supported by the U.S. National Science Foun- dation (NSF) CAREER under Grant No. 2034373 (Research Experiences for Undergraduates), and the Nuclear Engineering University Program (NEUP) of the U.S. Department of Energy (DOE) under Award No. DE-NE0008864. The supports are gratefully acknowledged.

REFERENCES

“1910.147 - The control of hazardous energy (lockout/tagout).” (2011). Occupational Safety and Health Administration, U.S. Department of Energy, <https: //www.osha.gov/laws-regs/regulations/standardnumber/1910/1910.147> (Nov. 3, 2021).

Using Computer Vision to Reduce Human Errors of Operating 31

“51-8011 Nuclear Power Reactor Operators.” U.S. Bureau of Labor Statistics, U.S. Bureau of Labor Statistics, 31 Mar. 2021, www.bls.gov/oes/current/oe s518011.htm. Agarwal, Vivek, et al. “Wireless Online Position Monitoring of Manual Valve Types

for Plant Configuration Management in Nuclear Power Plants.” IEEE Sensors Journal, IEEE, 5 Oct. 2016, www.osti.gov/pages/servlets/purl/1363919. Cao, Yuan, et al. “Computer-Vision-Based Abnormal Human Behavior Detection and Analysis in Electric Power Plant.” IEEE Xplore, IEEE, 30 Nov. 2021, https: //ieeexplore.ieee.org/abstract/document/9601435/references#references. Cheng, Yong, et al. “Federated Learning for Privacy-Preserving Ai.” Association for

Computing Machinery, Viewpoints, 1 Dec. 2020, https://dl.acm.org/doi/pdf/10. 1145/3387107. Hitchin, Penny. “Valves and Actuators: Changing Demands on Power Plants.” Power Engineering International, 21 Jan. 2016, https://www.powerengineeringint.com/ coal-fired/changing-demands-on-power-plants/. Hsieh, Min-Han, et al. “A Decision Support System for Identifying Abnormal

Operating Procedures in a Nuclear Power Plant.” Nuclear Engineering and Design, North-Holland, 15 May 2012, www.sciencedirect.com/science/article/pi i/S0029549312002130?casa_token=n9IPQseC044AAAAA%3AtHIbk-Dpnc-d vwjg3Q0igGNCqEDvS7T5VpPaRitaB3sPEvNYX86FdV4eXO35nbmycFIKkya TPw. Kozal, Jason. Arlington, TX, 2017, pp. 19–20, Grand Gulf Nuclear Station - NRC Special Inspection Report. Lochbaum, Dave, and Ed Lyman. “Cooper: Nuclear Plant Operated 89 Days with

Key Safety System Impaired.” All Things Nuclear, Union of Concerned Scientists, 14 July 2017, https://allthingsnuclear.org/dlochbaum/cooper-near-miss-2017/. Merriaux, Pierre, et al. “The Vikings Autonomous Inspection Robot: Competing in the ARGOS Challenge.” ResearchGate, Mar. 2019, https://www.researchgate.net/publication/327944928_The_VIKINGS_Auto nomous_Inspection_Robot_Competing_in_the_ARGOS_Challenge. Mo, Kun, et al. “A Dynamic Neural Network Aggregation Model for Transient Diagnosis in Nuclear Power Plants.” Progress in Nuclear Energy, Pergamon, 30 Mar. 2007, www.sciencedirect.com/science/article/pii/S0149197007000030?ca sa_token=hMjQNGwyyJsAAAAA%3Ajey5kFMAf4d05HS1Knx9dYnGmsUmj mTetR2AGcZ8YoUsT2poGwreOTAwBua__Gk-K1yecuJIFQ. Nuclear Energy Agency, and International Atomic Energy Agency. “Nuclear Power

Plant Operating Experience from the IAEA/NEA International Reporting System for Operating Experience 2015–2017.” Nuclear Power Plant Operating Experie- nce, 2020, doi:10.1787/2bdd0383-en. Redmon, Joseph, and Ali Farhadi. “Yolov3: An Incremental Improvement.” ArXiv.org, Cornell University, 8 Apr. 2018, https://arxiv.org/abs/1804.02767. Sun, Zhe, et al. “Predictive Nuclear Power Plant Outage Control through Computer Vision and Data-Driven Simulation.” Progress in Nuclear Energy, Pergamon, 27 July 2020, https://www.sciencedirect.com/science/article/pii/ S0149197020302006. Wang, Wenlin, and Ming Yang. “Implementation of an Integrated Real-Time Process Surveillance and Diagnostic System for Nuclear Power Plants.” Annals of Nuclear Energy, Pergamon, 28 June 2016, www.sciencedirect.com/science/articl e/pii/S0306454916303681?casa_token=heYBQzSJMQQAAAAA%3AfhimkE1c 0x2eBdxHqYUJRO8oum_k8b0FLrQ6yETEmqWXiMa4V2h15p8_4P3rlWXx pM8Fpmr2sQ.

“Welcome to Colaboratory.” Google Colab, Google, 2018, https://colab.research.g oogle.com/?utm_source=scs-index. Zhang, Cheng, et al. “Automatic Crane-Related Workflow Control for Nuclear

Plant Outages through Computer Vision and Simulation.” ISARC Proceedings, IAARC, 21 July 2016, https://www.iaarc.org/publications/2016_proceedings_of _the_33rd_isarc_auburn_usa/automatic_crane_related_workflow_control_for_ nuclear_plant_outages_through_computer_vision_and_simulation.html.

Cover of the proceedings volume Human Factors in Energy: Oil, Gas, Nuclear and Electric Power

Explore the complete digital volume

Human Factors in Energy: Oil, Gas, Nuclear and Electric Power

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-30-8

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →