Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Biofeedback Assistant to Improve Control Room Operators Reliability

Ergonomics and Information Technology Research Institute Budapest, Hungary ↗
Publication year
Digital Object Identifier10.54941/ahfe100145

Abstract

Control room operators’ reliability is a focal issue in automation not only because of the famous major industrial catastrophes like Chernobyl, Bhopal or Three Mile Island, but because of the high persisting human-related risks in e.g. transport, chemistry, energy plants. Extended research has been made on the genesis and prevention strategies of human error, as well as on the physiological and behavioural aspects of the control room operators performance. The concept presented in this paper includes the “actual attention need of the automation system” which is a prediction based on the general state and trends of the behaviour of the process controlled, and redefines the “expected / desired operator’s state” accordingly. The experimental system presented includes the data acquisition, the data processing, the intervention function and the interface to the automation system. The system monitors the operators’ physiological and behavioural data related to their readiness, the intervention function executes various feedback mechanism, meanwhile the data processing unit chooses the desired actions. This biofeedback assistant helps to improve the whole system reliability, because it can prevent the falling asleep, can detect the absence of the operator, or can initiate an unscheduled system review during under loaded time periods.

INTRODUCTION

Despite of decades of research on human reliability triggered by the enormous industrial catastrophes like Chernobyl or Three Mile Island cases, the question how to improve the reliability of complex, human-technical industrial systems is still open: new technologies in the control rooms decrease some operational risks but at the same time they lead to more complex systems and create new application areas where control rooms are used. The need to increase the productivity demands less human workforce, and the answer of automation is unmanned or single operator systems.

CHANGES IN CONTROL ROOM OPERATORS' TASK The huge roller skate control rooms of the electro-mechanical age was replaced by control terminals installed in a smaller room. The telemetric, remote effectors, video surveillance, geographical positioning system together with the high speed data processing and communications possibilities have changed the control room again. Control rooms today looks like standard computer workstations with several big, multi-purpose flat displays. Instead of jungle of knobs and bottoms or special keyboards QWERTY keyboards and mouse are used.

The controlled process has been changed too. The even more sophisticated automation shifted operators’ task down in Rasmussen’s skill-, rule-, and knowledge-based human performance level (Rasmussen, 1983) structure, degrading problem solving to regulated and to routine activity, leaving less demanding (and interesting) knowledge based task, and eliminating the majority of routine tasks, and finally leaving less visible work. At the end the concentrated operation, the merge of control rooms meant geographically extended controlled systems.

The changes of operators’ work in Parasuraman’s four classes of functions of an automation. (Parasuraman et al.,

2000):

  • Quantity and quality of raw information increased, operators’ information access improved, remote optical surveillance systems have been implemented.
  • Need of information processing and analysis increased, however the advance in computing created free human capacity.
  • With the enlargement of controlled system relatively to operators’ number the decision and action selection became more complex.
  • The interface for action implementation changed to new interface styles.

Control room operators work characterized by the dominantly low mental load at the time of normal operation which can turn suddenly a critical, highly demanding problem solving situations with the request of fast, responsible actions. According to Mumaw there’s an optimal level of stress for control room operators’ task (Mumaw, 1994), and extensive research has been made to understand operators' performance in different workload conditions (Lin et al. 2014). A recent study shows that the reliability of a complex system can be improved with workload assessment and prediction in the design phase. (Gregoriades & Sutcliffe, 2008) Adaptive automations

In human-automation interaction several static models have been suggested, e.g. Sheridan and Verplank distinguishes 10 automation levels, ranging from full manual („human does the whole job”) to full automatic control („computer does whole job if it decides it should be done, and if so tells human, if it decides he should be told”). (Sheridan and Verplank, 1978) The changes around and inside control rooms demand function allocation with macroergonomics approach which ensures the effective system design (Challenger et al., 2013) and with human factors addressed (Feigh et al., 2014). To achieve better performance and meet requirements derived from human diversity and fluctuation of abilities the concept of adaptive automation was invented. According to Byrne “adaptive automation is an approach to automation design where tasks are dynamically allocated between the human operator and computer systems. […] Adaptive automation represents a unique domain for the application of psychophysiology in the work environment.” (Byrne et al., 1996) Sauer identified four dynamic function “(re)allocation decisions: (1) the occurrence of critical events (such as a fault condition), (2) performance degradations below a stipulated level, (3) real-time assessment of psychophysiological status (excessive workload or fatigue), or (4) the identification of symptomatic patterns of system malfunctioning, as embodied in behavioral models”. (Sauer et al., 2013) To make these allocation decisions a continuous monitoring of the process and the operators is necessary.

In the new function allocation humans was heavily replaced by technical elements, the number of co-workers on the field or in the team decreased, and often applications or agents took over humans’ role. To describe the quality of such changes metrics of function allocation was invented, e.g. Pritchett identified 8 specific metrics from workload to mission performance. (Pritchett et al., 2014) Biofeedback applications

In the last years as a new discipline neuroergonomics emerged. According to Mehta neuroergonomics is “defined as the study of the human brain in relation to performance at work and in everyday settings” and can be applied to “(1) physical work parameters; (2) physical fatigue; (3) vigilance and mental fatigue; (4) training and neuroadaptive systems; and (5) assessment of concurrent physical and cognitive work.” (Mehta et al., 2013) There are several wearable biofeedback applications. Winkley presented a wearable health monitor device that gave continuous feedback to the user for domestic health care (Winkley et al., 2012), Dorneich’s wearable adaptive system classifies users' cognitive state in a military setting (Dorneich et al., 2012). Prinzel proved that an EEG based feedback has impact on workload. (Prinzel et al., 2000)

We hope one hand that we can implement more and more complex, practical feedback functions, and on the other hand we can collect data to overcome the adaptive systems difficulties described by Feigh “the majority of adaptive systems have been experimental rather than practical because of the technical challenges in accurately perceiving and interpreting users' current cognitive state; integrating cognitive state, environment, and task information; and using it to predict users' current needs”. (Feigh et al., 2012)

One example is the Reconfigurable Main Control Room Simulator which is capable of recording heart rate, respiration and galvanic skin response data in conjunction with recorded video participant behavior.” (Elks et al., 2012)

THE EXPERIMENTAL SYSTEM The ProErgoBio Biofeedback assistant (Fig. 1) consists of

  • Interface to the automation system,
  • ProErgoBio Sensors,
  • Interface to actuators, environmental control, Human-Machine Interface.
  • ProErgoBio Box as a central unit,
  • Internet connection.
Figure 1. Structure of the ProErgoBio Biofeedback assistant Functions of the PROERGOBIO BOX
Original PDF Figure 1.

Farkas et al. analyzed control operators’ task and found that in reality it is much more rich and complex than the classic control room operators’ work as it was assumed in previous studies. We agree with Dzindolet that “the human operators and automated aids must be flexible, capable of responding to rare or unanticipated events”. (Dzindolet et al., 2006), and we agree with Niederée that “designing highly automated systems especially in safety-critical domains” needs more investigation (Niederée et al., 2012), but we believe that the management of the whole shift activity can be an other way to improve the overall performance. Operators’ time spent in the control room mainly dedicated to auxiliary tasks e.g. scheduled maintenance, supervision of field-operations, administrative work, training sessions, etc or it’s simply eventless. Indisputably the goal of automation design is to optimize the overall performance and establish the safest operation. To achieve this goal the support of operators’ work must be extended from the emergency related problem solving procedures to the whole shift including all operator’s activity e.g. reading, eating, refreshing or making exercises.

A full-shift adaptive system must reconcile short-term and long-term goals. Short term goals have priority when an event occurs and the system must behave according to the broadly used classic control room operators’ work activity approaches. However “between events”, when the system otherwise is stable there’s room to consider long term goals and carry out auxiliary tasks or gain resources, improve operators capacity.

The adaptability proposed here is based on system state, operator state and environmental conditions and can be achieved by the biofeedback assistant. The central unit monitors which sensor or actuator is available and acts accordingly. The central unit is responsible for collecting data of on-line sensors, triggering actuators, communicating with the automation system and to manage remote access.

Interface to the automation system The ProErgoBio Box’s interface will allow the Box to connect to the most common automation system manufacturer’s products. The box will receive information regarding the occurrence of critical events, the overall performance (system stability). The box will supply information to the automation to initiate some limited adaptations, e.g. start a scheduled check with operators contribution.

Data acquisition To describe human operators readiness or performance several monitoring system have been implemented. For example Borghini presented a neurophysiological instrument to detect drowsiness based on increased blink rate and decreased HR values. (Borghini et al., 2012) Durantin found that “Functional near infrared spectroscopy (fNIRS) and Heart Rate Variability (HRV) are sensitive to different levels of mental workload […] and are suitable for mental overload detection.” (Durantin et al., 2014) According to Taylor Electroencephalogram (EEG) is more suitable for adaptive automation than electrocardiogram (ECG) or eye tracking. (Taylor et al., 2010)

In a comparison study Mehta summarized that “While some neuroimaging methods are expensive and are immobile, such as the MRI, functional magnetic resonance imaging (fMRI), positron emission tomography (PET), and diffusion tensor imaging (DTI), portable methods such as electroencephalography (EEG), functional near infrared spectroscopy (fNIRS), and transcranial Doppler sonography (TCDS), are more likely to be adopted in applied ergonomics research.” (Mehta et al., 2013) Since our system will be used in real working environment it must usable 7/24 hours, which means visual solutions are preferred, and invasive interventions are out of scope. We create a set of sensors, and it’s up to the central unit to recognize which is available. The proposed set of sensors:

  • Operators’ presence and location e.g. in place, out, congregate.
  • Operators’ posture and movements e.g. standing, sitting.
  • Operators’ physiological conditions e.g. heart rate, temperature.
  • Operators’ behaviour e.g. normal, crying, paralysed.
  • Stress level.
  • Sleeping or loss of consciousness.
  • Galvanic skin response.
  • Eye movement.

After the literature research to estimate the psychological condition of the observed operator is a high priority. Based on our research and the cited articles, we choose to measure the operator’s heart rate and derived HRV parameters and the operator’s galvanic skin response (GSR). Furthermore 3D cameras will be used for measure the eye movement and body posture of the operator.

There are several other indicators for psychological state, like Electroencephalogram (EEG), but it would be uncomfortable to use it in 8 hour work time, every day. Blood pressure measurement is also excluded, because it would be too disturbing during the work.

Actuators, actions or intervention The experimental system action repertoire is limited by the set of available actuators and by the set of rules the central unit can handle. We begin with simple, short loop feedbacks, and expect the system to learn how operators in general or how operators as individuals behave and perform under various working and personal conditions. The proposed set of actuators and interventions:

  • Adjusting environmental conditions e.g. lighting or shading of natural lighting, temperature, humidity.
  • Adjusting intensity of system alarms to the noise level of the control room.
  • Adjusting display parameters e.g. orientation, brightness.
  • Closing / locking / opening doors or windows.
  • Vibrating chairs.
  • Sending SMS messages.
  • Acoustic system messages on supportive and comforting (female) voice or on commanding male voice.
  • Phrasing messages as suggestions or command.
  • Encouraging or cautionary messages.
  • Giving feedback on operator performance e.g. congratulate.
  • Initiate / suspend extended tasks e.g. maintenance, training, scheduled tasks.
  • Initiate operators activity e.g. take a nap, exercise, drink some water, eat, play a game, wash face.

CONCLUSIONS

In this paper a new automation feature, a full shift adaptive system, the biofeedback assistant was introduced. This system collects human physiological, behavioral and performance data as well as the control room environmental parameters, process and automation indicators.

The main components of the ProErgoBio Biofeedback assistant are data acquisition, the data processing, the intervention function and the interface to the automation system. The system monitors the operators’ physiological and behavioural data related to their readiness, the intervention function executes various feedback mechanism, meanwhile the data processing unit chooses the desired actions. This biofeedback assistant helps to improve the whole system reliability, because it can prevent the falling asleep, can detect the absence of the operator, or can initiate an unscheduled system review during under loaded time periods.

With the ProErgoBio Biofeedback assistant a new research possibility is created to test different adaptive and time management methods to improve operators’ whole shift performance.

ACKNOWLEDGEMENT

This research was supported by the New Hungary Development Plan, Economic Development Operational Programme, project ID P1325 “Improved Control Room Operators’ Reliability with ProErgoBio Biofeedback assistant”

REFERENCES

266. Sauer, J., Nickel, P., Wastell, D. (2013), “Designing automation for complex work environments under different levels of stress”, Applied Ergonomics, Volume 44, Issue 1, January, pp. 119-127 Sauer, J., Nickel, P., Wastell, D. (2013). “Designing automation for complex work environments under different

levels of stress”, Applied Ergonomics, Volume 44, Issue 1, January, pp. 119-127, ISSN 0003-6870 Sheridan, T. B., Verplank, W. L. (1978). “Human and computer control of undersea teleoperators” MASSACHUSETTS INST OF TECH CAMBRIDGE MAN-MACHINE SYSTEMS LAB. Taylor, G., Reinerman-Jones, L., Cosenzo, K., & Nicholson, D. (2010). “Comparison of multiple physiological

Cover of the proceedings volume Advances in Safety Management and Human Factors

Explore the complete digital volume

Advances in Safety Management and Human Factors

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-4951-2100-5

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →