Tamsyn Edwards, Sarah Sharples, Barry Kirwan, John Wilson, Nora Balfe (2021). Identifying Markers of Performance Decline in Air Traffic Controllers. In: Advances in Human Aspects of Transportation: Part II. AHFE Open Access, vol. 16. https://doi.org/10.54941/ahfe100715
Air traffic control (ATC) is a safety critical environment in which air traffic controllers (ATCOs) must maintain a consistently high standard of human performance in order to maintain safety. It is therefore essential to understand the impact, of human factors such as workload on human performance. Traditionally, a wide range of methods have been utilized to investigate the association between performance-influencing factors and human performance, from experimental to research in the wild (Sharples et al., 2012). However, little research has focused on the identification of ‘signs and symptoms’, or indicators, that a factor may be negatively influencing human performance. The aim of this study was to identify indicators that were commonly associated with specific factors (workload, stress and situation awareness) and the associated influences on human performance. Findings identified internal and external, observable indicators that were common across all controllers and signified when ATCO performance was threatened. Findings also suggest that controllers use internal markers to adapt control strategies in order to maintain performance. Findings have relevance for existing aviation training, online measurement of performance during live operations, and measurements during simulations. Further research should investigate the application of indicators in measurement of live operations.
Air traffic control (ATC) is a safety critical environment (Chang & Yeh, 2010). Air Traffic Controllers (ATCOs) are responsible for the safety and efficiency of all air traffic. Unlike other safety critical industries, there are no physical barriers or defences that protect aircraft in flight. It is therefore essential that controllers maintain a consistently high standard of human performance in order to maintain flight safety.
With a large potential for incidents, Kirwan (2011) suggests that Air Traffic Management (ATM) is remarkably reliable. However, when aircraft have breached standard regulated minima, termed an ‘incident’, human error has been attributed as a primary or secondary cause in 75-90% of cases (Mackieh & Cilingir, 1998). Human factors (such as workload, fatigue, inadequate communications) are “major determiners of a human error” (Park & Jung, 1996, p330) and have been repeatedly shown to negatively affect human performance (Chang & Yeh, 2010; Park & Jung, 1996).
Traditionally within the discipline of human factors, the investigation of human performance, and the association with influencing factors such as workload and stress, has focused on a wide range of methods, from experimental to research in the wild (Sharples et al., 2012). However, little research has focused on the identification and application of ‘signs and symptoms’, or indicators, of the negative association between human factors and performance, or indicators that may provide a signal that a controller is reaching their personal ‘edge’ of performance.
An investigation of the indicators of potential performance decline in ATCOs may enhance knowledge of the use of indicators within an operational environment, and examine if indicators could be utilized to support performance prior to a performance decline, potentially contributing to preventing performance – related incidents. In addition, within the air traffic domain, identifying indicators of potentially declining ATCO performance may have implications both for research and live operations. A challenge of conducting research in the safety critical domain of air traffic control is the restriction of measurements during live operations. Consequently, air traffic control research is predominantly conducted within simulated, or other controlled, environments. Knowledge of consistent indicators would provide a basis for observation metrics, which may be employed in operations rooms, supporting research in live operations. In addition, knowledge of consistent indicators may support training for operations personnel. Results may provide a standardized foundation of knowledge for identifying when controllers maybe reaching the ‘edge of performance’. Supervisors and unit competency examiners may be provided training to ensure a standardized foundation of knowledge. Performance could then be supported, potentially preventing performance decline and subsequently, performance-related incidents.
METHOD
Design A total of 22, one hour, face to face semi-structured interviews were conducted with en-route ATCOs. Participants were selected to represent several sub-groups of controllers, based on age, sex and experience, role and nationality. Interviews took place in the Maastricht Upper Area control centre (MUAC). An interview schedule was developed to guide the semi-structured interview; participants were asked pre-designed lead questions which were then followed by probes. A protocol was used to standardize the interview method procedure. Participants were asked to list indicators of potential performance decline that were perceived to be associated with a specific factor (e.g. “What indicators of potential performance decline are associated with the influence of high workload on performance?”). The factors included in the interview study were: workload, stress and SA. Interviews were tape recorded and were then orthographically transcribed. Thematic analysis was applied as the analysis strategy.
Participants In total 22 en-route controllers were interviewed. The majority of participants were male (17, or 77.27%), compared to female participants (5, or 22.73%). All participants worked as en-route controllers in the Maastricht Upper Airspace control (MUAC) centre. Participants’ ages ranged from 21-60. Years of experience as an ATCO (excluding training) ranged from 1.5-31 (M=14.55, SD=8.68).
RESULTS AND DISCUSSION
A total of 22 one-hour interviews were orthographically transcribed and analysed using thematic analysis. Recurrent and relevant themes are presented below with illustrative quotes from participants.
Use of indicators of potential performance decline in air traffic control All participants (22/22) were familiar with the concept of indicators and agreed that they occurred in the ATC operations room. All controllers stated indicators are commonly used in ATC to provide information regarding when a controller may not be controlling optimally, or a factor (such as fatigue, stress) is negatively influencing performance. All controllers monitor their own personal indicators as well as colleagues’ indicators: “…We work close together, we monitor each other you would be very aware of the person sitting beside you whether they’re on the ball or whether they’re slightly less, whether they’re tired, whether they’re distracted by whatever or something, it’s part of the job and you do, you make allowances” (Participant 2).
Controllers used indicators for the primary purpose of gaining information, which subsequently led to supporting performance. After identification of a marker, controllers frequently reported applying a supportive mechanism to maintain performance, such as a change in control strategy to mitigate the issue that was negatively influencing the controller: “…it's that point [of recognising something is wrong] where you have to, well in my opinion you have to change the way that you're controlling the traffic” (Participant 10). This process occurs both for controllers to support their own performance, and controllers who apply adaptive strategies to support their colleagues’ performance. The application of the compensation strategy can result in the maintenance of performance even when associated with negative factor influences. The experienced negative influence may therefore not be reflected in the controllers’ performance even though the influence is present. Therefore, some indicators will not strictly associate with ATCO performance, but rather the ATCO’s subjective state.
Categorisation of indicators used by air traffic controllers
Figure 1 provides an initial broad categorisation of indicators based on qualitative data from controllers. It should be noted that the categories are not mutually exclusive. An indicator can be both experienced internally and also overtly reflected in performance, such as a feeling of being ‘slow’ and falling behind with traffic. Participants distinguished between internal, subjectively experienced indicators and external, observable indicators.
Indicators
Internal – subjective to the
External –overt, observable
controller
indicators
Compensati
Performance
Physical on strategy decline change
Figure 1. Categorisation of indicators produced from qualitative data
Figure available in the original PDF paper.
Internal indicators Internal indicators may alert the controller to specific state or negative influence on performance: “I know that when I start thinking, ‘Oh it’s going fine’ I’ve learned that I force myself to tighten the bolts and to really pay extra attention” (Participant 1).
External indicators By contrast, external indicators are overt and observable. Three broad sub-categories of external indicators emerged from participant responses (Figure 1). Changes to a controller’s performance, such as a performance decline, serve as an external marker to the controller his/herself and their colleagues that the controller may not be comfortable. Another sub-category of external indicators is behavioural and physical changes in a controller. Examples include a face becoming red, or fidgeting, which provide information to other controllers: “You see it coming, you see them getting nervous, you see them talking faster” (Participant 2). Adaptive changes to the control strategy to mitigate negative influences on performance are a sub-category of external indicators. Compensation strategies as indicators suggest that a controller is feeling uncomfortable, but is aware of the potentially negative influence, and is attempting to protect and maintain performance. As one controller summarised “When somebody is just extra careful, I suppose that it’s because they feel that they have to be extra careful” (Participant 1).
If a controller is aware of internal indicators, a compensation strategy may be selected and applied. This is represented in Figure 1 by an arrow with a dashed line connecting the internal marker to the compensation strategy. This is then observable in the method of controlling. Alternatively, a decline in performance may alert the controller that a compensation strategy is required to protect performance. This is represented by an arrow with a dashed line connecting performance decline and compensation strategy. In addition, colleagues may observe the controller’s performance decline and apply their own compensation strategy to support the controller’s performance.
Sub-categories of external indicators may provide distinct information Performance declines or errors may provide a more serious indication that a controller is experiencing difficulties with the task: “If I see that someone is correcting themselves very often then I would pay a lot of attention to what he’s actually doing … I really follow every single clearance. I try to support as well like giving hints” (Participant 11). A physical change (e.g. red face, yawning, laid back posture) may indicate a change in controllers’ cognitive state (Sharples et al., 2012), although may not be related to feelings of discomfort. Finally, adaptive changes in control strategy inform colleagues that the executive controller (EC) is experiencing discomfort with the control task, although the EC is aware of this and is attempting to maintain performance with the application of a compensation strategy.
Indicators of specific factor influences Participants were presented with a list of factors (workload, stress, situation awareness), accompanied by standardised definitions, developed from previous research (see Edwards, Sharples, Wilson & Kirwan, 2012). Participants were asked to identify internal or external indicators that they believed to be associated with a specific influencing factor. Although not contained in the study questions, all participants naturally reported adaptive compensation strategies that were applied in response to the detection of potential performance decline. These strategies can also serve as observable indicators. Therefore compensation strategies that controllers use in response to factor influences are also reported.
Workload Controllers differentiated between high workload and low workload. Each form of workload was reported to be associated with different indicators.
High workload Participants reported internal (Table 1) and external (Table 2) indicators of potential performance decline that were associated with high workload. Findings were grouped into categories. Changes to subjective feelings and performance changes were reported as important indicators that a controller may be reaching the edge of performance.
Table 1: Indicators of high workload internal to the controller Category IndicatorsTable 2: Observed indicators of high workload Category Indicators
Controllers reported using specific compensation strategies in high taskload periods if they were aware of potential performance decline. These were primarily control strategies such as reducing efficiency to ensure safety, or going ‘back to basics’ to ensure all aircraft are safe. Preparation for a high taskload was reported to be the most effective strategy.
Low workload
Table 3 and Table 4 list the internal and external and compensation strategy indicators of reaching the edge of performance during low workload. In comparison to high workload, the indicators reflect a potential influence on performance through potential boredom or relaxation, leading to distraction: “In low workload, there's nothing to do so you start doing other things, boredom becomes an issue and then you start talking or having a chat or doing whatever and it's, yeah, you can miss things” (Participant 10). One particularly interesting marker during low workload is that controllers report leaving a problem develop for longer or creating complex situations to reduce boredom.
Table 3: Indicators of low workload internal to the controller Category IndicatorsTable 4: Observed indicators of low workload Category Indicators
Several distinct types of performance decline are associated with workload, such as overlooking aircraft (vigilance issues). Workload may therefore negatively influence other factors such as vigilance and situation awareness which are then observed to be causal factors of performance decline. This is important for understanding underlying causes of performance decline that may manifest as a different factor.
Stress Respondents differentiated between stress resulting from personal situations and task-related stress, which were both reported to negatively influence performance. Respondents suggested that the indicators and effects of stress were the same regardless of cause. Respondents also differentiated between positive, or ‘excited’, stress and stress which results in negative feelings and potentially performance change.
Only indicators of stress that influenced controllers negatively were discussed. Respondents emphasised changes in subjective feeling, such as feeling tense, uncomfortable and anxious, as unambiguous indicators of stress (Table 5). This suggests that stress may affect subjective experience and associated cognitive changes rather than performance directly. Several observable indicators () were manifestations of emotional responses, such as frustration and demonstrations of anger, and associated physiological changes such as vocal changes, shaking and fidgeting. Compensation strategies (Error: Reference source not found) were designed to counteract the influences of stress on the controller, such as emotion regulation and practical strategies such as reducing rate of speech. Support was sought from the CC to further protect performance.
Table 5: Indicators of stress internal to the controller Category IndicatorsTable 6: Observed indicators of stress Category IndicatorsTable 7: Compensation strategies which are also indicators of stress Category Indicators
Situation Awareness Controllers referred to a decline or loss of situation awareness (SA) as ‘losing the picture’. The loss of SA was reported to be progressive and occur in stages which were associated with different indicators: “It starts off by just falling behind a bit. So you might just be a few steps behind what you’re supposed to be doing and if that builds up too much then you will get to the point where you start to lose the picture” (Participant 20). Therefore, the internal and observable indicators for SA were categorised into indicators which indicate a controller is progressing to losing the picture, or that a controller has lost the picture, and therefore SA (Table 8, Table 9). The context of high or low taskload was reported to influence SA, and the associated indicators of SA, differentially. The progressive decline of SA was only reported under conditions of high taskload. In low traffic the loss of awareness was more instantaneous: “We sort of relaxed, ‘Oh, it’s done now’, eating a sandwich and both of us had forgotten about it [the aircraft]” (Participant 4). Controllers felt that recovery of SA was relatively easy in periods of low traffic as compared to high traffic.
Compensation strategies protect performance when a controller is losing the picture. The compensation strategies from the EC attempt to make the situation safe when awareness is degraded. Conversely, compensation strategies by the CC are tactical and appear to facilitate the EC in rebuilding the picture. The more degraded awareness is, the more reliant the EC may be on the CC to protect performance and rebuild the picture.
Table 8: Indicators of reduction and loss of the picture internal to the controller Indicators internal to theTable 9: Observed indicators of reduction and loss of the picture
The importance of awareness of indicators, and effects of individual differences All participants reported that indicators provide information that performance is, or may soon be, negatively influenced by factors such as workload and stress. Indicators, and controller awareness of these indicators, therefore appear to have a critical role in the maintenance of an extremely high level of human performance, even when potentially negatively influencing factors are present. However, a common theme throughout the results was that employing a compensation strategy to support performance was dependent on the controller’s awareness of the available internal and external indicators. One participant summarised: “I’d say 300%, if you know that you’re not being top performing today then that’s fine, just adapt your working style and you’ll get through the day…if you don’t know it and you’re still trying to do the same then it might end in tears” (Participant 12).
When an influencing factor is present (e.g. extremes of workload, stress, reduction in SA), performance may be protected by several ‘barriers’ (created from awareness and compensation strategies) before becoming vulnerable to factor influences. Based on participant responses, if an influencing factor occurs (e.g. workload extremes) internal indicators such as feelings of discomfort may alert the EC and trigger the application of a compensation strategy. Performance may then be maintained. If an internal marker did not occur or was not detected, another opportunity to detect the issue may occur through observable indicators. For example: “It’s getting busy… you start speaking fast and then somebody says “Say again” and then that’s it, you have a hint. ‘Okay good, I have to slow down’”
(Participant 1). However, if the EC is not aware of indicators, protection of performance is dependent on a colleague’s (i.e. the CC’s) awareness: “You’re not aware that you’re working to the edge of your performance then you need to rely on other people to tell you” (Participant 15). If neither controller notices an issue, participants suggested that performance is more likely to decline than if a compensation strategy was applied. This concept may not be applicable in all situations, although captures many elements of controllers’ answers.
Controllers reported that indicators of potential performance decline are not formally taught but are learned through experience. Therefore, differences in experience may limit a controllers’ awareness of internal and external indicators. Controllers need to build awareness of the respective indicators prior to being able to identify the specific indicators. This is particularly relevant to inexperienced controllers.
Individual differences in awareness may alter the extent that a controller is aware of these indicators. Participants appeared to differ in the extent of conscious awareness of personal indicators. A minority of controllers (3/22) suggested that they personally ‘sense’ or ‘just know’ when they are reaching a performance limit but could not identify how they knew. In contrast, most controllers could identify personal indicators.
OVERALL DISCUSSION Expert opinion regarding indicators of potential performance decline was generated by conducting one hour face-to- face interviews with 22 en-route controllers from MUAC. Interviews were orthographically transcribed and analysed using thematic analysis. Results revealed that indicators were used in an ATC setting by all respondents, as indication of when a controller was reaching the edge of performance, or a factor was negatively influencing performance. It was considered a natural process that all controllers used. Participants differentiated between internal indicators, representing a subjective experience, and external indicators, which were observable. Three sub- categories of external indicators were identified: changes in performance, physical signs and application of a compensation strategy.
Participants confirmed that specific factor influences on performance were associated with specific internal and external indicators. Indicators were identified for the factors of workload, stress and SA. Participants also reported adaptive changes in control strategy that were applied to maintain performance when the influencing factor was present, which resulted from the identification of indicators. Compensation strategies are therefore an integral aspect of the application of indicators in ATC. In addition, compensation strategies were also identified to be used as external indicators themselves of controller discomfort.
Specific factors influenced performance differentially, which in turn influenced the associated indicators. The influences of workload and stress may not be visible in performance but manifest as other factor influences, such as overlooking aircraft (a vigilance issue). It is therefore important for aviation professionals to acknowledge the underlying issues of performance declines to gain a valid and comprehensive understanding about factor influences and to understand how to best protect performance. Factors such as SA appear to influence performance directly.
Awareness emerged as an integral element in the use of indicators; controllers needed to be aware of their own or colleagues’ indicators in order to apply a compensation strategy. It was suggested that there were individual differences in overall levels of awareness. In addition, controllers suggested that it was harder to be self-aware than aware of colleagues’ indicators. This was especially true for inexperienced controllers who were perceived to not have the experience in order to identify indicators and apply adaptive strategies. Awareness is critical in the identification of indicators and subsequent application of adaptive strategies to protect performance. Supervisors and controllers may benefit from a more standardised list of generic indicators, and in addition, possibly learn different indicators to monitor from each other. Workshops that provide standardised indicators for which to monitor, and support development of awareness of indicators for self and colleagues, may support controllers in protecting and maintaining performance in the presence of negative influences. This may also support trainees in protecting performance whilst developing the required experience to identify specific, personal indicators.
CONCLUSIONS
The findings from this qualitative interview study with 22 air traffic controllers suggests that indicators of potential performance decline are currently utilized in air traffic control operations rooms. Indicators are reported to have a functional purpose which enables controllers to adapt to, and mitigate, potential negative influences on performance, therefore facilitating controllers’ maintenance of an exceptionally high level of performance. In order for indictors to be utilized to maintain performance, operational personnel’s awareness of indicators emerged as a central theme. This finding suggests operational personnel such as supervisors may benefit from a formal training to raise awareness of these indicators, or utilization of a list of generic indicators. Findings have implications for both live operations, and research in the air traffic control domain. Further research must now investigate the validity and reliability of utilizing these indicators as a basis for observation of potential performance declines. In addition, further research may investigate the application of standardized indicators to highlight to supervisors when a controller’s performance may benefit from supportive strategies. In this way, performance declines, and ultimately, performance-related incidents, may be prevented.
REFERENCES
Chang, Y. & Yeh, C. (2010), Human performance interfaces in air traffic control. Applied Ergonomics, 41, 123-129. Edwards, T., Sharples, S., Wilson, J. R., and Kirwan, B. (2012). Factor interaction influences on human performance in air traffic control: The need for a multifactorial model. Work: A Journal of Prevention, Assessment and Rehabilitation, 41(1), 159- 166. Kirwan, B. (2011). Incident reduction and risk migration. Safety Science, 49(1), 11-20. Mackieh, A. & Cilingir, C. (1998). Effects of performance shaping factors on human error. International Journal of Industrial
Ergonomics, 22, 285-292. Park, K. S. & Jung, K. T. (1996). Considering performance shaping factors in situation-specific human error probabilities. International Journal of Industrial Ergonomics, 18, 325-331.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Advances in Human Aspects of Transportation: Part II
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-4951-2098-5
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional