Åsa Svensson, Maximilian Peukert (2022). User experiences and expectations towards automation and safety in air traffic control. In: Human Factors in Transportation. AHFE Open Access, vol. 60, pp. 632–639. https://doi.org/10.54941/ahfe1002499
Various forms of advanced automation in air traffic control are being developed to reduce air traffic controller (ATCO) workload, support performance, and increase safety. However, higher levels of automation can negatively affect the ATCO's performance, reduce situation awareness (SA), and the usage of automation also depends on user acceptance. Previous studies have shown that automation can be met with resistance from ATCOs, which might lead to disuse. Thus, this paper continues the exploration of ATCOs' expectations of automation to deeper understand ATCOs’ fears and anticipations. Through a questionnaire, operational ATCOs answered questions regarding their current experiences and future expectations regarding automation extend, safety, SA, and workload. The quantitative analysis shows that automation and workload are expected to increase in the future compared with today and that safety and SA are expected to decrease. Even though the participants highlighted that they do not want to become the backup system to the automation, it is in that direction the development is heading. These are worrying results and something the air traffic management community should see as a wake-up call. A well-calibrated attitude towards automation seems crucial to ensure the best human-automation interplay.
The air traffic management/control (ATM/ATC) industry faces challenges such as higher efficiency demands and complex traffic situations while main- taining high safety. To meet such challenges, higher levels of automation (Endsley, 2018; Kaber, 2018; Sheridan, Verplank, & Brooks, 1978) is seen as one solution (SESAR, 2006, 2017). Various forms of automation are being developed to reduce operator load, support performance, and increase safety (SESAR, 2017). However, automation might also introduce risks for decrea- sed safety, even though safety is a fundamental basis in ATM (SESAR, 2006). Higher levels of automation can negatively affect operator’s performance since the operator could be removed from the control loop (Baxter, Rook- sby, Wang, & Khajeh-Hosseini, 2012; Endsley & Kaber, 1999; Endsley & Kiris, 1995), resulting in loss of situation awareness (SA) (Endsley & Kaber, 1999) and out-of-the-loop problems (Kaber & Endsley, 1997). The usage
User Experiences and Expectations Towards Automation and Safety 633
of automation (e.g., tools and functions) depends on user acceptance of the automation (Bekier, 2013; Hilburn & Flynn, 2001). However, automation can be met with resistance and scepticism from the operators (Borst, Westin, & Hilburn, 2012; Hilburn & Flynn, 2001; Svensson, Lundberg, Forsell, & Rönnberg, 2021), and several factors affect whether automation is accepted and used (Bekier, 2013; Eurocontrol, 2000, 2004; Lee & See, 2004; Mirchi et al., 2015; Nijhuis et al., 1999; Parasuraman, Duley, & Smoker, 1998; Thompson & Bailey, 2000). A recent study explored air traffic controllers’ (ATCO) experience of current ATM systems and focused on teamwork betw- een humans and automation (Svensson et al., 2021). However, few studies have addressed the ATCOs’ fears and anticipations regarding the increase of automation in operational settings. This study presents ATCOs’ expe- ctations regarding automation, safety, situation awareness, and workload, explored through a questionnaire and workshops with operational ATCOs in Sweden.
BACKGROUND
A previous survey study showed that ATCOs believed that automation would increase in the future, that situation awareness (SA) and safety would decre- ase, and that workload will stay at the same level in the future compared with today (Svensson et al., 2021). Introducing automation that intrudes on the core tasks of maintaining SA (Endsley, 1995; Stanton, Salmon, Walker, Salas, & Hancock, 2017) and decision-making for ATCOs (Eurocontrol, 1996) changes the ATCO’s role fundamentally. It can be difficult for many ATCOs to accept new automation concepts, even more so if the ATCOs cannot see its possible benefits (Eurocontrol, 2004). Experience has taught operators to be cautious (Hopkin & Wise, 1996). Hence, they can be reluctant to resign responsibility for decision-making or control to the automation (Hil- burn, 2000). ATCOs may accept advisory automation, but only as long as the ATCO makes the decisions (Hilburn, 2000). Svensson et al. (2021) and Bekier, Molesworth, and Williamson (2011) found through survey studies that few respondents approved of a tool that involved shifting decision- making away from the operator. However, ATCOs can agree on automated tools that assist in organizing materials for decision-making (Bekier et al., 2011). Studies have shown that ATCOs also feared that future automation within ATM will include more monitoring tasks (Svensson et al., 2021) and, thereby, greater risks of fatigue (May & Baldwin, 2009) and mistakes when taking control if the automation fails (Hilburn, 2000). Thus, the willingness to accept automation appears to be task-dependent and dependent on the purpose of the automation (Bekier et al., 2011).
The present study explores ATCOs’ fears and anticipations towards auto- mation, what they think will happen in the future regarding automation, safety, SA, and workload, and positive and negative aspects of automation. By investigating ATCOs’ experiences and expectations, automation could be developed to support the need of the ATCO and implemented in an acceptable and usable way.
634 Svensson and Peukert
METHOD
An online questionnaire was distributed through ATM managers to operati- onal ATCOs in Sweden, including ATC sites such as area control, approach control, and tower control. The questionnaire was voluntary and could be done during work hours. A total of n = 113 licensed ATCOs (68 male and 41 female, 4 choose not to disclose gender) responded to the online questi- onnaire. The respondents’ mean experience was M = 20.9 years (SD = 7.6), ranging between 2 and 35 years. Age was divided into four groups: 30 years old or younger (n = 1), between 30 and 39 years (n = 21), between 40 and 49 years (n = 51), and 50 years or older (n = 40). The questionnaire was developed explicitly for this study and consisted of 50 questions and statements split into four sections. The questions asked about ATCOs’ experiences and expectations of current and future ATM systems, teamwork between the ATCO and current and future automated systems, and demography. Likert scales (Fink, 2013) ranging from 1 = ‘very low’ or ‘very much disagree’ to 5 = ‘very high’ or ‘very much agree’ were used and free-text sections. Respondents were free to move back and forth between questionnaire sections and change their responses.
To discuss the results from the questionnaire study and ask follow-up que- stions on a qualitative level, semi-structured group interview workshops with operational ATCOs were conducted. Nine workshops were held, with three to five ATCOs in each workshop; a total of n = 35 operational ATCOs par- ticipated in the workshops. The work experience of the participants in the workshop varied from 1 year to 25 years and a mix of male (n = 16) and female (n = 19) participants. The outcome of the workshops and the free- text answers from the questionnaire were analysed with a thematic analysis (Creswell, 2009).
RESULTS
Even though the questionnaire consisted of 50 questions and statements, for this paper, the teamwork questions from the questionnaire are not quantita- tive analysed and reported. Instead, this paper focuses on automation, safety, situation awareness, and workload from the questionnaire and the worksh- ops. Quotations from the respondents in the questionnaire and workshops, translated from Swedish, are given as illustrations.
Quantitative results: For statistical analysis, SPSS Statistics Version 26 (IBM Cooperation, USA) was used. Paired-samples t-test were carried out to investigate the difference between participants’ experiences of current ATM system and expectations for future ATM systems. The results show significantly differences between current and future automation amount, (t(112) = −10.31, p < .001), safety (t(112) = 11.82, p < .001), situation awareness (t(112) = −16.31, p < .001), and workload (t(112) = −4.86, p < .001). See figure 1.
Qualitative results: Responses from the questionnaire and workshops indicate that the ATCOs think there are many positive aspects of automa- tion in ATC. For example, the respondents from both the questionnaire and
User Experiences and Expectations Towards Automation and Safety 635
Figure 1: Compared with today, the participants believed that the grade of automation will be higher, that safety and SA will decrease, and that workload will increase in the future. Error bars indicate standard deviation, and dots represent outliers.
the workshops mentioned tools such as MTCD (medium-term conflict dete- ction) and STCA (short-term conflict alert) as assisting the ATCO in detecting conflicts, contributing to the ATCO saving time for other tasks, and enha- ncing safety. In addition, existing automation (e.g., conflict detection tools and communication tools) makes the work a little more efficient for the ATCO and, hence, lowers the ATCO’s workload. One response from the questionnaire: “In most parts very positive. Makes the total workload lower compared to less automation but with the same amount of traffic.” According to the responses, automation also removes monotonous tasks such as basic communication tasks, in today’s system performed by the technical system instead of the ATCO. The ATCO can also handle a higher capacity and a more extensive traffic volume with more automated tools. One response from the questionnaire: “Today’s tools are not too advanced and often simpler auto- mation. This makes it relatively easy to use and does not interfere with the work in position. The tools we have today are supportive of the operator, but you are still the one who makes the final decision”. Many participants saw automation as a safety net, and that the system hinders the ATCO from making errors. One response from the questionnaire: “Creating more time for the most important things, to control flights and flight safety”. Several participants requested an automated tool to assist them with inbound clea- rance (for en-route sectors), which takes unnecessary time and capacity from the ATCO and needs to be done for every inbound aircraft. One response from the questionnaire: “I would like to be able to focus on more com- plex tasks and letting the system handle routine stuff, like giving inbound clearance”.
Even though the participants believed there are many benefits with auto- mation, they also thought there are (and will be) many issues with it as well. During the workshops, many participants were sceptical towards more
636 Svensson and Peukert
automation and believed that automation often failed them in their current work. One response from the workshops: “It seems to work, but in rea- lity, we make it work”, meaning that even though the automation is there to support them, there are many malfunctions the ATCOs need to handle as well. If malfunctions occur today, there are safety procedures to follow, which also will be the case for future automation. One response from the question- naire: “When it comes to CPDLC (controller pilot datalink communication), it could be more reliable. Sometimes it works, sometimes it does not.” Many participants also believed that today’s conflict detection tools are helpful but not always trustable. The participants in the workshops discussed that false alerts sometimes are so frequent that they “numb” the ATCOs in reacting to the warnings. Another aspect that the participants highlighted is that they could trust the system too much. One response from the questionnaire: “Easy to get ‘blind’ and trust the systems too much.” In addition, the ATCO often double-checks information provided from the system, increasing the wor- kload, which was also a fear for future automated systems. However, the participants found it concerning to increase traffic volumes along with com- plex automation. This is due to today’s complexity of closing down sectors and moving traffic, especially if SA is decreased (figure 1) due to automation taking many tasks. Decreased SA was one of the major concerns regarding implementing more automation, seen in both the quantitative and qualitative results. One response from the questionnaire: “I don’t think we are there just yet, but we might lose control and situational awareness with more automa- tion.” Decreased SA might jeopardize the ATCO’s ability to stay in the loop, control the situation, take control from the system if necessary, and, thus, contribute to a decreased safety level.
There was a big concern about what would happen if the automation would, for any reason, fail. Can the ATCO handle a transition after a sudden or unexpected degradation of automation level, or when the auto- mation completely stops working? According to the quantitative results, the participants believed workload will increase in the future compared with today due to increased automation that is imperfect. If automation is introduced and used more frequently, the participants stated that they need to know that it works in all situations. The ATCOs need to under- stand when and why the automation might degrade in automation level and know that they can handle the situation without the automation. There were many concerns regarding increasing the amount of automation; for example, one response from the questionnaire: “Balancing the amount of automation so that the controller still remains in the loop”. Keeping the human operator in the loop is essential to maintain safety when the levels of automation increase. According to the participants, this will require more training, education, and information about the automation that will be implemented.
CONCLUSION
Some of the major reasons for introducing higher automation levels in ATC are to increase capacity, cost efficiency, and safety. Automation is introduced
User Experiences and Expectations Towards Automation and Safety 637
as a safety net support to improve safety performance. However, if the auto- mation fails, the ATCO is supposed to take control of the situation (e.g., with a higher and more complex traffic volume than today), making the ATCO a backup system to the automation, something the participants from this study and previous studies (Svensson et al., 2021) fear. The results also indi- cate a concern regarding the expected increase of workload and automation in the future, while safety and SA are expected to decrease. However, bala- ncing workload and automation is difficult in today’s ATC environments due to automation, e.g., MTCD, giving “false alerts” (due to parameter settings in the system that cannot be changed easily). These results align with pre- vious results (Svensson, 2014, 2015). In addition, according to the results, trust towards automation should be well-calibrated since it can easily be ero- ded due to automation failures. The ATCOs should be involved in all early stages of automation development, from idea development based on previ- ous research and experience to discussion groups, workshops, development prototypes, etc. This is particularly important when the automation can mal- function, and the ATCO needs to be the backup system. The expectations highlighted from this study must be understood and managed through change management, communication, and dialogues between research, development, and end-users. The ATCO needs to know the benefits of the automation, how it works, why it performs in specific ways, why it malfunctions, and what happens when it does. This calls for better integration of human capa- bilities and transparency of the system functionalities. Extensive automation training is necessary for the ATCO to keep safety at a high level. In addition, follow-up studies with updated questionnaires are important to capture exi- sting concerns, fears, and anticipations amongst the end-users (in this case ATCOs).
ACKNOWLEDGMENT
The authors would like to thank all the respondents who answered the questi- onnaire and participated in the workshops. The present study was funded by the Swedish Transport Administration, and the LFV Air Navigation Servi- ces of Sweden, via the project Flygtrafikledning i Sverige: möjligheter och farhågor med automation.
REFERENCES
Baxter, G., Rooksby, J., Wang, Y., & Khajeh-Hosseini, A. (2012). The ironies of automation: still going strong at 30? Paper presented at the Proceedings of the 30th European Conference on Cognitive Ergonomics, Edinburgh, United Kingdom. Bekier, M. (2013). Automation acceptance in air traffic management. (PhD). New
South Wales, Sydney, Australia. Bekier, M., Molesworth, B. R., & Williamson, A. (2011). Tipping point: The narrow path between automation acceptance and rejection in air traffic management. Safety Science, 50(2), 259–265. Borst, C., Westin, C., & Hilburn, B. (2012). An Investigation into the Use of Novel
Conflict Detection and Resolution Automation in Air Traffic Management. Paper presented at the SESAR Innovation Days, Braunschweig, Germany.
638 Svensson and Peukert
Creswell, J. W. (2009). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches (Vol. 3). Thousand Oaks, California, US: Sage Publications, Inc. Endsley, M. R. (1995). Toward a Theory of Situation Awareness in Dynamic Systems.
Human Factors, 37(1), 32–64. Endsley, M. R. (2018). Level of Automation Forms a Key Aspect of Autonomy Design. Journal of Cognitive Engineering and Decision Making, 12(1), 29–34. Endsley, M. R., & Kaber, D. B. (1999). Level of automation effects on performance,
situation awareness and workload in a dynamic control task. Ergonomics, 42(3), 462–492. Endsley, M. R., & Kiris, E. O. (1995). The out-of-the-loop performance problem and level of control in automation. Human Factors, 37(2), 381-394. Eurocontrol. (1996). Model for Task and Job Descriptions of Air Traffic Control-
lers. European Air Traffic Control Harmonisation and Integration Programme (HUM.ET1.ST10.DEL-01). Brussels, Belgium: Eurocontrol. (2000). Attitudes towards future automation concepts: A litterature review. Brussels, Belgium: Eurocontrol. (2004). Age, Experience and Automation in European Air Traffic
Control - Survey in the ECAC Area. Brussels, Belgium: Fink, A. (2013). How to conduct surveys: a step-by-step guide (5th ed.). London, UK: Sage Publisher. Hilburn, B. (2000). Conflict Resolution Assistant Level 2 (CORA2) Controller
Assessments: Final Report. Hilburn, B., & Flynn, M. (2001). Air traffic controller and management attitudes toward automation: an empirical investigation. Paper presented at the Air Traffic Management Reserach and Development Seminar, Santa Fe, USA. Hopkin, V. D., & Wise, J. A. (1996). Human Factors in Air Traffic System
Automation. Automation and Human Performance: Theory and Applications, 192. Kaber, D. B. (2018). Issues in human-automation interaction modeling: Presumptive aspects of frameworks of types and levels of automation. Journal of Cognitive Engineering and Decision Making. Kaber, D. B., & Endsley, M. R. (1997). Out-of-the-loop performance problems
and the use of intermediate levels of automation for improved control system functioning and safety. Process Safety Progress, 16(3), 126–131. Lee, J. D., & See, K. A. (2004). Trust in Automation: Designing for Appropriate Reliance. Human Factors, 46(1), 50–80. May, J. F., & Baldwin, C. L. (2009). Driver fatigue: The importance of identifying
causal factors of fatigue when considering detection and countermeasure techno- logies. Transportation research part F: traffic psychology and behaviour, 12(3), 218–224. Mirchi, T., Vu, K.-P., Miles, J., Sturre, L., Curtis, S., & Strybel, T. Z. (2015). Air Traffic Controller Trust in Automation in NextGen. Procedia Manufacturing, 3, 2482–2488. Nijhuis, H., Buck, S., Kelly, C., Goillau, P., Fassert, C., Maltier, L., & Cowell, P.
(1999). RHEA. Role of the Human in the Evolution of ATM systems. Final report. Parasuraman, R., Duley, J., & Smoker, A. (1998). Automation tools for controllers in future air traffic control. The Controller. SESAR. (2006). Air Transport Framework - The Current Situation. Brussels, Belgium: SESAR. (2017). European ATM Master Plan Level 3 - Implementation Plan. Brussels,
Belgium:
User Experiences and Expectations Towards Automation and Safety 639
Sheridan, T. B., Verplank, W. L., & Brooks, T. L. (1978). Human/computer control of undersea teleoperators. Paper presented at the Proceedings of the International Conference on Cybernetics and Society, Tokyo-Kyoto, Japan. Stanton, N. A., Salmon, P. M., Walker, G. H., Salas, E., & Hancock, P. A.
(2017). State-of-science: situation awareness in individuals, teams and systems. Ergonomics, 60(4), 449–466. Svensson, Å. (2014). Controllers’ use of the air traffic management tools. Student report. Department of Computer and Information Science. Linköping, Sweden. Svensson, Å. (2015). Efficiency of air traffic management tools. Department of
Computer and Information Science. Linköping, Sweden. Svensson, Å., Lundberg, J., Forsell, C., & Rönnberg, N. (2021). Automation, teamwork, and the feared loss of safety: Air traffic controllers’ experiences and expectations on current and future ATM systems. Paper presented at the European Conference on Cognitive Ergonomics 2021, Siena, Italy. Thompson, R. C., & Bailey, L. L. (2000). Age and attitudes in the air traffic control
specialist workforce: an initial assessment. Oklahoma City, USA.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors in Transportation
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-36-0
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional