Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Continuing Development of a Novel Framework for Visual Air Traffic Controller Tasks: Determining metrics for evaluating spatial relationships

Publication year
Digital Object Identifier10.54941/ahfe1002495

Abstract

The purpose of this paper is to present a combination of metrics for evaluating spatial relationships while utilizing new airport traffic control tower (ATCT) technologies that replace a traditional out-the-window view. Johnson’s criteria of object detection, recognition, and identification are used as an objective metric when siting new ATCTs. However, Airport Traffic Control Specialists (ATCSs) must incorporate additional cues into their decision-making process for the variety of tasks relying on visual information. In particular, ATCSs utilize perceptual cues from the airport's environment to ensure both runway separation and sequencing/spacing based on visual cues provide safe and efficient operations. For both safety and efficiency, ensuring runway separation is one of the most important services ATCSs provide. When an ATCS is responsible for ensuring runway separation using a display screen instead of a window, additional factors need to be considered when evaluating these technologies’ ability to provide the necessary visual information. In addition to Johnson’s criteria for detecting, recognizing, and identifying a single object, a combination of factors, such as landmarks, relative speed, crossing intersections, etc., need to be utilized to ensure ATCSs can determine spatial relationships between two objects, and therefore provide runway separation. This paper will discuss the application of these factors in an evaluation of new technologies for ATCTs.

INTRODUCTION

The purpose of this paper is to present a framework for assessing spatial relationships in visual Air Traffic Control Specialist (ATCS) tasks. New eva- luation techniques are needed to assess emerging technologies intended for providing airport traffic control tower (ATCT) services that require visual information. Some of these technologies fundamentally change how ATCSs complete visual tasks. For example, changing the traditional out-the-window view to a video screen displaying a live camera feed. Because of these fun- damental changes, evaluators need to be creative in developing techniques to ensure the continuation of the safe and efficient provision of ATCT services.

© 2022. Published by AHFE Open Access. All rights reserved. 600

Continuing Development of a Novel Framework for Visual Air Traffic Controller Tasks 601

One accepted method for evaluating visual tasks is using Johnson’s criteria of detection, recognition, and identification (FAA, 2022). This metric calcu- lates the probability of detection of an object imaged by an optical system (Sjaardema, Smith, and Birch, 2015). Currently, the Federal Aviation Admi- nistration (FAA) has been using the accepted work of Johnson’s Criteria in the tower siting process. However, Johnson’s criteria only encompass the perception of one object at a time. Many ATCT visual tasks require ATCSs to compare an object with another object to determine spatial relationships (e.g., same runway separation or sequencing and spacing). An objective way of assessing ATCSs’ ability to determine spatial relationships is needed to ensure new technologies support safe and efficient ATCT operations.

DEVELOPING HUMAN SYSTEMS INTEGRATION TOOLS TO SUPPORT SYSTEMS DESIGN To create more structured evaluation criteria for the visual ATCS tasks, terms were defined to account for the lack of certain perceptual cues. The terms “detect,” “recognize,” “identify”, “observe”, and “verify” are verbs used to help create additional visual criteria specifically for visual ATCS tasks (Kinsella, et al., 2019).

Johnson’s criteria are typically used under the assumption that normal human perceptual cues are in place. However, with some new technologies, these cues are not perceivable in the same way as they are in an out-the- window view. For example, when transitioning from an out-the-window view to a display screen, certain environmental perceptual cues are not available or must be perceived differently than from a traditional “brick-and-mortar” ATCT. Humans use environmental depth cues to build a mental model of depth. Environmental cues are the primary source of depth information used by the brain (Cutting & Vishton, 1995; Blake & Sekule, 2006). Changing the way ATCSs perceive depth could lead to difficulties in evaluating spe- cific ATCS visual tasks, such as providing sequencing and spacing services and runway separation. To assess these tasks, it is important to establish the ability to perceive depth and spatial relationships using the new technology. With this in mind, new evaluation techniques expanding on Johnson’s criteria were developed by the FAA to evaluate new technologies and ATCS’ ability to provide ATCT services while utilizing these technologies.

New Technologies and the Need for Additional Framework Humans use all available depth cues to gain depth information to perceive the spatial and absolute depth of an object and its relationship to other objects. Three ranges determine the effectiveness of each perceptual cue: personal space (0 – 1.5 meters), action space (1.5 – 30 meters), and vista space (over 30 meters) (Cutting and Vishton, 1995). Figure 1 shows the effectiveness of each cue based on distances. For many ATCT visual tasks, ATCSs will be viewing objects in vista space and must rely on environmental cues.

Controllers in a traditional “brick-and-mortar” ATCT can use a three- dimensional view to perceive depth and spatial relationships which provides important support for their control tasks. One of these tasks is to ensure

Figure 1: Effectiveness of select perceptual cues by viewing range. Green boxes indi- cate useful cues, yellow boxes indicate supporting cues, and orange boxes indicate not useful cues..
Original PDF Figure 1:

runway separation is efficient between arriving and departing aircraft. To provide runway separation services in a “brick-and-mortar” tower, ATCSs would utilize occlusions, shadows/shading, and aerial perspective to deter- mine when an aircraft passes the 3,000 ft, 4,500 ft, or 6,000 ft runway markers. Controller judgment is also included in decision-making and is a mental and cognitive process (Ellis and Liston, 2011). The visual environ- ment should allow the controller to identify the aircraft type, and state (e.g., lifting off or touching down), as well as recognize the aircraft’s relative speed, direction, and position. When instructions are issued to aircraft, the control- ler must be able to verify compliance. Assurance of spatial relationships is needed to provide same runway separation (Ellis and Liston, 2011). To pro- vide this service using new technology (that uses cameras and display screens, for example), it will be important to assess how effective these cues are in providing spatial relationship information to the ATCS.

In the example of runway separation, using Johnson’s criteria for the asses- sment of new technologies will not be enough. This is because Johnson’s criteria only account for perceiving a single object. For same runway sepa- ration and other visual ATCS tasks, perception of a single object in relation to a different single object needs to be assessed. Object detection, recogni- tion, and identification are only part of the visual processing of the controller when looking at two aircraft via screens. The controller must use other visual cues such as occlusion, size, shadows/shading, aerial perspective, and motion parallax. Evaluators and controllers can determine whether these cues can be perceived in a new technology by observing aircraft direction of flight, rela- tive speed, relative altitude of the arriving aircraft, distance to landmarks, and relationship to runway and taxiway intersections.

A NEW FRAMEWORK A new framework for determining whether spatial relationships can be perceived when utilizing a new technology has been developed. This frame- work incorporates ideas used in Johnson’s criteria but expands to include metrics to ensure other environmental perceptual cues can be perceived.

First Johnson’s criteria are used not only to ensure grounded aircraft can be detected, recognized, and identified, but also that airborne aircraft can

Continuing Development of a Novel Framework for Visual Air Traffic Controller Tasks 603

be detected, recognized, and identified within a reasonable distance from the airport. Once detection, recognition, and identification have been established, these new metrics for additional environmental perceptual cues should be evaluated: — Aircraft Direction of Flight — Aircraft Relative Speed — Aircraft Relative Altitude — Aircraft Spatial Relationship to Landmarks — Aircraft Spatial Relationship to Runway and Taxiway Intersections

Utilizing these metrics will ensure important environmental cues are percei- vable not only in traditional brick-and-mortar ATCTs but also in using new technologies. For example, if an ATCS can observe aircraft direction of flight, relative speed, and relative altitude, then the human factors expert can extrapolate that data towards the observation of motion parallax, size, and shadows/shading in those conditions. If an ATCS can observe where an aircraft is in relation to specified landmarks, then the human factors expert can extrapolate that data towards aerial perspective and occlusion in those conditions. If an ATCS can determine when an aircraft passes runway and taxiway intersections, then the human factors expert can extrapolate that data towards aerial perspective, occlusion, and stereopsis in those conditions.

EXAMPLE APPLICATION New ATCT technologies are currently being evaluated by the FAA, ICAO, EUROCONTROL, and a variety of Air Navigation Service Providers (e.g., Germany’s DFS, Sweden’s LFV, and UK’s NATS). When evaluating whether cameras and display screens can replace the traditional out-the-window view for ATCSs, it is important to not only utilize Johnson’s object detection metrics but to utilize additional criteria to assess object perception in relation to other objects. To ensure ATCSs can provide runway separation, additio- nal metrics should be utilized. The following sections outline an example application assessment.

Aircraft Movement Aircraft movement cues can be very effective metrics in discerning aircraft spatial relationships and are important variables in conflict detection (Leplat and Bisseret, 1966; Lamourex, 1999). As such, it is important to ensure these movement cues are perceptible in new technologies. Aircraft movement cues to consider are the direction of flight, relative speed, and relative altitude. ATCSs use these three aircraft movement cues consistently while controlling aircraft. By utilizing these cues, ATCSs can determine where the aircraft is in relation to the runway and other airborne aircraft. For example, if the aircraft appears to be too high for landing, the ATCS can plan for a potential go-around. This will allow for control decisions, such as holding a departure short of the runway rather than issuing a takeoff clearance, that ensure safe operations.

To ensure aircraft movement cues are perceptible while utilizing new technologies, evaluations of aircraft movements should be conducted. Obse- rvational data with targets of opportunity can provide a quick look at whether a technology allows for the perception of aircraft movements, such as the direction of flight, relative speed, and relative altitude. Observational data collection can provide insight into any specific areas or scenarios that need additional evaluation. For those areas, scripted scenarios that are desi- gned to assess a specific circumstance can be carried out. One such scripted scenario idea to consider is a simulated runway incursion which examines the aircraft’s relative speed on the taxiway. The scripted scenario can evalu- ate if the ATCS can use the perceptional cues for movement to determine if the taxiing aircraft’s speed approaching the hold short lines will permit for the aircraft to remain clear of the runway. Performing this aircraft movement cue script (or other scripts deemed appropriate by the environment and eva- luation) assesses how well controllers can use cues such as relative altitude and relative speed to determine where an object is in space and is in relation to surrounding objects.

Spatial Relationship to Landmarks ATCSs utilize landmarks in many different ATCT tasks. Landmarks can be used as reporting points to aid in determining the aircraft’s distance from the ATCSs point of view. Additionally, landmarks are highly important when providing runway separation and sequencing/spacing for arriving and departing aircraft. Assessing the use of landmarks when evaluating new tech- nologies is important when determining the provision of safe and efficient ATCT services. The use of landmarks requires ATCSs to discern an aircraft’s position in relation to that landmark. Therefore, Johnson’s criteria alone will not be enough to evaluate this.

A few different techniques can be utilized when assessing the use of lan- dmarks to help with determining spatial relationships. First, observational data collection with pilots reporting landmarks will give an initial look at how landmarks will help with discerning spatial relationships. SME judgment of aircraft location compared to pilot reports in relation to the landmark can determine whether spatial relationships are perceivable. ATCSs must first build the base knowledge of perception cues (e.g., relative size and orien- tation) for what an aircraft presents as in relation to a static landmark – regardless of if the ATCS is providing services from a brick-and-mortar tower or using new technologies. Once that base knowledge has been established, scripted scenarios can be conducted to replicate pilot misreporting in relation- ship to landmarks/reporting points. For example, the ATCS may use a water tower as a reporting point for aircraft arriving from the east for a downwind entry. The ATCS will build the base knowledge of merged environmental cues, such as relative height, size, and orientation, to determine what an aircraft’s visual information is at the water tower. A scripted scenario where a pilot misreports being over the water tower when not over the water tower will permit for perceptual cues used in the determination of spatial relationships to landmarks to be examined.

Continuing Development of a Novel Framework for Visual Air Traffic Controller Tasks 605

Spatial Relationship to Runway/Taxiway Intersections An aircraft’s position in relation to runway and taxiway intersections can be a helpful tool to determine how well spatial relationships can be discerned from a specific technology. A primary responsibility of ATCSs is to sequence aircraft usage of runways and taxiways, which includes conflict prevention and resolution at intersections, ensuring a runway is clear of vehicles and aircraft for use by another aircraft, and issuing instructions to avoid foreign object debris (FOD). Being able to project an aircraft’s movement as well as determine if an aircraft has entered or exited any portion of the runway is imperative for the management of runway usage.

Evaluation techniques for assessing whether ATCSs can perceive spatial relationship information relative to intersections are like that of landmarks and aircraft movements. ATCSs must first establish a base knowledge of air- port field landmarks and runway/taxiway intersections for the provision of runway separation and conflict prevention. That base knowledge of perce- ptual airport information is used for understanding spatial relationships to runway/taxiway intersections and should be assessed in conjunction with other environmental cues. Controlled scripted scenarios can be used to deter- mine if the ATCSs can use the visual information necessary for determining these spatial relationships (e.g., determining if an aircraft approaching the hold short line is slowing down and will remain clear at the runway threshold or can stop taxiing to prevent conflict at a runway exit (aircraft movement cues); if a landing aircraft is past the appropriate runway/taxiway marker used for runway separation (spatial relationship cues)).

SME Involvement & Verification Identifying the appropriate team of SMEs will predicate the success of the eva- luation. SMEs should have expertise in the structure, process, and outcome of the project domain (Kasper, 1995; Wilson & Corlett, 2005). However, even though SMEs are experts in the domain, the assessment is of new tech- nology. Therefore, training SMEs on that technology and how it will fit into their domain is important for the success of the project. Building the base knowledge of the technology and the environment is key.

SME involvement and verification are a vital part of any evaluation of new technology. A core group of SMEs in whatever sector the new technology belongs should be trained and present in every aspect of the evaluation. While objective evaluation metrics exist (e.g., comparing detection timestamp with a GPS), there are many subjective aspects in the assessment that necessitate SME involvement.

In addition to providing insight on subjective data collection, SME involve- ment can validate and verify the data collection process and assure operations are being carried out according to FAA Orders and Policies. They can also confirm the data collected can be generalized to other ATCT tasks, thus allowing evaluators to apply results across all ATCT tasks.

CONCLUSION

A new framework for assessing spatial relationships in new technologies has been presented. This framework uses Johnson’s criteria as a starting point

to first establish object detection, recognition, and identification. Then, new metrics to account for object perception in relation to a second object were discussed. These metrics provide an objective way to determine whether spa- tial relationships can be perceived in new technology. Determining spatial relationships is at the forefront of many visual ATCS tasks, and it is neces- sary to assess whether new technologies provide ATCSs with that capability. Without the perception of spatial relationships, many ATCT services could not be provided utilizing new technologies.

REFERENCES

Blake & Sekuler. (2006). Perception. New York: McGraw Hill. Cutting & Vishton. (1995). Perceiving layout and knowing distances: the integration, relative potency, and contextual use of different depth information. Perception of Space and Motion, 1995, 69–117. Ellis, S.R., Liston, D.B. Static and motion-based visual features used by airport tower

controllers: some implications for the design of remote or virtual towers (2011). FAA, 2022. FAA - Human Factors Division | Visibility Tool. [online] Hf.faa.gov.

Available at: <https://www.hf.faa.gov/visibility/>. Federal Aviation Administration (2017). Air Traffic Control (FAA JO 7110.65.

Washington, DC. Kasper, C. (1995). Going through the motions: The ethics of process. Journal of Cardiovascular Nursing, 9(3), 62–67. Kinsella, A., Smith, L, Berry, K., Collins, R. An Eye in the Sky: Developing a Novel

Framework for Visual Airport Traffic Control Tower Tasks (2019). Lamoureux, T., 1999, The influence of aircraft proximity data on the subjective mental workload of controllers on the air traffic control task. Ergonomics, 42, 1482–1491. Leplat, J. and Bisseret, A., 1966, Analysis of the processes involved in the treatment

of information by the air traffic controller. The Controller, 1, 13–22. Sjaardema, Tracy A., Smith, Collin S., & Birch, Gabriel Carisle. History and Evolution of the Johnson Criteria. United States. https://doi.org/10.2172/1222446 Wilson & Corlett. (2005). Evaluation of Human Work. Boca Raton: CRC Press.

Cover of the proceedings volume Human Factors in Transportation

Explore the complete digital volume

Human Factors in Transportation

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-36-0

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →