Thomas Strybel, Vernol Battiste, Kim-Phuong L. Vu, Panadda Marayong, Stacey Ahuja, Maegan Schmitz, Justin Cheung, Chloe Culver, Andrew Alfaroarevalo, Praveen Shankar (2024). Evaluation of Voice vs. Text Communication Modes in Simulated UAM Operations. In: Neuroergonomics and Cognitive Engineering. AHFE Open Access, vol. 126, pp. 113–122. https://doi.org/10.54941/ahfe1004744
Urban Air Mobility (UAM) is a system that is expected to operate within and around metropolitan environments, utilizing electric, vertical takeoff and landing (e-VTOL) aircraft, to create on-demand, highly automated passenger and cargo-carrying air transportation services. Many stakeholders are developing such systems, although several barriers to UAM operation remain. Two barriers being addressed in our simulation facility are pilot training and air traffic management operations. Although the UAM industry is focused on autonomous operations, the initial UAM operations will have ground or onboard pilots, yet the forecasted pilot shortage will be problematic. Completely autonomous systems will result from an evolution, of pilot-based operations, to onboard operators flying with automated assistance and finally completely autonomous vehicles. We are currently investigating and evaluating concepts for simplified vehicle operations and air traffic management using a virtual UAM eVTOL vehicle in a CAVE virtual environment. In the first test of the UAM vehicle (e.g., Strybel et al., 2021) pilots and reported that the simulation was sufficiently realistic for tests of UAM operations. Here we report on a subsequent investigation of communication modes for pilots flying UAM routes over the San Francisco metropolitan area. The routes consisted of stops at six vertiports, either at airports or other locations, for picking up/dropping off passengers. UAM pilots communicated with air traffic control and vertiport managers using either voice or text messaging. Voice communications were consistent with current day air traffic control operations. Text communications were via a custom message pad application that enabled standard messages to/from ATC (requests and responses) via touch input. Six certified pilots flew two routes using each mode of communication. We evaluated the impacts of these communication modes on pilot performance (flight time and communication latency) and subjective responses (workload and subjective feedback). Feedback regarding the messaging application and simulation facilities were also collected and will be described in the presentation.
Urban Air Mobility (UAM) is a conceptual transportation system that is expected to facilitate on-demand air transportation services for passengers and cargo in urban (and surrounding) areas. Development of the UAM system has been ongoing for several years. It is viewed by some as a novel trans- portation system for passengers and freight that should relieve urban traffic congestion, while being profitable for UAM vehicle and system developers. There is much optimism associated with this concept, but it is also acknowl- edged there are many barriers that must be addressed before the societal and commercial benefits can be realized.
UAM flight operations initially will require onboard, certified pilots flying routes in compliance with current airspace regulations. Organizations such as Boeing and NASA expect that UAM will transition in the midterm (roughly 2032) to a few remotely piloted vehicles supervised by vehicle operators, com- bined with special flight procedures to enable automated vertical guidance to and from vertiports. Eventually this will evolve into a fully mature UAM sys- tem having many pilotless vehicles supervised by multiple vehicle operators. Increased traffic density will be possible with automated flight operations, and special airspace concepts such as UAM corridors that would be super- vised by traffic managers. In addition, flight planning and communications will be automated.
As noted, in the near-term, implementations of UAM services will be oper- ated with onboard, certified UAM pilots flying either VFR or IFR rules. This is a reasonable starting point, except that it creates significant demand for new UAM pilots, which may be problematic due to the current shortage of qualified pilots, resulting from fleet growth, retirement, and attrition (Bat- tiste et al., 2023). Progression in UAM development from current day to midterm operations means that pilots will be an important element in the design of new autonomous systems. Pilots will be required to test automated systems for their suitability in different scenarios, environmental conditions and airspaces, and will serve as a fail-safe function, responding to automation failures. In summary, although the maximum benefits of UAM operations will be realized with autonomous vehicles, onboard pilots and operators will be essential to the design of these UAM systems (Ahuja et al., 2023). The increased demand for trained operators makes this a significant barrier to the success of UAM in the future.
One solution to this barrier is known as simplified vehicle operations (SVO; e.g., Lombaerts et al., 2020). SVO make the vehicle easier to operate and reduces the complexity of flight-system interfaces, which should reduce training time and time to certification. Battiste et al. (2023) suggested that a stability augmentation system coupled with a pathway-in–the sky show- ing where the vehicle should be flown should reduce the complexity of the manual control needed by the flight task and produce a smooth ride for UAM passengers. SVO can be achieved through a direct mapping of inceptors inputs to task goals – climb, descend, move backwards, etc.
SVO also includes the use of advanced automation for mission manage- ment, flightpath management and tactical operations (Wing et al., 2020). Simplifying additional tasks, such as separation management, navigation, and communication, requires research and testing of novel concepts for achieving these tasks and simulation tools for evaluating vehicle interface designs and airspace operation concepts. The BeachCAVE at California State University Long Beach has developed a virtual UAM tool for these purposes. This system is a VisCubeTM M4 CAVE Immersive 3D Display (Visbox, Inc.) that includes a four-wall projection system, an eight-camera advanced real-time full body motion capture system, surround sound, and a graphics workstation. The CAVE system (as opposed to a head mounted display) is appropriate for applications where a wide field of view facilitates a greater sense of immersion in the virtual environment while still allowing participants to interact with physical controls and experience the airspace as if they were
Evaluation of Voice vs. Text Communication Modes in Simulated UAM Operations 115
sitting in a real cockpit. Moreover, a virtual UAM simulator allows greater flexibility in the design of the vehicle, cockpit interface and airspace, which enables testing of novel concepts of automation.
Previously, two simulation tests of this system have been reported. The initial test of this simulator (Strybel et al., 2022; Ahuja et al., 2023) was conducted using a short flight from the Ferry Building in downtown San Francisco to the San Francisco Airport and back. Both certified pilots and non-pilots were tested. Both groups found the vehicle relatively easy to fly and thought that this system was realistic enough to provide adequate tests of UAM cockpit interfaces and airspace concepts of operation. Subsequently, Haneji et al. (2023) examined the use of tactile alerts for UAM vehicles that were deviating from and automated flight path. In the present paper we report on another test of the UAM virtual simulator, that incorporated a more difficult flight task involving stops at six different vertiports, and controller-initiated clearances requiring pilot acknowledgements during the flight. Two modes of communication were examined, voice and text. For each communication mode we measured pilot workload, situation aware- ness and performance measures to compare the effectiveness of these modes of communication.
METHOD
Participants Six participants were tested, all were certified pilots. Two were Certified Flight Instructors II, and three were instrument rated. Pilots reported between 200 and 3500 hours of flight (M = 981hrs, SD = 1288 hrs), and two pilots participated in a previous UAM simulation using CAVE VR. Two partici- pants reported experience with virtual reality applications, and two reported experience with remote flight (one flew drones and one flew model aircraft). Participants were paid $50 per hour for participation. The simulation took approximately 4 hours.
Simulation Facility The virtual UAM vehicle was adapted using Blender, an open-source graphic software (The Blender Foundation), from a quadcopter base model purchased through the Unity Asset Store (Unity Technologies, Inc.) and has been cus- tomized via code to enable easier participant control of the aircraft and out-the-window views. UAM operators in the CAVE wore special glasses to facilitate viewing of the 3D simulation with head tracking to automatically adjust the environment to the operators’ view. The aircraft can be flown in autonomous or manual mode, and dimensions of the cockpit display were set to conform to the point of view of a seated operator. For additional details on the development and design of the simulated UAM vehicle and test environment, see Marayong et al. (2020) and Shankar et al. (2022).
The vehicle was controlled with an integrated Attack 3 joystick (Figure 1A), which controls all flight parameters (heading, speed, altitude, etc.). The flight stick also supported adjusting the map display and switch- ing between operational modes, either Flight Mode or Ground Mode. In Flight Mode, participants used the joystick to move forward, change speed,
climb/descend and control the heading of the quadcopter. Ground Mode was used for the final approach to the landing pad; the participants could move the vehicle forward and backward, laterally, and rotate the vehicle. Moreover, turning, rotating and accelerating could be achieved at slower speeds. The joystick flight control arrangement (Figure 1) approximated SVO concepts described in Wing et al. (2020) for a single joystick based on NASA’s EZ-Fly Concept for simplifying V/TOL flight handling, with some exceptions.
The cockpit display (Figure 1) also contained an integrated display of current and assigned flight parameters and a moving map showing vehicle position over the city of San Francisco.
Figure 1: Simplified flight controls (left) and display (right) in the virtual cockpit: the joystick controls vehicle heading, speed, altitude, flight mode, and map zoom; the cockpit interface displays a moving map, heading, speed, altitude, and flight mode.Figure 2: Message application used for sending and receiving text messages from air traffic and vertiport controllers. The small buttons labelled with V’s brought up a map of the specific route leg.
Communication by text was achieved with a messaging application that was developed for pilots. This application was run on a Samsung tablet that was strapped to the leg of the pilot, and functioned as a knee board. An example of the interface is shown in Figure 2. The leftmost column of but- tons consists of the vertiports at which the pilots would be flying to. When
Evaluation of Voice vs. Text Communication Modes in Simulated UAM Operations 117
the vertiport was selected, a set of canned messages were presented for that vertiport in the “Outgoing to ATC” window. The pilot could send a canned message by selecting a message and then clicking the send button. Messages arriving from ATC were shown in the right “Incoming from ATC” window. The pilot acknowledged the incoming message by clicking either “Roger”, “Reject” or “Wilco”.
In addition to the messages, the messaging application had a set of maps that the pilot could access by clicking the map icon below the vertiport but- ton. This brought up a detailed map of the specific leg of the route, showing required communications, route change landmarks, and changes in airspace class requiring that the pilot Squawk a specific frequency. Note that the order- ing of the buttons corresponded to the order of the vertiports the pilot would access on their route.
The application was also available in the voice communication mode, but messages were sent verbally. Pilots could see the text messages, so that they would know what message was required with each leg of the route.
Scenario Pilots flew circular routes around the San Francisco metropolitan area, stop- ping at six vertiports. Vertiports were located at either existing airports (SFO, OAK, HWD) or at new vertiports created for this simulation (Ferry Building, Daly City Transit Station, Presidio Park), shown in Figure 3. Each pilot flew routes in clockwise and counter-clockwise directions starting and ending at the Ferry Building Vertiport. Within each leg, pilots would receive flight plan changes from ATC, that consisted of either altitude or speed changes. Pilots were instructed to acknowledge the clearance and then comply with it.
Figure 3: Flights were initiated at the Ferry building in downtown San Francisco and continued around the San Francisco/Oakland area (either clockwise or counterclock- wise) with six vertiport stops: FERRY (FER), Oakland Airport (OAK), Hayward Airport (HWD), San Francisco Airport (SFO), Daly City (DLY) and Presidio (PDO).
Procedure Pilots flew two routes, one clockwise and one counter-clockwise, using voice communications or text communications, for a total of four routes. The communication mode and order of direction were counterbalanced. At each vertiport landing, pilots completed an ATWIT measure of workload (Stein, 985), in which pilots rated their workload on a 1–10 scale (1 = low, 10 = high). At the end of each communication condition, pilots completed the NASA TLX and Cooper-Harper workload instruments, and the SART questionnaire (Taylor, 1990) to measure situation awareness.
Upon arriving at the BeachCAVE lab, pilots were briefed on the UAM concept, and the flight controls for the UAM vehicle. This was followed by practice using the controls and displays in the UAM vehicle and then instructions as to the routes and communication protocols. At the end of the simulation, a custom survey was administered, and this was followed by a debriefing session.
RESULTS
We examined workload, situation awareness and performance on flight time and response times to ATC instructions, using repeated measures analyses of variance (ANOVAs). Separate ANOVAs were run for the clockwise and counter-clockwise conditions in some cases because of the differences in flight demands for the same route leg. For all analyses we adopted a value of p < .10 for statistical significance due to the small sample size.
Workload As shown in Table 1, workload scores were moderate (Grier, 2015). A repeated measures ANOVA on these scores showed all main effects and inter- actions to be nonsignificant. Overall SART measures of situation awareness were computed based on Taylor (1990); the mean SART score for each con- dition is shown in Table 1. The main effect of communication was significant (F(1,5) = 5.65; p = 0.06), with the overall SART score being higher for voice communications (M = 27.2; SEM = 2.2) compared with text com- munications (M = 22.8; SEM = 2.0). The main effect of direction was also significant (F(1,5) = 7.19; p = 0.04), with SART ratings higher for counter- clockwise flights (M = 27.2; SEM = 1.97) compared with clockwise flights (M = 22.8; SEM = 2.35). The interaction between communication mode and flight direction was not significant.
Table 1. Mean workload (NASA TLX) and situation awareness (SART) scores.
Table available in the original PDF paper.
Communication Mode Flight Direction Mean (SEM) TLX Mean (SEM) SART Text CCW 43.2 (4.1) 24.2 (2.0) CW 43.6 (4.0) 22.0 (2.6) Voice CCW 45.8 (6.0) 30.2 (1.2) CW 46.8 (4.0) 23.8 (2.3)
Evaluation of Voice vs. Text Communication Modes in Simulated UAM Operations 119
ATWIT workload ratings were obtained at each vertiport landing, result- ing in six ratings per route. Separate ANOVAs were run for each flight direction because the direction of flight produced differences in the length, route complexity and communication requirements. As shown in Figure 4, the interaction of communication mode and flight leg was significant for Counter-Clockwise flights (F(1, 25) = 3.92; p = 0.009) but not for Clockwise flights (F(1, 25) = 0.67; p = 0.65). From Figure 4, it appears that ATWIT workload was higher for the FER-PDO and PDO-DLY legs, but all post-hoc comparisons were nonsignificant.
Figure 4: Mean ATWIT ratings for each flight leg. Left: counter-clockwise flights; and right: clockwise flights.
Performance We examined performance for each communication mode in terms of the number of times the pilot failed to respond to ATC and the time to respond to ATC commands. Whereas only 3% unacknowledged responses were observed in the voice mode, pilots failed to respond to text clearances 30% of the time for counter-clockwise flights, and 45% of the time for clockwise flights. Moreover, the time to acknowledge the ATC instructions was on aver- age 3 times longer in the text mode (M = 14.5 s; SEM = 4.0 s), compared to voice communications (M = 1.5 s; SEM = 0.11 s).
Figure 5: Mean flight time for each route leg. Left: counter-clockwise route; right: clockwise route.
Repeated measures ANOVAs were performed on total flight time for each flight direction with the variables of communication mode, route leg, and direction as factors. For each direction, only flight leg was significant (CCW: F(5, 25) = 6.12; p = 0.0008; CW: F(5, 25) = 37.99; p <0.0001). For both directions, the longest flights occurred between SFO and HWD airports as shown in Figure 5.
Pilot Feedback Pilot subjective responses to questions about the simulation are shown in Table 2. Pilots used a seven-point scale (1 = “Extremely Unrealistic”, 7 = “Extremely Realistic”) to rate six questions on the realism of the UAM vehicle, cockpit interface, communications and scenario. In addition, one question asked about the ease of flying (1 = “Extremely Difficult”, 7 = “Extremely Easy”). The lowest scores were found for the realism of the cockpit interface and airspace. Pilots were concerned about the lack of infor- mation regarding vehicle status, and the novelty of controlling the vehicle with a single joystick. With respect to the airspace, pilots suggested that adding traffic, weather, and winds would improve the fidelity of the simu- lation environment. Note that pilots rated ease of flying very high (median rating = 6.0).
DISCUSSION
The main purpose of this simulation was to evaluate two modes of pilot- ATC communications, either voice or text. Voice communications were more natural to our pilots, as they were highly trained and experienced with this communication mode. An advantage of text messaging is that it provides a history of previous communications that pilots can refer to if they missed or forgot a specific instruction, but pilots were not familiar with this mode of communication.
Table 2. Summary of subjective ratings at the end of the simulation.
Moreover, pilots failed to acknowledge ATC text messages nearly 50% of the time. However, this high rate could have been a result of the design of the messaging application. If the pilot brought up a map of the current route leg, a message from ATC would not be seen until the specific communication page was made visible again, creating both failures to acknowledge messages and long response times. In future studies, we intend to solve this problem
Evaluation of Voice vs. Text Communication Modes in Simulated UAM Operations 121
by either adding an audio alert that an incoming message was received, or by displaying the message on top of the current page the pilot was looking at. Another advantage of voice communication was that pilots reported higher situation awareness. Most likely, text messaging produced more head down time, thus reducing overall situation awareness.
We did not find differences in UAM flight times between communication modes. This may be due to the simplicity of the flight controls and the lack of important variables such as traffic, weather and winds, that would increase the difficulty of the flight. We are currently updating the simulation facility to include these factors for future UAM simulations.
Finally, pilots were again positive about the ease of flying the UAM vehi- cle, suggesting that efforts to implement SVO at least with respect to vehicle handling should be an important direction for early UAM operations that will require onboard pilots.
ACKNOWLEDGEMENT
This research was supported in part by NSF Major Research Instrumentation award (Award #1626655) and San Jose State Research Foundation Grant #2116145736 (PI: Sean Laraway). The authors also would like to thank William Deabaapilux for his assistance on the project.
REFERENCES
Ahuja, S., Strybel, T. Z., Vu, K-P., L, Marayong, P., Shankar, P., & Battiste V. (2023).
Development and validation of a virtual UAM transportation system. Proceedings of the International Symposium of Aviation Psychology. Battiste, V., Strybel, T. Z. (2023). Development of Urban Air Mobility (UAM) Vehicles for Ease of Operation. In: Duffy, V. G., Krömker, H., A. Streitz, N., Konomi, S. (eds) HCI International 2023 – Late Breaking Papers. HCII 2023. Lecture Notes in Computer Science, vol. 14057. Springer, Cham. https://doi.org/10.1007/978-3- 031-48047-8_14 Grier, R. A. (2015). How high is high? A meta-analysis of NASA-TLX global work-
load scores. Proceedings of the Human Factors and Ergonomics Society 59th Annual Meeting. 1727–1731. Haneji, K., Leung, K., Tran, A., Cheung, J., Deabaapilux, W., Marayong, P.,
Vu, K., Shankar, P., Strybel, T. Z., & Battiste, V. (2023). Use of Tactile Alerts in Urban Air Mobility Vehicles. In: Gesa Praetorius, Charlott Sellberg and Riccardo Patriarca (eds) Human Factors in Transportation. AHFE (2023) Inter- national Conference. AHFE Open Access, vol. 95. AHFE International, USA. http://doi.org/10.54941/ahfe1003836 Levitt, I., Phojanamongkolkij N., Horn, A., & Witzberger, K. (2023). UAM Airspace Research Roadmap. NASA/TM-20210019876 Levitt, I., Phojanamongkolkij N., Witzberger, K., Rios, J., & Cheng, A. (2021). UAM
Airspace Research Roadmap Rev 2.0. NASA/TM-20230002647. Lombaerts, T., Kaneshige, J., & Feary, M. (2020, June 15-19). Control concepts for simplified vehicle operations of a quadrotor eVTOL vehicle. In AIAA AVIATION 2020 FORUM, virtual event. https://doi.org/10.2514/6.2020-3189 Marayong, P., Shankar, P., Wei, J., Nguyen, H., Strybel, T. Z., & Battiste, V. (2020).
Urban air mobility system testbed using CAVE virtual reality environment. 2020 IEEE Aerospace Conference, 1. doi: https://doi.org/10.1109/AERO47225.2020. 9172534.
Shankar, P., Marayong, P., Strybel, T., Battiste, V., Nguyen, H., Cheung, J., & Viramontes, J. (2022). Urban Air Mobility: Design of a Virtual Reality Testbed and Experiments for Human Factors Evaluation. ASME International Mechanical Engineering Congress and Exposition, Columbus, OH. Stein, E. S. (1985). Air Traffic Controller Workload: An Examination of Workload
Probe. DOT/FAA/CT-TN84/24. Strybel, Z., Battiste, V., Marayong, P., Shankar, P., Viramontes, J., Nguyen, H., & Cheung, J., (2022, October 10-14). Preliminary validation of a virtual UAM vehicle and simplified cockpit interface. Abstract. Human Factors & Ergonomics Society 66th International Annual Meeting, Atlanta, GA, United States. Taylor, R. (1990). Situation awareness rating technique (SART): The development of
a tool for aircrew systems design. In Situation awareness in aerospace operations (AGARD-CP-478). NATO-AGARD, Neuilly Sur Seine, France, pp. 3/1–3/17. The Boeing Company (2023). Concept of Operations for Uncrewed Urban Air Mobility Version 2.0. Wing, D. J., Chancey, E. T., Politowicz, M. S., & Ballin, M. G. (2020). Achieving
Resilient In-Flight Performance for Advanced Air Mobility through Simplified Vehicle Operations, AIAA Aviation 2020 Forum, June 8 2020.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Neuroergonomics and Cognitive Engineering
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-02-1
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional