Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

The Comparative Hazards and Benefits of EFBs and Paper Documents in the Cockpit 2016 - 2022

1 California State University, Long Beach, Long Beach, CA 90840, USA ↗
2 San Jose State University Research Foundation and NASA Ames Research Center, San Jose, CA 95112, USA ↗ ↗
Publication year
Digital Object Identifier10.54941/ahfe1004278

Abstract

In aviation, the Pilot oversees the gathering all material required to be present in the cockpit. These materials are referred to as a “flight bag” and include performance calculations, charts, and other documents critical to the flight. Due to the volume of documents which can weigh up to 35-pounds, many pilots utilize electronic flight bags (EFBs). A previous archival study from 1995 through 2015 found significant human factors issues with EFBs that include a lack of training, inhibited access to information, and distraction/workload (Sweet & Strybel, 2016). Today, companies such as Airbus have made the use of EFBs as part of their standard operating procedures. The present archival study compared the hazards and benefits of carrying EFBs to the hazards and benefits of carrying paper documents in the cockpit utilizing the NASA Aviation Safety Reporting System (ASRS) from 2016 through 2022 because of the increased use of EFBs since 2016. We found several human factors issues with EFBs such as accessing information, missing or incorrect information in databases, and limitations of the physical EFB apparatus and its auxiliaries.

INTRODUCTION

In aviation, the Pilot in Command (PIC) is responsible for having all required materials such as performance calculations and charts, to be present in the cockpit. These materials are referred to as the “flight bag” (Fittzsimmons, 2002). Due to the volume of documents, many pilots utilize electronic flight bags (EFBs). Paper documents can clutter the cockpit and become a source of distraction for pilots (Haddock & Beckman, 2015). According to a study conducted in 2013, American Airlines estimated a 1.2 million-dollar savings on fuel consumption by replacing their 35-pound flight bags with Apple iPads (Stribbe, 2013). The cost-saving data triggered major airlines and the mili- tary to shift to equipping aircraft with EFBs and moving towards a “paperless cockpit”in the mid-2010s (Özkan et al., 2021). A previous archival study that examined EFB and Paper documents from 1995 through 2015 found signif- icant human factors issues with EFBs, including a lack of training, restricted access to information, distraction and workload (Sweet, 2016; Sweet, Vu, Battiste & Strybel, 2017). Sweet et al. reported a significant relationship between Device Type (paper (PFB) vs. EFB) and human factors issues. Access to information, distraction/workload and insufficient training were more frequent in the EFB reports but missing document, missing/incorrect informa- tion, outdated information and pilot error were cited more frequently in the

© 2023. Published by AHFE Open Access. All rights reserved. 580

The Comparative Hazards and Benefits of EFBs and Paper Documents 581

PFB reports. However, these results were limited by the relatively infrequent use of EFBs during this period.

With the increased use of EFBs, the present archival study compared the hazards and benefits of carrying EFBs to the hazards and benefits of car- rying paper. The present study will extend Sweets’ (2016) investigation out to 2022 where EFBs are more prevalent. The current study will also utilize pilot reports (2016-2022) from the National Aeronautics and Space Admin- istration (NASA) Aviation Safety Reporting System (ASRS). The ASRS is a database of reports voluntarily submitted by pilots after an aviation inci- dent, federal violation, or anomaly occurred. Pilots who voluntarily report their violations by submitting reports within 10 days of the incident to this database and cannot be held responsible for violations by either the National Transportation Safety Board (NTSB) or Federal Aviation Administration (FAA) (Federal Aviation Administration, 1997).

METHODOLOGY

Reports for this study were collected from a search of ASRS for the years 2017-2022. To be included in our analyses, the report had to mention the use of an electronic or paper tool, and the use of the tool had to bear some causal relationship to the eventual incident. Reports were categorized in one of two groups, electronic and paper, and each category was determined by a unique set of search terms. The search terms for electronic tools were: “EFB”, “iPad”, and “Jeppesen”. The search terms for paper tools were “Paper”, “TAC (Terminal Area Chart)”, and “Sectional Chart”. Pilots rarely used terms such as “paper plate” or “paper chart.” The search term “Jeppesen” was used as a search term for the electronic group because Jeppesen software is the leading manufacturer of EFB software for the major airlines (Özkan et al., 2021) and pilots used this term to indicate an EFB in their report.

Human Factors Issue was the primary variable used in this analysis. This is a categorical variable comprised of 10 issues: Access to Infor- mation, Climate/Environment/Automation, Display/Format Configuration, Distraction/Workload, Information Architecture, Insufficient Training, Miss- ing Document, Missing or Incorrect Information, Outdated Information, Physical Platform/Auxiliaries (Chandra and Kendra 2009; 2010). The cat- egorization described the nature of the interaction between the pilot and tool that contributed to the outcome, and they are applicable to both paper and electronic tools.

Additional variables extracted from the ASRS report were Year, Operating Regulation (Federal Aviation Regulations Part (FARs) Part 91, Part 121, and Part 135), Flight Phase, and Outcome. The range of Years included in the present study was 2016 to 2022. For comparison, data were included for the years 1995 through 2015 from Sweet (2016). The categories of the Flight Phase variable were Parked, Taxi, Takeoff, Initial Climb, Climb, Cruise, Descent, Initial Approach, Final Approach, and Landing. The Outcome variable contains the following categories: Airspace Violation, Clearance Deviation, Erroneous Performance Data, Flight Deck Event, Incorrect Crit- ical Information, Missing Critical Information, Heading, Speed or Altitude

Deviation, and Runway / Taxiway Excursion. Subject matter experts were recruited to validate the data categorization process. Two professional pilots (one Commercial / Flight Instructor, one Airline Transport) were provided with the same 8 reports (4 EFB, 4 Paper) and instructed to categorize each report based on the human factors issue categories described above. An inter- rater reliability analysis using the Kappa statistic established the reliability of the categorization scheme, as all Kappa values were greater than .7.

RESULTS

A total of 794 reports were selected for the present study, 690 reports for EFBs and 104 reports for PFBs. Chi-square tests of independence were used to determine the relationship between variables. The analyses used the for- mat of the flight bag involved (Electronic or Paper) and year the report was made to determine which of the human factors categories was more likely to occur. For the chi-square analyses below, absolute values of an adjusted stan- dardized residual value close to or above 3.0 indicated a strong contributor to the overall significant chi-square value and a residual value between 2.0 and 2.99 indicated a moderate contributor. Residual values smaller than 2.0 was assumed to not contribute to the chi square.

A Chi-square test of independence compared the frequency of each Human Factors Issue in EFB reports with Paper reports from 2016 through 2022. A significant relationship was found, χ2 (10, N = 794) = 106.79, p <.001, as shown in Figure 1. Six cells resulted in adjusted residuals greater than 2.0 (absolute). There was a higher-than-expected frequency of Paper reports compared to EFB reports in the Missing Document category (eadj = 3.2). The other four cells were associated with a greater than expected frequency of EFB reports compared to Paper reports. These occurred in Access to Information (eadj = 2.8) and Physical Platforms / Auxiliaries (eadj = 3.1). The residu- als indicated issues with missing documents were more likely to occur while using paper tools and issues accessing information and physical platforms / auxiliaries were most likely to occur while using EFBs.

Figure 1: Relative frequency of reports by human factors issue and type of report.
Original PDF Figure 1:

The Comparative Hazards and Benefits of EFBs and Paper Documents 583

Comparison Between 2016–2022 and 1995-2015 Sweet (2016) examined the NASA ASRS data base for electronic and paper flight bags for the period 1995–2015 and obtained 404 reports (220 EFB and 184 PFB). The results of the present study were compared with the data of Sweet for paper reports and EFBs separately. For paper reports, a significant relationship was found between time period and Human Factors Issue, χ2

Mathematical equation reproduced from the original PDF
Equation reproduced from the original PDF.

Four cells in the contingency table resulted in adjusted residuals greater than 3.0 (absolute) and ten cells had adjusted residuals greater than 2.0 (absolute) Eight of those cells were associated with a higher-than-expected frequency of Paper reports 1995 through 2015 compared with Paper reports 2016 through 2022. Greater numbers of reports were found in 1995–2015 for EFBs Display Format (eadj = 2.4), Information Architecture (eadj = 2.4), Outdated Information (eadj = 2.1), and Pilot Error (eadj = 2.2). Six cells were associated with a greater than expected frequency of Paper reports 2016 - 2022 compared with 1995 - 2015. These were found for Missing Document (eadj = 2.5), Missing or Incorrect Information (eadj = 3.3), and Physical Platform / Auxiliaries (eadj = 3.3). The residuals indicate issues with display format, information architecture, outdated information, and pilot error were more likely to occur between 1995 and 2015, but issues with missing docu- ments, missing or incorrect information, and physical platform / auxiliaries were more likely to occur between 2016 and 2022.

Figure 2: Relative frequency of human factors issue reports and time period for paper reports only. Note: the 1995 – 2015 data are from Sweet (2016).
Original PDF Figure 2:

A similar test was run for EFBs. The frequency of Human Factors cat- egories for 1995 - 2015 was compared with Human Factors categories for 2016 - 2022. A significant relationship was again found, χ2 (10,

N = 901) = 224.00, p <.001, as shown in Figure 3. Sixteen cells pro- duced residuals greater than 2.0 (absolute). Twelve of these had adjusted residuals greater than 3.0 (absolute). Ten cells had a higher-than-expected frequency of EFB reports in 1995- 2015 compared with 2016 - 2022: Cli- mate / Environment (eadj = 2.7), Display Format (eadj = 3.3), Distraction / Workload (eadj = 3.2), Insufficient Training (eadj = 4.4), and Missing Doc- ument (eadj = 4.1). The other 6 cells were associated with a greater number of reports in 2016 - 2022: Access to Information (eadj = 2.0), Missing or Incorrect Information (eadj = 3.2), and Pilot Error (eadj = 3.7). Note that for EFB reports in the period 1995 - 2015 there were no reports of Pilot Error. This may be due to fewer number of reports collected for EFBs in 1995- 2015 compared with - 2016-2022. The residuals indicate issues with climate / envi- ronment, display format, distraction / workload, insufficient training, and missing documents were more likely to occur between 1995 and 2015. The residuals also indicated issues with access to information, missing or incor- rect information, and pilot error were more likely to occur between 2016 and 2022.

Figure 3: Relative frequency of human factors issue and time period for EFB reports only. Note: the 1995 – 2015 data are from Sweet (2016).
Original PDF Figure 3:

Operating FAR and Flight Phase A Chi-Squared test of independence between Operating FARs (Part 91, Part 121, and Part 135) and Type of Report (EFB vs Paper) showed a significant relationship, χ2 (2, N = 796) = 6.20, p <.001). However, all residuals were less than 2.0. A Chi-Squared test of independence comparing the Flight Phase with Type of Report was significant χ2 (9, N = 796) = 33.68, p <.001).

Two cells out of 20 in the contingency table resulted in adjusted residu- als greater than about 3.0 (absolute). A higher-than-expected frequency of Paper reports compared to EFB reports were found for the Parked flight

The Comparative Hazards and Benefits of EFBs and Paper Documents 585

phase (eadj = 4.3). The residuals indicate that reported events involving a Paper tool were more likely to occur when the aircraft was parked.

DISCUSSION

The objective of the present study was to examine the transformation of suc- cesses and hazards of EFBs since their mass implementation in the mid-2010s. The archival study assessed the different types of issues reported by pilots when using an EFB or related electronic tool compared to equivalent paper tools. These were subsequently compared for time frames before and after EFBs became the standard in the cockpit. In the present study, data were collected from the ASRS for the period 2016-2022. These were compared to the results of a previous study utilizing the ASRS database for the period 1995 - 2015 (Sweet, 2016). The primary variable considered in this study was the human factors issues representing the nature of the error committed by the pilots while interacting with the tool.

While findings of this study highlight benefits of utilizing EFBs in the cock- pit such as less issues with Missing Documents, the study also reveals current EFB issues that need to be resolved. The largest issues with EFBs are access to needed documents and information, and problems with the physical iPads or charging cables themselves and because some EFBs do not have all the needed information on them. Many reports found that the iPad operating systems were not updated to the most recent software update and that caused the iPad to be unable to turn on or to spontaneously shut down. Issues were found with the iPad mounts, iPad charging cables breaking, iPads unable to hold a charge, iPads losing charge, and iPads overheating and shutting down during critical phases of flight. These issues led to pilots spending more time head-down, trying to fix the issues with their EFBs which led to problems during all phases of flight. The findings of the present study are discussed in greater detail below.

A previous archival study discussed the hazards and benefits of implement- ing EFB tools in the cockpit compared to paper tools (Sweet, 2016). The previous study documented issues that were evolving during the mass tran- sition to the “paperless cockpit” during the mid-2010s. The results of that study found that many pilot issues with EFBs came from insufficient train- ing. This was to be expected because the technology was new to pilots who completed their training and logged thousands of flight hours utilizing paper tools. The results of the present study found that the issue of insufficient training is no longer the primary concern of EFB use in the cockpit. Pilots now have enough time to train and adapt to the use of EFBs; consequently, EFBs are now the standard tool in the cockpit. Presently, new pilots have completed their training solely utilizing EFBs. The results of this study also found there was an increase in Pilot Error between 2016 and 2022 which is likely due to the more widespread use of EFBs and fewer reports collected on EFBs between 1995 and 2015. It is important to note that reports on paper tools were still analyzed between 2016 and 2022 because paper tools are used as a backup for airline pilots in commercial aviation and paper tools are still regularly used by general aviation pilots.

One possible explanation of these results is that pilots tend to become more preoccupied with their EFBs compared to paper tools. This preoccupation occurred for a multitude of reasons and led to an increased number of errors from prolonged head-down time. Moreover, many reports found that these prolonged periods of head-down time occurred during the critical phases of flight: taxi, takeoff, and landing (Federal Aviation Administration, 2014). Although non-significant, it is important to note that many of the reports were for taxi phase of flight.

During taxi, many pilots reported issues with accessing information, pro- gramming their route, and attempting to troubleshoot EFB. Enduring a prolonged period of head-down time during taxi can lead to many signifi- cant outcomes such as clearance deviations, taxiway incursions, accidentally taxiing onto an active runway, or hitting a parked aircraft. EFBs showed a statistically increased number of Clearance Deviations, and many of these deviations occurred during the taxi. Standard procedure for pilots during the taxi phase of flight involves practicing a “sterile cockpit,” that is, keeping distractions to a minimum and only having conversations that are necessary to the flight. This ensures that the proper amount of attention is dedicated to taxing (Wiener, 1985). Time spent head-down focusing on EFBs creates conflicts with the “sterile cockpit” environment.

Another issue that increases head-down time occurred when pilots could not access information efficiently. Reports in our study showed that pilots were more likely to struggle with accessing information on EFBs and this became a larger issue compared to the reports between 1995 and 2015 (Sweet, 2016). Pilots were forced to spend time looking at their EFBs while trying to fix issues such as the device freezing, starting software updated at an inconvenient time, or unintentionally powering off. These issues can cause the pilot to become lost, violate airspace, or lose control of the aircraft among other potential issues.

Missing or incorrect information is another issue that has been found to produce more head-down time in the cockpit. Pilots reported missing or incorrect information that included incorrect frequencies and incorrect pro- cedures on approach plates. Therefore, pilots spend more time searching for correct frequencies instead of focusing on priorities critical to flying the air- craft. Incorrect procedures on approach plates led to pilots violating a Federal Aviation Regulation (FAR), deviating from local procedures and interrupting traffic spacing, or violating airspace.

The results of the present study brought to light many current issues pilots are experiencing with the Physical Platform / Auxiliaries. One of the main issues reported by pilots were issues with the EFB mount. It is a common prac- tice for pilots to mount their EFBs or iPads in the cockpit, so the EFB is within their line-of-site while they are visually looking outside of the aircraft. Pilots reported mounts falling during flight and mounts blocking instruments and flight controls. Mounts falling during flight could inadvertently activate but- tons, manipulate flight controls, and generally, distract the pilot. EFB mounts restricting the pilot’s ability to view all instruments and properly operate flight controls may result in the pilot losing track of their present course.

The Comparative Hazards and Benefits of EFBs and Paper Documents 587

Presently, there is no current standard EFB mount used across Part 121, Part 135, or Part 91 Operations.

Finally, pilots encountered problems with EFB charging cables, leading to restrict the pilot’s access to flight controls. In addition, the charging cables often did not work in the cockpit. This could cause the EFBs to lose charge and power-down during a flight. If this occurs, the pilot loses access to much of the information necessary to complete the flight. The pilot may have access to paper tools as a backup or rely on radio navigation to retrieve informa- tion from Flight Service Stations, but this can unnecessarily increase to pilot workload.

SUMMARY AND RECOMMENDATIONS EFBs have become standard equipment in the cockpit since airlines prioritized the financial benefits and proven benefits to pilots compared to paper tools in the mid-2010s. This archival study revealed some of the current challenges with EFBs that need to be solved. The major issues with EFBs found in this study are problems accessing information, missing or incorrect information in the EFBs, and problems with physical EFBs themselves.

ACKNOWLEDGMENT

The authors would like to acknowledge pilots Taha Yazdandoust and Carlo Espinosa for volunteering their time as aviation subject matter experts.

REFERENCES

Chandra, D. C., & Kendra, A. (2009). Review of Safety Reports Involving Electronic Flight Bags. In Proceedings of the 15th International Symposium on Aviation Psychology. Dayton, OH: Wright State University. Chandra, D. C., & Kendra, A. (2010). Review of Safety Reports Involving Electronic

Flight Bags. In Proceedings of the 15th International Symposium on Aviation Psychology. Dayton, OH: Wright State University. Electronic Flight Bag, the new standard. (2021, February 18). Airbus Aircraft. https://aircraft.airbus.com/en/newsroom/news/2021-02-electronic-fligh t-bag-the-new-standard Federal Aviation Administration. (2014). Guidelines for the Certifica- tion, Airworthiness, and Operational Use of Electronic Flight Bags. In https://www.faa.gov/documentLibrary/media/Advisory_Circular/AC_120- 76C.pdf. Federal Aviation Administration. (2017). AC 120-76D - Authorization for Use of Electronic Flight Bags. https://www.faa.gov/regulations_policies.gov Federal Aviation Administration. (2022). Federal Aviation Administration Regula-

tions. https://www.faa.gov/regulations_policies/faa_regulations Fitzsimmons, F. S. & Air Force Academy Colorado Springs Co Inst For Information Technology Applications. (2002, August 1). The Electronic Flight Bag: A Multi- Function Tool for the Modern Cockpit. DTIC. https://apps.dtic.mil/sti/citations/ ADA455315 Haddock, K. N., & Beckman, W. S. (2015). The Effect of Electronic Flight Bag Use

on Pilot Performance during an Instrument Approach. Collegiate Aviation Review International, 33(1). https://doi.org/10.22488/okstate.18.100460

Özkan, N. F., Aksoy, E., & Sensoy, G. (2021). EVALUATION OF JEPPESEN AND GARMIN ELECTRONIC FLIGHT BAGS (Efbs) APPLICATIONS IN TERMS OF COGNITIVE WORKLOAD AND AVAILABILITY. DergiPark (Istanbul University). https://dergipark.org.tr/tr/download/article-file/1787377 Stribbe, M. (2013, May 5). American Airlines Pilots Lose 40lb with Apple Ipad

Electronic Flight Bag. Forbes. https://www.forbes.com/sites/matthewstibbe/2013/ 05/03/american airlines-pilots-lose-40lb-with-apple-ipad-electronic-flight-bag Sweet, J. (2016). The comparative benefits and hazards of EFBs and paper documents in the cockpit. California State University, Long Beach. Sweet, J., Vu, K-P. L., Battiste, V. & Strybel, T. Z. (2017). The comparative benefits

and hazards of EFBs and paper documents in the cockpit (Abstract). Proceedings of the 42nd Human Factors and Ergonomics Society Meeting, 99–100. Wiener, E. L. (1985). Beyond the Sterile Cockpit. Human Factors, 27(1), 75–90.

https://doi.org/10.1177/001872088502700107

Cover of the proceedings volume Human-Centered Design and User Experience

Explore the complete digital volume

Human-Centered Design and User Experience

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-90-2

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →