Mark Miller, Bettina Mrusek, Jeff Herbic, Sam Holley, Leila Halawi (2024). Implementing an AI Fatigue Risk Management System for Aviation Maintenance SMS: A Technology Enhanced Critical Process Human Factors Safety Plan. In: Human Factors in Design, Engineering, and Computing. AHFE Open Access, vol. 159, pp. 1–11. https://doi.org/10.54941/ahfe1005562
Commercial aviation maintenance is a safety-critical process that requires adherence to maintenance procedures. Unfortunately, when this maintenance process fails due to human error, it can come as a costly event and potentially keep an aircraft out of revenue service. The researchers have advocated using some form of human factors risk management safety reporting system within a Safety Management System (SMS) framework for airline maintenance to mitigate human error. But the current shortages of aviation maintenance technicians (AMTs) create a fatigue inducive environment that calls for better fatigue mitigation. What is the point of having a good SMS and human factors safety reporting system if AMTs are often exposed to hazardous fatigue levels? Strategically, both a strong human factors risk management safety reporting system and a proactive fatigue risk management (FRM) system would have to work complementary with each other to keep the critical maintenance process safe within the SMS framework. With AMT fatigue in United States (US) identified as a problem, the researchers then analyzed the current Federal Aviation Administration (FAA) FRM system (AC 120-115). From the analysis, the researchers propose a solution in the form of an AI FRM system. To accomplish the proposed AI FRM system design, a research-supported AI integration system framework called CHAAIS was adopted. The proposed AI FRM system complementing a human factors risk management reporting system in SMS could greatly enhance airline safety.
Fatigue in commercial aviation has been identified by aviation safety experts in the U.S. for over three decades. The National Transportation Safety Board (NTSB) has made a fatigue recommendation on its Most Wanted List of safety improvements since 1990 and recommended ‘reduce fatigue-related accidents’ for 2019–2020 (NTSB, 2023). Most of the aviation industry’s emphasis on fatigue mitigation spotlights pilots in the form of improved FAA crew rest regulations and FRM programs. However, not enough has
been done to reduce the fatigue issues in airline maintenance. While modern commercial flight in the U.S. has an enviable safety record, the fact remains that an airline flight in the 21st century still involves a critical flight safety process where a small mistake could lead to deadly circumstances. Although often overlooked by the traveling public, the AMT plays a significant role in the critical flight safety process within the Federal Aviation Regulations Part 121 subpart L (commercial airline maintenance) by maintaining safe aircraft flight standards. While the AMTs are professionals working behind the scenes of a burgeoning industry, the COVID 2019 pandemic forced many AMTs into retirement and replacements have been slow to materialize. A projected shortage over the next 20 years in North America is 132,000 AMTs by 2040 (Boeing, 2021). The experience level of AMTs coming into the industry is meagre. Maintenance is now challenged to do more work with fewer AMT personnel, inviting fatigue as a serious threat.
Dangers of Aviation Maintenance Issues: Alaskan Airlines Flight 1282 To emphasize how dangerous a failure in the maintenance procedure can be in the critical process of commercial aviation, the recent example of Alaskan Airlines Flight 1282 is evidence. The mistake of not installing the four bolts to secure the door plug caused the 737 MAX9 aircraft to experience an emergency decompression (NTSB, 2024) with no fatalities. It should be noted that a mother sitting in a row directly behind the missing door plug had to restrain her son from being sucked out of the aircraft during the decompression. This accident was caused by a failure in the maintenance procedure in the aircraft delivery from Boeing to Alaskan Airlines and revealed the dangers of poor inspections.
ASRS FAR Part 121 Maintenance Reports on Fatigue in Maintenance Current dangers of AMT fatigue are best underscored by analyzing the number of ASRS (Aviation Safety Reporting System) incidents in U.S. commercial aviation maintenance that were attributed to fatigue from 2010 to 2020. The ASRS data revealed 828 fatigue-related incidents over those 10 years. Of this total, 163 of the 828 incidents were attributed to fatigue as the primary cause of the incident according to the FAA analysis team. In this data set, 163/828 = 0.196 or 19.6% (20% rounded) of the incidents were caused directly by AMT fatigue (Herbic, 2021). This trend of 20% for fatigue-related events directly causing maintenance incidents shows that AMT fatigue is profound, especially since these were voluntarily reported incidents. This voluntary incident data is just a small percentage of the related incidents occurring and should alarm safety experts.
The Problem of Fatigue in Aviation Maintenance Moving Forward Aviation accident report data has revealed that 80% of commercial aviation accidents in the U.S. were caused by human error. Of that 80%, 10% were caused by maintenance-related mistakes (Rankin, 2007). Of that 10% caused by human maintenance-related human error, 38% of the time maintenance error was related to using the procedure (Chapparo, 2002). Other human
Implementing an AI Fatigue Risk Management System 3
factors that contribute to human errors in maintenance have been identified in Transport Canada’s maintenance Dirty Dozen: poor communication, distraction, poor resources, complacency, lack of knowledge, poor teamwork, poor situational awareness, poor assertiveness, norms, pressure, stress, and fatigue (Dupont, 1993). AMTs should be trained to deal with the Dirty Dozen, however, the fatigue factor and the dangers that it imposes in conjunction with accomplishing the maintenance procedure should be prioritized. From an SMS perspective, the Dirty Dozen clearly would advocate for an effective maintenance human factors risk management reporting system like the REPAIRER (Miller and Mrusek, 2018) that is centered around maintenance procedures to identify, assess, and mitigate the human factors hazards. However, dangerous AMT fatigue levels would require a more effective SMS FRM system for aviation maintenance to have optimal safety coverage.
Extreme Fatigue Issues Facing the AMT Unlike pilots, the AMT has little government rest regulation to protect them from overworking and being thrust into cumulative fatigue. FAA regulation allows the AMT to work up to seven consecutive days before requiring a day off, although the company or union can mandate a version of hours-of-service limits. This lack of government regulation allows the AMT to work long hours to meet the demands of airline maintenance. Because airline operations are contingent upon quick maintenance turnaround to sustain seat and cargo revenue service, that condition forces airline maintenance operations into 24/7 status. For the AMT shift work is a reality, and night shift also means working in the window of a circadian low (from 2–5 in the morning) when the average human being is inclined to sleep. Not only is the night shift difficult for the AMT to manage fatigue, it also invites a potential for sleep disorders since it is difficult for the AMT to sleep during the day (Cleveland Clinic, 2021). The continuous pressures on airline maintenance to keep the aircraft available for revenue service also produce significant stress for the AMT. Stress-caused fatigue is compounded in that it may include extreme sleepiness, brain fog, loss of well-being and emotional exhaustion (Kocalevent et al., 2011). Stress can accelerate the human brain and nervous system into high gear and over time can cause chronic fatigue and health issues. The fact current AMT personnel shortages exacerbate these long-standing fatigue issues makes it important to manage the fatigue proactively.
Difficulty in Applying an FAA SMS FRM System to Maintenance With AMT fatigue being a serious problem, it is important to analyze the current recommended solution in the form of the FRM program represented by FAA AC 120–115 (FAA, 2016). AC 120–115 points out all the different aspects of fatigue in aviation maintenance while educating the AMT on managing their fatigue. However, AC 120–115 falls short in how to measure each AMT’s daily fatigue accurately. Because each AMT differs in relation to the multitude of fatigue variables it becomes difficult to form an accurate baseline fatigue assessment of each AMT. Accurate fatigue assessment of each
AMT would necessarily need to consider a myriad of variables infused into each worker’s life before they go to work each day including Personal and Work Factors, as portrayed in Figure 1. The fatigue levels of each AMT start on the right side of Figure 1 with key biological factors that are mostly portrayed as sleep-related but could also include diet and health factors such as illness. These biological factors have changed since the 2019 pandemic in that society now recognizes that mental health affects upwards of 40% of the workforce. This suggests that effective medication might be used for depression or other mental health conditions that could contribute to fatigue. Different sleep disorders contribute to fatigue. Multiple individual biological factors potentially affecting an AMT need to be accounted for. If the biological factors are not demanding enough, the diagram in Figure 1 shows how the AMT’s life at home can also contribute to their fatigue. Family and social life, and socioeconomic factors of providing for a family contribute to AMT fatigue. Just commuting to work, along with the stress levels at home and work, contribute to fatigue. The challenge is to accurately measure the Personal Factors in Figure 1 that the AMT is taking to work and then manage them with the Work Factors.
Figure 1: Factors contributing to employee fatigue, modified by Miller 2024. (Adapted from the Australian National Transport Commission, 2004).
Shortfalls of the Current FAA Commercial Maintenance FRM system While the commercial aviation environment and effects on the AMT are changing, the FRM system recommended for aviation maintenance operations in the U.S. by the FAA in AC 120–115 needs to catch up (FAA 2016). Although it is supported by strong research (Hobbs et al., 2011) and clearly shows the relationship between the AMT, maintenance work, the dangers of fatigue, three areas of fatigue mitigation, and a path
Implementing an AI Fatigue Risk Management System 5
to SMS integration, it is still a recommendation and not mandatory for airline maintenance workers. Because they are guidelines on applying FRM principles in terms of SMS along with the gained benefits, the guidance falls short of being standard and establishing an accurate FRM system that could be applied more widely to U.S. airline maintenance. The International Civil Aviation Organization (ICAO) provides a prescriptive approach that uses regulations to manage fatigue hazards. ICAO considers an airline FRM system linked to SMS, like the one recommended by the FAA in AC 120–115, a performance-based approach. Due to the cost and complexity of implementing a FRM system, ICAO clearly states that FRM are not for all airlines (ICAO, 2016). However, the preponderance of fatigue analyses points to U.S. airlines needing a more standardized performance-based maintenance FRM system.
The Difficulty of Implementing FRM Systems from FAA AC 120–115 Using the FAA AC 120–115 FRM system becomes difficult is when an AMT ends up with a concerning fatigue assessment and still needs to work. Whatever baseline FRM assessment was completed must be used to reevaluate the AMT through FRM mitigation. These mitigations are formed into three well-organized categories, as shown in Figure 2, consisting of direct individual fatigue countermeasures, work interventions, and reducing the consequences of fatigue- related mistakes (FAA, 2016). For direct individual fatigue countermeasures, the FAA first encourages that hours-of-service limits be placed on the AMT, followed by AMT fatigue education, work breaks, naps and limited use of caffein. Next the FAA proposes the mitigation area of fatigue countermeasures in the form of work interventions to address at- risk tasks while AMTs work with manageable fatigue. The FAA recommends scheduling the identified fatigue-inducive tasks to be undertaken first on a shift. As shown in Figure 2, the FAA sometimes proposes modifying the task by requiring supervision, teamwork, task rotation, checklists, and requiring experienced AMTs to be involved. Other work interventions call for shift handover briefings, breaking down complex work, adding inspections and additional self-checks. As a final measure to reduce fatigue in maintenance operations, the FAA proposes reducing the actual consequential outcomes of fatigue-induced maintenance errors. This is accomplished by minimizing the maintenance work for fatigued AMTs with progressive restrictions. While the concept of an FRM SMS is noble and will reduce the fatigue in maintenance, a flaw is that a manager or team would be required to run such a complex system and still complete their other maintenance-related work. These issues, mixed with the demands of airline maintenance, make a FRM SMS type of program costly, time consuming, difficult to implement, and hard to maintain. With fatigue becoming such a growing threat to airline maintenance the possible solution of a performance-based FRM system could lie in integrating the FAA AC 120–115 recommendations into an AI-based FRM system that could become a standard for the U.S. aviation industry.
Creating an AI-Based Standard FRM System for Aviation Maintenance To create a hypothetical AI-integrated FRM system model for aviation maintenance based on the FAA AC 120–115, it would first be important to have a current working model where AI has already been successfully implemented to address a major complex problem and has improved AI teaming to enhance decision making significantly. To this end, the authors have opted to use a working model known as CHAAIS. CHAAIS represents the Climate-Focused Human-Machine Teaming and Assurance in the AI Systems framework. This model was originally applied toward integrating AI into high-risk domain climate change decision-making and was demonstrated in a forestry wildfire management case study by Gladkova et al., (2023). As CHAAIS was designed to address how to integrate AI for human-machine decision-making that was more accurate and effective, it was determined that it could also be used to answer the question of how AI could be used by maintenance management to run a more accurate and effective FRM SMS system. Under the first part of the CHAAIS model in Figure 3 of ‘Addressing the Needs’, step 1 of ‘identifying current decision maker workflow and pains’ was previously accomplished by analyzing the growing AMT fatigue problem along with issues in the FAA AC 120-115 FRM guidelines. From the issues discovered in the FAA FRM system, it was determined that AI should be integrated into the FRM system to collect daily AMT fatigue risk assessment data and to reduce the dangers of the fatigued AMT by improving the fatigue mitigation process of maintenance management.
Implementing an AI Fatigue Risk Management System 7
Defining AI Portions of an FRM System (Assessment Survey) Using the CHAAIS framework portion of ‘addressing needs’ and step 2 of ‘define which parts of the process the AI is supporting’, for a new FRM system, the first area where AI human-machine teaming could make a profound difference is in improving the fatigue assessment of the AMT. By having the potential to collect multiple daily data points on each AMT, the advantage AI could bring to the individual daily fatigue assessment is it can easily account for multiple fatigue variables needing to be assessed in the form of a proactive survey. This could be accomplished by the AI agent actively asking or texting the questions and having the AMT participant rate them (NovoPsych, 2024). This survey could also be separated into areas related to fatigue, such as sleep, home, medical, time at work, nature of work, work site conditions, and stress conditions. Each category could be assessed and rated for risk exposure and severity (National Response Team, 2024). Appropriate fatigue-related scales could be integrated into the different questions such as adopting the Standford Sleepiness Scale. As an interactive AI-driven FRM survey it could readily be adjusted daily by having the AMT update only fatigue-related life changes. The biggest gain of using an AI-based FRM survey is that the data could be analysed for each AMT and collectively for the maintenance organization, creating a proactive trend data approach that would drive the FRM SMS and help management make better AI-driven human-machine decisions for individual AMTs, shift AMTs, and AMTs collectively.
Figure 3: CHAAIS framework process AI human machine teaming. (Gladkova et al., 2013).
Defining AI Portions of an FRM System (Physiological Assessment) The value of an AI FRM system would also be in collecting of physiological data from each AMT in the form of technological fatigue tests. Several technologies can detect dangerous levels of AMT fatigue that are computer- based before work or during a shift. Tests could be taken on the smartphone, laptop or tablet the AMT uses with the AI FRM system. Test data could be fed into the AI FRM system that measures the person’s speed of response or other fatigue-related measurements (Dinges & Powell, 1985; Thorne et al., 2005). Voice analysis shows promise to detect fatigue in operational personnel (Greeley et al., 2007) as do a range of alertness monitoring systems. Fatigue may be detected by monitoring eye movements, blink rate, and performance measures (Williamson and Chamberlain, 2005). Fatigue detection systems can monitor eye closures to assess drowsiness (Dinges et al., 1998). Technologies like the Alertmeter (Predictive Safety, 2024) which is based on NASA’s Psychomotor Test, could also be used to collect individual AMT alert data. This data could be merged by the AI FRM system with other physiological fatigue test data and be integrated with daily AMT survey data. The combination of physiological data and individual survey data would then determine the AMTs state of fatigue via a standardized fatigue risk assessment table. The AI FRM system would give maintenance management a more accurate rating of the AMT’s fatigue before work in terms of low, moderate, extreme, and unacceptable as shown in Figure 4.
Figure 4: Levels of AMT fatigue risk management assessment. (Adopted from FAA, 2016).
Implementing an AI Fatigue Risk Management System 9
Defining AI Portions of an FRM System (Recommended Mitigation) Even with an accurate FRM risk assessment at hand, maintenance management could still find themselves challenged to reduce the hazard of identified fatigued AMTs scheduled for work due to the complexity of each AMT’s fatigue assessment and matching that to an optimal mitigation strategy. Using the appropriate combination of mitigation techniques available in Figure 2 could become overwhelming for management with a high number of fatigued AMT’s coming to work each shift. This is where maintenance management could team with the AI/FRM system and create an optimal fatigue mitigation strategy for each fatigued AMT. Using that same accurate AMT assessment data and programming the AI FRM system to apply the different fatigue risk mitigation strategies in Figure 2 could derive the optimal mitigation for each AMT with fatigue issues. In terms of FRM this would reduce the individual AMT fatigue risk assessment values and help position AMTs with higher fatigue levels into lower levels. Using the AMT fatigue assessment data along with the current level of fatigue of the AMT, the specific type of work for that day, and a standard FRM assessment chart for the organization, the AI FRM system could provide management the best mitigation for that fatigued AMT to work safely while making the best proactive work safety decisions for SMS.
Understanding the Impact of Using AI in FRM System for Maintenance In transitioning the AI FRM analysis into the bottom part of the CHAAIS model ‘Understanding Impact’, in Figure 3, ethical and technical aspects need to be addressed in step 3. Ethically such an AI-integrated FRM system would improve safety and efficiency by reducing fatigue-induced human errors in maintenance, which in turn would reduce costs and prevent accidents. While this appears to be the higher ethical ground, the other realistic issue would be in the amount of employee protection afforded by the AMT labor union to make work conditions safer, while at the same time requiring a substantial amount of private information from each AMT. In the case of AI technical barriers in step 3, developing an AI FRM system for maintenance to make better fatigue-related decisions would be in how much the airline is willing to spend to develop the AI FRM system by contracting or completing the project within the company. Considering the human-AI machine interface concerns of step 4, the AMT would need some form of training on how to use the AI FRM system and understand its purpose for their welfare. Maintenance management training would be required to understand how to optimally use both the fatigue assessment data and the AI-generated mitigation. Step 5 is the importance of using metrics to capture how the AI FRM system is working for adjustments to be made. These metrics must measure the AMT’s fatigue individually and organizationally in fatigue levels over set times with a goal of fatigue reduction. Another important metric would be collection of organizational maintenance-related incidents and accidents and their reduction while using the AI FRM system.
Conclusion on the Integration of AI In an FRM Maintenance System The authors examined the impact of declining AMT levels in a growing U.S. industry, exacerbating problems of fatigue, costs, and hidden dangers. While FRM systems for maintenance have improved in terms of policy and knowledge, they still need to improve in efficiency and effectiveness in application. Current FAA policy adequately supports an airline maintenance FRM system within an SMS, however, the authors posit that the best way to implement such a system is through integrating it with AI as part of a global trend. The estimated AI spending for global commercial aviation is one billion dollars over the next six years to 2030 (Kumar, 2023). Through the proven CHAAIS research framework, the authors have identified how powerful an AI FRM system could be. Such a system could enhance the daily fatigue assessment of each AMT and use the data proactively for strategic safety decisions required in SMS to help maintenance management reduce the risk of fatigued AMTs committing maintenance errors. Such an AI- integrated FRM system could be an immeasurable safety and efficiency enhancement for airline maintenance and other industries.
REFERENCES
Australian National Transport Commission (2004). Fatigue management within the rail industry: Review of regulatory approach. Melbourne, Australia. Boeing. (2021). Pilot & technician outlook 2021–2040. Report_PTO_R4_09131
AQ-Q-A. Chaparro, A., Groff, L., Chaparro, B., Scarlett, D. (2002). Survey of Aviation Technical Manuals: Phase 2 Report. User Evaluation of Maintenance Documents (Technical report No. DOT/FAA/AR-02/34), Washington, DC FAA, Office of Aviation Research. Cleveland Clinic. (2021). Shift Work Sleep Disorder (SWSD): Symptoms &
Treatment. At Cleveland Clinic online: https://my.clevelandclinic.org/health/ diseases/12146-shift-work-sleep-disorder Dupont, G. (1993). Human Performance Factors for Elementary Work and Servicing.
Transport Canada, TC14175, pp. 1–27. Dinges, F., Powell, J. (1985). Microcomputer Analyses of Performance on a Portable, Simple Visual RT Task During Sustained Operations. Behavior Research Methods, Instruments & Computers. 17.6 (1985): 652–655. Dinges, D., Mallis, M., Maislin, G., Powell, J. (1998). Evaluation of Techniques
for Ocular Measurement as an Index of Fatigue and the Basis for Alertness Management. DOT HS 808 762. Washington, D. C.: National Highway Traffic Safety Administration. FAA. (2016). FAA Advisory Circular 120–115 Maintainer Fatigue Risk Management. FAA Website: https://www.faa.gov/regulations_policies/advis ory_circulars/index.cfm/ Gladkova, T., Dandikota, D., Bapat, S. Allison, K. (2023). CHAAIS: Climate- Focused Human-Machine Teaming and Assurance in Artificial Intelligence Systems Framework Applied Toward Wildfire Management Case Study. In: Proceedings of the AHFE2023, Artificial Intelligence, Social Computing and Wearable Technologies.
Implementing an AI Fatigue Risk Management System 11
Greeley, H., Berg, J., Friets, E., Wilson, J., Greenough, G., Picone, G., Whitmore, J., Nesthus, T. (2007). Fatigue estimation using voice analysis. Behavior Research Methods, 39, 610–619. Herbic, J. (2020). Analysis of Database Online Aviation Safety Reporting System.
In ERAU Graduate Capstone Project September 2020, Data retrieved from https: //asrs.arc.nasa.gov/search/database.html. Hobbs, A., Avers, K., Hiles, J. (2011). Fatigue Risk Management in Aviation Maintenance: Current Best Practices and Potential Future Countermeasures. Technical Report DOT/FAA/AM-11/10. Washington, D. C.: FAA, Office of Aerospace Medicine. ICAO. (2016). ICAO Fatigue Management Approaches. ICAO website: https://www.icao.int/safety/fatiguemanagement/Pages/FM-Approaches.aspx Kocalevent, R., Hinz, A., Brahler, E., Klapp, B. (2011). Determinants of Fatigue and
Stress. Department of Mental Health, University of Leipzig, Germany, Published online. Kumar, P. (2023). Artificial intelligence in aviation market. Allied market research. Available: https://markwideresearch.com/artificial-intelligence-inaviatio n-market/ Miller, M. D., Mrusek, B. (2018). The REPAIRER Reporting System for Integrating Human Factors into SMS for Aviation Maintenance. In: Proceedings of the AHFE2018, Advances in Safety and Management Systems, vol. 791. Springer. National Response Team. (2024). Fatigue Management Risk Assessment Tool. NRT
Website: https://www.nrt.org/sites/21/files/fatigue_mgmt_tool.pdf NovoPsych. (2024). Fatigue Assessment Scale (FAS). NovoPsych Website: https://novopsych.com.au/assessments/health/fatigue-assessment-scale-fas/ NTSB. (2023). Most Wanted List Archive. National Transportation Board Website: https://www.ntsb.gov/Advocacy/mwl/Pages/mwl_archive.aspx NTSB. (2024). Preliminary Report of February 6, 2024, on ‘In Flight Structural
Failure, Alaskan Airlines Flight 1282. National Transportation Board Website: https://www.ntsb.gov/investigations/Pages/DCA24MA063.aspx Predictive Safety. (2024). AlertMeter Alertness Tracking. Predictive Safety Website: https://predictivesafety.com/alertmeter/ Rankin, W. (2007). MEDA Investigation Process. In Boeing AERO Issue 26_
Quarter02, Boeing.com/commercial/aeromagazine. Web. 5 Feb. 2022. 15–21. Thorne, D., Johnson, D., Redmond, D., Sing, H., Belenky, Shapiro, J. (2005). The Walter Reed Palm-Held Psychomotor Test. Behavior Research Methods. 37.1 (2005): 111–118. Williamson, A., Chamberlain, T. (2005). Review of on road driver fatigue monitoring
devices. Kensington, Australia: University of New South Wales, Injury Risk Management Research Centre.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors in Design, Engineering, and Computing
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-35-9
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional