Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Harnessing Emerging Technologies to Enhance Decision-Making in Competency-Based Training and Assessment

1 Purdue University, School of Aviation and Transportation Technology, West Lafayette, IN 47907, USA ↗
2 Coventry University, Faculty of Engineering, Environment & Computing, Coventry CV1 5FB, U.K ↗
3 Hellenic Air Force Academy, Athens, Greece ↗
4 Department of Production Engineering and Management, Technical University of Crete, Greece ↗
Publication year
Digital Object Identifier10.54941/ahfe1007101

Abstract

The transition from traditional training methods to Competency-Based Training and Assessment (CBTA) represents a paradigm shift in aviation education, aligning learning outcomes with operational realities rather than rote procedural mastery. This study advances CBTA by re-envisioning decision-making—a complex ICAO core competency—as a dynamic, human-centered process cultivated within adaptive and operationally authentic environments rather than through procedural repetition. Using the ICAO ADDIE framework, the research identifies recurrent decision-making challenges from accident data and training records, designs culturally intelligent and human-factors-integrated learning modules, develops AI-driven digital twins, immersive VR/AR simulations, and smart haptic systems to replicate complex operational contexts, and implements these innovations across pilot, air traffic, and maintenance training programs. Evaluation integrates quantitative metrics—reaction times, decision accuracy, workload indices—with qualitative insights from reflective debriefings and peer assessment to measure competency growth. AI enhances objectivity and reduces assessor bias through real-time behavioral analytics, while immersive and tactile simulations provide exposure to rare, high-risk scenarios that cannot be safely recreated in live training. The resulting ecosystem transforms CBTA from static evaluation toward a responsive, data-informed socio-technical model in which human expertise and technological adaptability co-evolve. The study contributes theoretically by redefining CBTA as an adaptive learning system, practically by producing validated decision-making modules, and strategically by offering policy guidance to regulators such as ICAO, EASA, and FAA for the inclusive and harmonized integration of emerging technologies into aviation training frameworks.

INTRODUCTION

Decision-making is the integrative competence that converts perception, knowledge, and teamwork into timely action. In line operations, choices are made under incomplete information, evolving risks, and shifting goals. Traditional scenario banks and instructor-led grading rubrics have helped standardize training, yet they struggle to reflect how decisions unfold in the wild: incrementally, iteratively, and interdependently. This paper positions emerging technologies as enablers of a more faithful training ecology—one where decision-making is practiced as a living process, observed at fine gran ularity, and improved through targeted feedback. The central thesis is that Competency-Based Training and Assessment (CBTA) can mature from a curriculum schema into a socio-technical ecosystem. In such an ecosystem, virtual and augmented reality (VR/AR) immerses learners in rare but safe ty-critical contexts; artificial intelligence (AI) captures latent patterns and offers explainable coaching; digital twins provide longitudinal continuity and benchmarking; and smart materials bring tactile realism to the train ing envelope. Together, these elements cultivate not only accuracy but also adaptability, fairness, and resilience.

DECISION-MAKING IN CBTA AND SAFETY SCIENCE CBTA reframes learning around observable behaviors linked to operational outcomes. Within this framing, decision-making sits at the nexus of atten tion management, situation assessment, option generation, risk appraisal, and commitment. Safety science contributes two additional imperatives: first, avoid brittle solutions by training for variability and surprise; second, value learning from successful adaptations (Safety-II) as much as from fail ure. These imperatives suggest that competency growth should be measured not only by correctness but by the quality of the process—how teams share intent, surface uncertainty, and revise plans.

Human factors research further indicates that decision quality is shaped by workload, interface intelligibility, and social context. Authority gradients, language load, and cultural norms can suppress challenge or delay clarifi cation at precisely the wrong moment. Decision-making must therefore be trained in environments that include these social frictions rather than ideal izing them away. This is where cultural intelligence and inclusive design enter as structural supports for competence.

METHODOLOGY

We implement the International Civil Aviation Organization (ICAO) Analysis, Design, Development, Implementation, Evaluation (ADDIE) cycle to embed decision-making into an adaptive, technology-enabled training architecture. In Analysis, historical events, Line Operations Safety Assessments (LOSA) excerpts, and simulator transcripts are coded for decision bottlenecks, ambig uous cues, and misaligned mental models. In Design, these insights trans late into interaction requirements for simulators, digital twins, and analyt ics—prioritizing interpretability, contestability, and inclusivity. Development

Harnessing Emerging Technologies to Enhance Decision-Making 425 produces VR/AR modules, AI observers, and haptic assets that reinforce mul timodal cues. Implementation integrates modules into recurrent programs for pilots, controllers, and maintenance staff with consistent scaffolding and debriefs. Evaluation triangulates quantitative measures with reflective narra tives and peer feedback to build a complete view of competency growth and transfer to line operations.

Table 1: Cultural variables and corresponding cockpit design responses.

Table available in the original PDF paper.

Decision-Making Challenge

Operational Manifestation

Technology Response

Training Artifact Indicative Metrics

Ambiguous system state

Mode confusion; late recognition of degraded automation.

Explainable AI overlays; state summarization in digital twin.

‘Why-now’ panels; intent strips with cues and confidence.

Detection latency; explanation sufficiency ratings; error recovery time.

Time pressure & workload spikes

Tunnel vision; premature closure; missed alternatives.

Scenario timers; micro-pauses; option-generation scaffolds.

Adaptive pacing in VR; workload- aware prompts; haptic pacing cues.

Option count; reconsideration events; workload index trends.

Cross-cultural communication

Dialog templates; participation balance analytics.

Hesitation to challenge; indirect phrasing; mishearings.

Language mediation; paraphrase echo; closed-loop confirmation widgets.

Readback accuracy; challenge timing; participation index.

Risk perception bias

Counterfactual branches; safety margin meters.

Over/ under-weighting low-frequency hazards.

Bayesian risk visualizations; near-miss replay; outcome simulators.

Risk calibration error; margin preservation; debrief coherence.

Team coordination drift

Fragmented intent; conflicting assumptions.

Plan–talk alignment probes; joint commitment markers.

Shared intent panels; co-authorable checklists; plan quality checks.

Intent alignment score; confirmation loops; task handoff fidelity.

DESIGN AND DEVELOPMENT: FROM SCENARIOS TO ADAPTIVE MODULES The instructional design philosophy positions the trainee as an active sense- maker rather than a passive recipient of procedural knowledge. Learning expe riences are constructed as evolving operational ecosystems in which environ mental cues compete for attention, ambiguity must be reconciled, and social interactions influence timing and coordination (Helmreich & Merritt, 1998).

Immersive virtual and augmented reality (VR/AR) environments recreate such contexts, enabling learners to engage in decision-making that mirrors the cognitive and emotional demands of real-world flight operations. Within these dynamic simulations, AI–enabled observers analyze decision episodes, identify untested assumptions, and generate probing questions that instruc tors can adopt or adapt as needed.

Complementary smart-haptic technologies introduce subtle tactile anchors—graded physical nudges that guide the learner’s attention during workload peaks. These gentle cues support timely verification actions with out relying on intrusive auditory alerts, thereby maintaining immersion and situational focus (Hollnagel, 2014). Decision-making is further decomposed into discrete yet interdependent micro-skills: framing the problem, surfacing uncertainties, generating options, managing graded commitments, and artic ulating intent. Adaptive training modules target these micro-skills through graduated difficulty and process-focused feedback, emphasizing how deci sions are made rather than merely what decisions are reached. This design principle resonates with the broader systems perspective that sees human performance as dynamic, context-dependent, and open to reflective refine ment (Leveson, 2012; Reason, 1997).

Emerging Technologies: AI, Digital Twins, and Immersive Integration Artificial intelligence introduces objective, consistent observation capabil ities that extend far beyond human perceptual bandwidth (EASA, 2023). Classifier algorithms detect patterns in decision generation, challenge tim ing, and confirmation loops, allowing for more granular insights into cog nitive behavior. Crucially, these systems are designed with explainability in mind. Visual analytics illustrate which cues triggered an inference and which alternative interpretations were plausible, transforming opaque machine logic into interpretable reasoning aids. Digital twin architectures preserve continuity across multiple training sessions, enabling longitudi nal mapping of individual and team competencies. Through these mod els, scenarios can be “rehydrated”—replayed with altered variables—to reinforce specific decision patterns or remediate persistent weaknesses. When integrated with VR and AR, these technologies recreate opera tional complexity at perceptual, cognitive, and social levels, ensuring that decision-making is practiced under realistic conditions that stress per ception, comprehension, and coordination simultaneously (ICAO, 2023). To preserve learner agency and trust, AI-generated guidance is explicitly framed as a suggestion rather than a verdict. Instructors remain the ulti mate arbiters of performance assessment and contextual interpretation. Trainees, in turn, are encouraged to interrogate the AI’s reasoning, com paring its analytical framing to their own situational understanding. This dialogic engagement transforms AI from an evaluative instrument into a

Harnessing Emerging Technologies to Enhance Decision-Making 427 co-creative learning partner, enhancing both interpretive trust and trans fer of competence across operational contexts (Ang & Van Dyne, 2008; Ziakkas & Henneberry, 2025).

Implementation: Cross-Functional Programs for Pilots, Controllers, and Maintenance Personnel Decision-making represents a unifying competency that transcends tradi tional role boundaries across aviation domains. Implementation strategies must therefore bridge pilot line operations, air traffic control (ATC), and maintenance coordination centers to ensure systemic coherence (Reason, 1997). While the foundational training principles—shared intent, explicit articulation of uncertainty, and closed-loop communication—remain con stant, their manifestations differ according to operational demands. For controllers, linguistic precision and pacing under high temporal pressure are central to maintaining situational clarity (Helmreich & Merritt, 1998). In maintenance environments, diagnostic ambiguity, asynchronous collab oration, and shift handovers demand structured reasoning traces that pre serve the evolution of technical hypotheses. Cross-functional cohort training facilitates perspective-taking and mutual understanding, aligning decision heuristics across roles and fostering a shared safety culture. Such interdisci plinary programs embody the human-centric vision advanced by ICAO and the European Union Aviation Safety Agency (EASA), in which collabora tive decision-making strengthens resilience across the entire aviation system (EASA, 2023; ICAO, 2023).

Evaluation: Mixed-Methods Assessment of Competency Growth The evaluation of competency growth within this ecosystem requires a bal ance between quantitative rigor and qualitative depth. Quantitative telem etry offers objective measures—such as decision latency, option diversity, readback fidelity, and preservation of safety margins—that provide a data- driven view of procedural fluency (Leveson, 2012). However, these metrics alone are insufficient to capture the nuances of cognitive adaptation and professional judgment.

Accordingly, qualitative methods—guided self-explanations, reflective debriefings, and peer assessments—reveal the evolving mental models that underpin effective performance (Ziakkas et al., 2026). This mixed-methods design ensures that evaluation does not devolve into score optimization but instead promotes robust, transferable learning. By uniting analytic precision with reflective inquiry, the assessment framework reinforces the broader objective of CBTA: cultivating adaptive expertise and decision resilience in complex, human-technology ecosystems (Hollnagel, 2014; Reason, 1997). Table 2 summarizes the followed research framework.

Table 2: ADDIE-aligned evaluation framework for decision-making in CBTA.

Table available in the original PDF paper.

Addie Phase Objective Instruments/Data Key Metrics Decision-Use

Accident/simulator reviews; LOSA excerpts; workload traces.

Bottleneck taxonomy; cue-conflict patterns; language-load indices.

Analysis Identify recurring decision bottlenecks and context stressors.

Select scenarios; set design constraints; prioritize risk areas.

Process markers; explainability criteria; fairness checks.

Design Specify adaptive modules and inclusivity safeguards.

Learning outcomes; dialog schemas; human- machine interface requirements.

Blueprint modules; define coaching prompts; guardrails for bias.

Prototype logs; usability studies; rubric validation.

Usability; rubric inter-rater reliability; explanation quality.

Development Build VR/AR, AI observers, and haptic assets.

Iterate features; tune thresholds; finalize assessment forms.

Implementation Deploy

Cohort telemetry; instructor notes; trainee reflections.

Participation rates; completion time; trust/acceptance indices.

across domains with instructor enablement.

Embed in recurrent Trainings; update lesson plans.

Evaluation Measure growth and transfer to line ops.

Effect sizes; retention; trend deltas in miscomms/ near-misses.

Scale programs; inform policy; refine models.

Pre/post tests; longitudinal twin data; safety management systems hooks.

DISCUSSION

Decision-Making in Competency-Based Training and Assessment Regulatory evolution must progress in tandem with innovation in training methodologies. The development of Acceptable Means of Compliance (AMC) for CBTA should therefore include demonstrable evidence that technological tools are intelligible, contestable, and free from discriminatory bias (EASA, 2023; ICAO, 2023). Establishing such confidence demands rigorous evalu ation protocols—such as linguistic robustness testing for AI-generated feed back, fairness analyses across diverse demographic groups, and systematic assessments of how analytics enhance inter-rater reliability. A harmonized approach among ICAO, EASA, and the Federal Aviation Administration (FAA) would help eliminate unnecessary duplication of effort and foster greater clar ity in procurement and certification standards (Leveson, 2012). Despite the growing integration of advanced technologies, instructor expertise remains the cornerstone of effective CBTA. The educator’s role is gradually shifting

Harnessing Emerging Technologies to Enhance Decision-Making 429 from that of a sole evaluator to a facilitator of sense-making and reflec tion, guided by transparent and interpretable analytics (Ang & Van Dyne, 2008; Ziakkas & Henneberry, 2025). Contemporary faculty development programs increasingly emphasize the interpretation of AI-driven insights, the management of structured debriefings, and the cultivation of reflective learn ing practices. Supporting this evolution, a curated repository of process-ori ented prompts encourages discussions that move beyond binary judgments of performance toward deeper explorations of reasoning, decision-making, and adaptive learning (Helmreich & Merritt, 1998).

Within this pedagogical framework, process-focused feedback structures debriefings around five iterative phases: framing, sensing, deciding, acting, and reviewing (Reason, 1997). Each phase captures both exemplary behav iors and developmental opportunities. Framing is assessed through the clar ity with which a trainee articulates goals and constraints; sensing, through the recognition and integration of uncertainty cues; deciding, through the quality, diversity, and timing of chosen options; acting, through the coor dination of team behaviors and preservation of operational margins; and reviewing, through the synthesis of experiences into future heuristics. Over time, such structured reflection fosters measurable growth in metacognitive regulation—the ability to monitor and adapt one’s cognitive processes under operational pressure (Hollnagel, 2014). For CBTA to be both technologically advanced and genuinely inclusive, its design must recognize the diversity of learners. Adaptive pacing and language mediation can significantly reduce cognitive load for non-native speakers, while visual aids and haptic feed back enhance learning for individuals with varied perceptual preferences. Equally, culturally attuned dialogue templates can prevent misunderstand ings and preserve psychological safety in multinational crews (Ziakkas et al., 2026). Accessibility features such as captioning, color-neutral symbology, and adjustable text size not only promote equity but also sustain operational realism without compromise (Ang & Van Dyne, 2008).

The increasing reliance on analytics and digital-twin environments intro duces new dimensions of ethical responsibility. Training organizations must establish transparent data-governance frameworks, defining retention limits, anonymizing performance traces, and ensuring a clear separation between formative analytics and summative, high-stakes personnel decisions. Such safeguards preserve trust and psychological safety within the learning envi ronment. Trainees should be informed about what data are collected, how they are analyzed, and must retain the right to request the removal of identi fiable voice or video elements once metrics have been derived. These practices align technological capability with ethical stewardship, reinforcing account ability and confidence in the training ecosystem (Leveson, 2012).

Finally, the business case for investment in technology-enabled CBTA extends beyond compliance or efficiency metrics. Tangible benefits include reduced go-arounds caused by miscommunication, fewer simulator re-runs through targeted remediation, and greater instructional throughput sup ported by analytics-assisted debriefs. Even conservative models suggest that modest reductions in decision-related retraining can offset initial platform costs within two recurrent training cycles. As programs mature, longitudinal

data further refine resource allocation, allowing organizations to focus improvement efforts on persistent decision bottlenecks and areas of cognitive strain. In this way, innovation serves not merely to modernize training, but to humanize it—supporting reflective, equitable, and resilient learning across the aviation community.

LIMITATIONS AND FUTURE WORK The present design emphasizes simulator-captured decision telemetry and may under-represent social complexity found in line operations. Field val idation using LOSA hooks and SMS trend analysis is essential to confirm durable transfer. Future work will expand cross-cultural validation, integrate maintenance and dispatcher contexts more deeply, and explore privacy-pre serving speech analytics that enhance safety while enabling learning at scale.

CONCLUSION

Decision-making is the capstone competence in CBTA. By mobilizing VR/ AR, explainable AI, digital twins, and smart haptics within a principled ADDIE cycle, aviation training can move beyond static scenario playbooks to a dynamic, inclusive, and resilient ecosystem. The result is not only better choices in the simulator, but better habits of sense-making and coordination in the sky.

ACKNOWLEDGMENT

The authors thank faculty members of Coventry University, Purdue University, Hellenic Air Force Academy and HF Horizons for their invaluable feedback contributing to this work.

REFERENCES

Ang, S., & Van Dyne, L. (2009). Conceptualization of cultural intelligence. In S. Ang & L. Van Dyne (Eds.), Handbook of cultural intelligence. M.E. Sharpe. European Union Aviation Safety Agency (EASA). (2023). Artificial intelligence

roadmap 2.0: Human-centric approach to AI in aviation. Cologne, Germany: European Union Aviation Safety Agency. https://www.easa.europa.eu/en/ document-library/general-publications/easa-artificial-intelligence-roadmap-20 Helmreich, R. L., & Merritt, A. C. (1998). Culture at work in aviation and medicine.

Routeledge. https://doi.org/10.4324/9781315258690 Hollnagel, E. (2014). Safety-I and Safety-II: The past and future of safety management. Ashgate. https://doi.org/10.1201/9781315607511 International Civil Aviation Organization (ICAO). (2023). Annex 19 to the

Convention on International Civil Aviation: Safety management (4th ed.). Montreal, Canada: International Civil Aviation Organization. https://store.icao. int/en/annex-19-safety-management.

Harnessing Emerging Technologies to Enhance Decision-Making 431 Leveson, N. G. (2012). Engineering a safer world: Systems thinking applied to

safety. MIT Press. https://doi.org/10.7551/mitpress/8179.001.0001 Reason, J. (1997). Managing the risks of organizational accidents. Ashgate. Ziakkas, D., & Henneberry, D. (2025). Investigating transport accidents: Human performance and intelligent systems. CRC Press. https://doi. org/10.1201/9781003801234 Ziakkas, D., Pechlivanis, K., & Plioutsias, A. (2026). Cultural intelligence in aviation:

Bridging social attitudes, culture, and safety in a global environment. Routledge.

Cover of the proceedings volume Intelligent Human Systems Integration (IHSI 2026): Disruptive and Innovative Technologies

Explore the complete digital volume

Intelligent Human Systems Integration (IHSI 2026): Disruptive and Innovative Technologies

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-964867-76-2

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →