William Phillips, Angelia Sebok, Anna Grome (2025). Design, Development, and Evaluation of a Crew Resource Management Learning Experience to Improve Freight Rail Safety. In: Advances in Human Factors of Transportation. AHFE Open Access, vol. 186, pp. 482–491. https://doi.org/10.54941/ahfe1006538
In 2015, the National Transportation Safety Board (NTSB) recommended that the Federal Railroad Administration (FRA) mandate Crew Resource Management (CRM) training for rail crews, based on CRM's proven success in reducing human error and enhancing safety in aviation. Recognizing parallels in the rail sector, and supported by recent FRA research (Rosenhand et al., 2012; Roth et al., 2013; Sebok et al., 2017), the rail industry stands to benefit from structured CRM training to improve communication, teamwork, and decision-making among train crews. Researchers at TiER1 Performance were engaged to investigate, develop, pilot, and assess a CRM training learning experience tailored to rail. This paper describes the design and development of a rail-industry CRM training prototype and discusses the results of a formative evaluation of the learning experience.
Crew Resource Management (CRM) is a Human Factors training process originally introduced in the aviation industry to improve communication, coordination, and teamwork skills among flight crews to enhance safety and reduce accidents resulting from human error (Kanki, Anca, & Chidester, 2019). CRM was developed in response to a series of aviation accidents in the 1970s, where poor crew coordination and communication were identified as significant factors. Due to its success in aviation, CRM training has been adapted in multiple safety critical industries (Dunn et al., 2007; O’Dea, O’Connor, & Keogh, 2014; Powell & Hill, 2006; Ricci & Brumsted, 2012; Sundar et al., 2007).
In 2015 the National Transportation Safety Board (NTSB) recommended CRM for the rail industry. While there have been attempts to design and implement CRM training in rail (Morgan et al., 2006; Morgan et al., 2007), CRM has not been consistently integrated in the freight rail industry. FRA- sponsored research studies have underscored the opportunity to improve
Design, Development, and Evaluation of a Crew Resource Management 483
safety in rail operations by providing CRM training to help teams work together more effectively. For example, Roth, Rosenhand, & Multer, 2013, (p. 27) asserted: “A focus on effective communication and increased crew resource management training would enhance teamwork and encourage joint problem-solving and decision making that could leverage the knowledge and skills of the entire train crew.” Furthermore, research (Sebok, Walters, & Wickens, 2017; Sebok et al., 2020) identified human errors, including setup and mode transition errors, that went undetected in simulator scenarios. In both cases, improved crew communication, including verbal callouts and cross-checking, may have helped the crew detect and mitigate these errors.
Based on the need identified by NTSB and research studies, this project was initiated to develop a CRM learning experience to improve communication and teamwork in the freight rail industry. The overarching objective of this work was to reduce the potential for human error in operations and increase rail safety. The work was conducted in three phases: 1) Front End Analysis and identification of training requirements, 2) Design of the prototype learning experience and development of materials and assets to support the learning experience, and 3) Pilot and formative evaluation of the prototype. Each of these phases is described below.
FRONT- END ANALYSIS AND IDENTIFICATION OF TRAINING REQUIREMENTS To design the CRM learning experience (i.e., training), the research team conducted a front-end analysis to identify the areas where teamwork and communication are most vulnerable and have the greatest implications for safety. The front-end analysis included a literature review, an analysis of rail- industry incidents, a set of interviews with SMEs, and on-site observations at a freight rail dispatch National Operations Center (NOC).
The literature review included 25 scholarly sources regarding CRM in safety-critical industries. The literature reviewed focused on how CRM is applied across safety critical incidents, coupled with literature addressing communication and teamwork in rail. An analysis of rail-industry incidents was supported by the Close Call Reporting System (C3RS) - an anonymous rail industry database that describes incidents and near-misses. The research team reviewed 50 reports from the database to determine common causes of accidents and CRM risk vulnerabilities to prioritize through the training prototype. This analysis identified patterns and specific areas of vulnerability that informed the training design.
Researchers conducted interviews with locomotive engineers and conductors, each averaging 20 years of experience, and dispatchers with an average of 12 years of experience. The research team identified members and roles within the rail team, the communication links, the methods by which communication occurs, and the automation that supports the various personnel. The interviews also focused on uncovering the most common vulnerabilities and risk areas in crew-dispatch communication and teamwork to identify CRM needs for the training program. In addition, three researchers visited a freight railroad NOC, shadowing and interviewing
dispatchers at the facility. The team observed dispatcher communications, tasks, and other responsibilities to understand operational challenges, and conducted interviews with dispatchers to deepen understanding of the observed activities and systems.
Through the front-end analysis, the research team identified eight core findings related to CRM training and operational challenges in the rail industry, listed below:
1. There is a current gap - and need for - training on teamwork, communication, and CRM-related topics in the rail industry. 2. The crew-dispatcher relationship is a particularly vulnerable interaction within the rail team. 3. There is a need for more frequent and proactive communication from dispatchers, as expressed by training crew members 4. The lack of an established model for effective communication and teamwork leaves a gap in training. 5. A cultural tension characterized by an “us versus them” mentality hinders trust and effective communication. 6. Rail operators have limited visibility into the training received by their colleagues. 7. Train crews have a strong preference for instructor-led, peer-based training over computer-based methods. 8. A potential opportunity for integrating CRM training lies within the annual full-day instructor-led training course.
The front-end analysis results were used to define requirements for the CRM training prototype. Training requirements aimed at addressing the identified communication and teamwork gaps, listed in Table 1.
Table 1: Requirements for the CRM learning experience.
Table available in the original PDF paper.
Requirement Training Experience Area
Scope Focus training on communication between crew and dispatcher Communication Address the different categories of communication that occur between crew and dispatchers, ranging from simple to more complex. Provide exposure to effective communication practices (words and phrases) Include a model of good communication practices Assertiveness Provide professional, constructive, respectful dissenting perspectives to solve problems Understanding/ Situational Awareness
Provide insights into others’ work – what do train crews experience, what does dispatch experience. Address how and why automation is used by both roles Customizability Include general communications training that can be customized to the audience Realism Leverage actual incidents to illustrate communication challenges Integration Integrate the training with existing safety training Continued
Design, Development, and Evaluation of a Crew Resource Management 485
Table 1: Continued Requirement Training Experience Area
Table available in the original PDF paper.
Duration Keep to 60 minutes max Modality Instructor led, with scenario-based practice and feedback Participants Include both crews and dispatch in the training Instruction Delivered by a SME with relevant operational experience
LEARNING EXPERIENCE DESIGN AND DEVELOPMENT The prototype learning experience was designed and developed through iterative design/evaluate/refine cycles where both trainees and facilitators reviewed the overall experience design, as well as the supporting materials, and provided feedback. The design and development of the training was informed by both the training requirements identified in Phase I and learning theories.
Learning Theories to Apply to Training In designing the CRM learning experience, the team incorporated social learning, constructivist, and situated learning theories, along with adult learning principles and learner-centered training approaches. Social learning theory (Chuang, 2021; Dirksen, 2016) highlights the value of peer interactions in learning, aligning with CRM’s focus on teamwork. The training fosters engagement by bringing together rail personnel to humanize interactions and improve communication. Constructivist theory (Chuang, 2021; Powell & Bodur, 2019) emphasizes active knowledge building, applied through interactive scenarios that encourage trainees to analyze past experiences and extract lessons relevant to their roles. Situated learning (Lave & Wenger, 1991; Niewolny & Wilson, 2009) underscores the importance of learning in real-world contexts, reinforced in this training through job-related discussions and peer-led instruction. Adult learning principles (Merriam & Bierema, 2013) emphasize relevance and active participation, thus indicating a need to encourage trainees to share experiences and reflect on how CRM applies to their work. Additionally, learner-centered training (Dirksen, 2016) indicates that practice opportunities, follow-up activities, and job aids are useful resources to ensure the retention and application of key concepts.
Experience Map Grounded in the training requirements and learning theories, the research team designed a high-level map of the intended learning experience with key phases and learning objective for each phase (See Figure 1). The learning experience was intentionally designed to bring together engineers, conductors, and dispatchers, ensuring a collaborative learning environment. The design is intended to actively engage participants by encouraging them to share personal experiences—either moments when communication breakdowns led to challenges or instances where effective communication prevented a crisis. To further reinforce key concepts, the experience is
designed to include scenario-based discussions led by a facilitator. Though materials and assets such as PowerPoint slides and participant guides provide structure, the experience is designed to be dynamic, featuring interactive discussions and video clips to maintain participant engagement.
Structured in three distinct phases—Prime, Enable, and Sustain—the learning experience follows a progressive approach. The Prime phase introduces the material with a safety leader engaging rail personnel in a preliminary discussion. To prepare for the course, participants receive pre- training materials, including two brief videos on the importance of CRM and a one-page overview of the training objectives. The Enable phase comprised the core, in-person training session, where concepts are explored in depth. Finally, the Sustain phase aims to reinforce learning through job aids and refresher training. However, the development of Sustain-phase resources was beyond the scope of this particular project. The design of the learning experience was vetted with both rail operators and instructors and refined based on their input.
Figure 1: The CRM learning experience map.
Training Materials and Assets Following finalization of the design, the research team developed the supporting assets and materials. The following training products were created to support the CRM learning experience prototype:
Prime
An e-mail to inform trainees about the upcoming training experience.
A list of talking points for the Facilitator to discuss with trainees.
A CRM overview video, discussing the importance of the principles and their relevance to the rail industry.
A Union Leader video, where an engineer discusses the importance of CRM in rail operations.
A learning experience blueprint (also considered a Sustain asset).
Design, Development, and Evaluation of a Crew Resource Management 487
Enable
Facilitator materials: – A PowerPoint slide deck with facilitator notes to guide the training. – A facilitator version of the scenario guide, with injects for increasing or decreasing the difficulty based on participant performance.
Participant materials: – A guide to give the trainees the opportunity to make notes and reflections and to read relevant information. – A set of role personas. – A participant version of the scenario guide.
DELIVERY AND FORMATIVE EVALUATION OF THE PROTOTYPE CRM LEARNING EXPERIENCE The CRM learning experience prototype was piloted with six engineers and conductors, averaging 25 years of experience. In addition to the in-person participants, three dispatchers participated virtually, via Teams, creating a hybrid pilot experience. The dispatchers averaged 12 years of experience. Many of the formative evaluation participants held safety positions or union leadership roles. Training was held on-site, at a railroad training facility. The purpose of the formative evaluation was to identify necessary improvements or upgrades to the training materials and overall approach.
The prototype learning experience required six hours of presentation and scenario discussion. To complete the training within the eight-hour timeframe, and provide time for gathering feedback, the team reduced the number of scenarios from five to two.
The prototype underwent a formative evaluation using three evaluation tactics. First, researchers attended the training with structured observational checklists. General observation was conducted by two observing in-person researchers and one virtual-participating researcher. Second, questions delivered by Mentimeter were given immediately post-training to capture participants’ feedback and identify key themes. Finally, a three-month follow- up focus group and survey were conducted to evaluate the impact of the learning experience on participants’ job performance and identify sustained outcomes.
Researchers observed and evaluated participant engagement, amount of distraction, extraneous discussion, and reflection of participants. These general observations yielded positive results with focused and engaged participants who were open about how they could better leverage CRM skills. As further indication of their engagement, multiple participants stayed an additional hour or two to discuss their enthusiasm for the learning experience with the facilitator.
Immediately after the session, participants responded to an anonymous online survey tool, Mentimeter. This same survey was used to collect responses at the three-month follow-up meeting. Results from the two administrations of the Mentimeter are shown in Table 2.
Table 2: Results of the mentimeter immediately after and three months after training.
Table available in the original PDF paper.
Question Rating Immediately Scale
Three Months After Training Later
To what degree do you believe this training is
1–10 9.2 9.6
relevant to the rail industry?
How likely would you be to recommend this
1–10 9.6 9.5
training to other rail crew members or dispatchers?
CRM Knowledge Gains This training helped me understand what CRM
1–5 4.9 4.5
is.
This training helped me understand why CRM
1–5 4.8 4.5
matters in my work in the rail industry.
1–5 4.6 5.0
This training helped me understand how good CRM (good communication and teamwork) can impact safety and efficiency of rail operations.
1–5 4.6 4.8
This training helped me understand the difference between effective and ineffective communication.
1–5 3.8 4.5
Impact of Training This training gave me more confidence in my ability to communicate effectively with my team.
1–5 4.2 3.8
This training gave me a better understanding of the roles and responsibilities of those I routinely interact with.
1–5 4.4 3.8
This training gave me a better understanding of the unique challenges and pressures facing other members of the team.
1–5 2.2 2.8
This training gave me a better understanding of the role that automation plays as a resource for the team.
There were two questions scored 1–10, focusing on the importance and interest of the training. Results averaged 9.4 out of 10 immediately after training and 9.6 for the second administration. Both results indicate a strong belief that the training is relevant and useful to the rail industry.
The four questions related to CRM knowledge gain and results averaged 4.7/5. These results were consistent at three months post-training, indicating that participants felt that they learned about CRM, its importance and relevance.
There were four questions related to skill development/impact from the learning experience. Excluding the question about automation, results averaged 4.1, with similar results at three months post training. The question asking about the impact of the training helping participants better understand
Design, Development, and Evaluation of a Crew Resource Management 489
automation had by far the lowest score of 2.5, averaged across the two administrations. The difference in “understanding the unique pressures of other team members” rating dropped by over 10 percent in the three- month post-training period. During the training, the attendees worked with personnel from a different craft and expressed enthusiasm for the value of the different perspectives. Over time, the participants likely became aware of the many crafts they did not interact with in the training, perhaps diminishing the value of that aspect of training.
When asked about the most valuable part of the training, participants focused primarily on the value of interacting with other roles and gaining an understanding of those roles. They additionally shared positive feedback surrounding the discussion and scenario-based practice. When asked how the training could be improved, participants recommended everyone be together in person, that the training be expanded to include additional crafts, and that the “automation as a teammate” references be removed.
Three months after the learning experience, researchers reconvened with over half of the training participants to hear if the participants had been able to use any of their learnings on the job and to gather feedback on the participants’ impressions of the utility of training. As discussed previously, the Mentimeter results were nearly identical to the results provided immediately following the initial training. Further most participants shared the training had been valuable in their role. One participant shared an example of using assertiveness in his role as an engineer to improve working conditions and safety in the locomotive cab.
Based on the formative evaluation, the research team made minor changes to the overall learning experience. All references to automation were changed from “team member” to “resource.” Researchers worked with dispatchers to create two additional dispatcher-centric scenarios.
CONCLUSION
There are multiple take-aways from this work. First, there are opportunities to improve rail safety through the implementation and utilization of CRM. This is especially important considering the 2015 NTSB recommendation to implement CRM and recent major accidents, like the train derailment in East Palestine, Ohio. Second, CRM should bring together distinct crafts to develop shared empathy, understanding, and to practice CRM skills together. Participants indicated that exposure to other crafts was the most valuable part of the training and wished to include additional crafts. Third, CRM training with distinct crafts addresses a felt need by industry personnel. The participants in this investigation were highly experienced, recognized as experts in their craft. Given that they found the training to be informative and valuable, it is likely that newer, less-experienced personnel would benefit even more from this training. Future research should investigate leveraging CRM training in additional contexts, like other railroads, with less-experienced participants, and with additional crafts included. Also, future research should explore building upon the training for a longer time duration. For example, a
railroad would conduct the training and supplement the learning by including a year of weekly cohort discussions on utilizing CRM through their work.
ACKNOWLEDGMENT
This document is disseminated under the sponsorship of the U.S. Department of Transportation’s Federal Railroad Administration in the interest of information exchange. The United States Government assumes no liability for its contents or use thereof. The United States Government does not endorse products or manufacturers. Trade or manufacturers’ names appear herein solely because they are considered essential to the objective of this report. The opinions and/or recommendations expressed herein do not necessarily reflect those of the U.S. Department of Transportation. The authors would like to acknowledge the project sponsors at the Federal Railroad Administration, Shala Blu and Michael E. Jones, for their support on this project (Contract 693JJ622C000036). Further, we gratefully acknowledge the rail industry SMEs who contributed substantially to this project.
REFERENCES
Chuang, S. (2021). The applications of constructivist learning theory and social learning theory on adult continuous development. Perf. Improvement, 60(3), 6–14. Dirksen, J. (2016). Design for How People Learn. Second Edition. New Riders. Dunn, E., Mills, P., Neily, J., Crittenden, M., Carmack, A., & Bagian, J. (2007).
Medical team training: Applying crew resource management in the Veterans Health Administration. Joint Commission Journal of Quality and Patient Safety, 33(6): 317–25. Kanki, B., Anca, J., & Chidester, T. (2019). Crew Resource Management (3rd Ed.).
Elsevier Academic Press. Lave, J. & Wenger, E. (1991). Situated Learning: Legitimate Peripheral Participation.
Cambridge University Press. Merriam, S. B., & Bierema, L. L. (2013). Adult Learning: Linking Theory and Practice. John Wiley & Sons, Incorporated. Morgan, C., Olson, L., Kyte, T., & Roop, S. (2007). Rail Crew Resource
Management (CRM): Pilot Rail CRM Training Development and Implementation. Wash., D. C.: Department of Transportation, Federal Railroad Administration. Morgan, C., Olson, L., Kyte, T., Roop, S., & Carlisle, T. (2006). Railroad Crew Resource Management (CRM): Survey of Teams in the Railroad Operating Environment and Identification of Available CRM Training Methods. Wash., D.C.: U.S. Department of Transportation, Federal Railroad Administration. Niewolny, K. L., & Wilson, A. L. (2009). What happened to the promise? A
critical (re) orientation of two sociocultural learning traditions. Adult Education Quarterly, 60(1), 26–45. O’Dea, A., O’Connor, P., & Keogh, I. (2014). Meta-analysis of the effectiveness of crew resource management training in acute care domains. Postgrad Medical Journal, 699–708. Powell, C. G., & Bodur, Y. (2019). Teachers’ perceptions of an online professional
development experience: Implications for a design and implementation framework. Teaching and Teacher Education, 77, 19–30.
Design, Development, and Evaluation of a Crew Resource Management 491
Powell, S., & Hill, R. (2006). My copilot is a nurse—Using crew resource management in the OR. AORN Journal, 178–202. Ricci, M., & Brumsted, J. (2012). Crew resource management: Using aviation
techniques to improve operating room safety. Aviation, Space, and Environmental Medicine, 441–444. Roth, E., Rosenhand, H., & Multer, J. (2013). Using Cognitive Task Analysis to Inform Issues in Human Systems Integration in Railroad Operations. Wash., D.C.: U.S. Department of Transportation, Federal Railroad Administration. Roth, E., Rosenhand, H., & Multer, J. (2020). Teamwork in Railroad Operations
and Implications for New Technology. Wash., DC: U.S. DOT, FRA. Sebok, A., Gacy, A., Haggit, J., & Wickens, C. (2020). Human Error Potential in Human Automation Interaction: Investigations of Performance in the Locomotive Cab. Final Presentation delivered on August 20, 2020. Wash., DC: U.S. DOT, FRA. Sebok, A., Walters, B., & Wickens, C. (2017). Simulator-Based Investigations of
Human Error Potential with Railroad Automation in the Locomotive Cab. HFES Proceedings. 1964–1968. Sundar, E., Sundar, S., Pawlowski, J., Blum, R., Feinstein, D., & Pratt, S. (2007).
Crew resource management and team training. Anesthesiology Clinics, 283–300.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Advances in Human Factors of Transportation
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-62-5
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional