Vlad Zotov (2022). Army Crew Training: Coaching with ITS. In: Applied Human Factors Research. AHFE Open Access, vol. 67, pp. 390–396. https://doi.org/10.54941/ahfe1002694
The training of military crews of armoured vehicles can be enhanced by applying AI-based methods to the training drills. Defence Research and Development Canada used a Human Behaviour Representation approach to create an armoured crew simulation trainer for the Canadian Armed Forces. The Human Behaviour Representation (HBR) approach is a form of rule-based AI that applies a cognitive task analysis to derive a synthetic operator. The cognitive task analysis resulted in a Task Network Model (TNM) for each crew member of the Light Armoured Vehicle (LAV) and for the entire crew. These TNMs were inputted into a discrete event simulator to create a synthetic training environment that combines virtual and human members of the LAV crew. The training platform allows a human member of the team to interact with the synthetic crew through voice production software that was integrated with the synthetic environment.The paper presents the development of the Intelligent Tutoring System module for the LAV crew simulation platform that serves as a human instructor for conducting basic LAV drills. The paper outlines the architecture, functionality, and testing of the module. The work shows how the HBR approach can be used to develop a synthetic coach for training a military crew. The work is a step in developing and testing a general training system for small military teams. The training system will allow to conduct basic crew drills, in which a human crew member will be trained with the synthetic crew members, thus overcoming some of the obstacles that military crew training faces: a logistic difficulty to gather a full crew at the same time and place and a deficiency of qualified instructors. The paper outlines the steps for the follow-up work required to develop a generic AI-based autonomous systems for basic training of small military teams.
The Canadian Armed Forces (CAF) is increasingly adopting simulation and synthetic environments in its overall training strategy (Army Training Autho- rity, 2015). One of the key topics in this trend is the development and testing of training systems represented by a mixed team of synthetic agents and human participants, which can be utilized for training crews of CAF’s armou- red vehicles. Access to team members can be particularly challenging to justify or arrange when they are required mainly to support the training of one team role and receive little training value themselves. Accordingly, researchers have been investigating the use of artificial intelligence (AI) agents to play the role of teammates in the team training contexts (e.g., Demir et al., 2015; Gunzel- mann, Gaughan, Alexander & Tremori, 2014). These synthetic teammates
could help CAF to develop a more comprehensive simulation-based training capability.
In view of the potential for using synthetic teammates in the CAF’s bro- ader simulation and training vision, Defence Research and Development Canada (DRDC) was tasked to investigate application of synthetic team- mates for small team training. The modelling architecture of choice was Human Behaviour Representation (HBR), which is an AI-based approach for modelling synthetic agents, especially suitable to military training. Among the requirements of military training are the abilities to generate observable goal-driven behaviors, mimic physical movements, generate decision-making outcomes, and represent communication processes (Pew & Mavor, 1998). These requirements are not very different from those identified by a broa- der AI community for training applications, but there is a stronger emphasis on making HBR suitable for carrying out tactical and operational tasks that involve coordination and realistic environment changes, including a need to create adversarial force agents.
The HBR modelling process for a training procedure includes two stages: 1) a task analysis of operator’s domain knowledge, doctrines, procedures, and task constraints. The end result of this analysis is a task network model (TNM) for each operator; these TNMs include all necessary information to model operators’ activities for a specific drill or a set of tasks; 2) an integra- tion of TNMs into existing simulated environments, thus enabling them to run in real time and communicate with a human operator. The HBR appro- ach presented in this work was initially developed in DRDC to support the training of maritime helicopter Landing Signals Officers: an HBR-based arch- itecture called Simulated Operator for Networks (Cain, Magee, & Kersten, 2012). At the initial stages of this work, the HBR model had already been deve- loped for the Leopard 2 tank simulation tool and a follow-up work was developed for the Light Armoured Vehicle (LAV) crew simulation package— Interim Crew Gunnery Simulation (ICGS)—with a similar purpose to train LAV crew. In both instances of Leopard 2 and LAV, the simulations represent virtual cockpits of the corresponding vehicles.
The LAV is an infantry fighting vehicle specifically designed for the CAF by General Dynamics Land Systems-Canada. The vehicle has a crew of three: a Crew Commander (CC), a gunner (GNR), and a driver (DRV). Considering that simulating vehicle maneuvering is very difficult in the ICGS virtual envi- ronment, the DRV representation in the LAV crew model is superficial and not currently suitable for vehicle control. Thus, the tasks of CC and GNR as well as their interactions are the leading positions in the LAV application of fire drill documents (DND, 2016).
The inter-related tasks sequences of LAV crew were done with a Hiera- rchical Task Analysis (HTA). HTA describes tasks and sub-tasks at the level of details that allows designers to analyze different potential task sequences that may occur through an interaction with a system (Annett, 2003). The final product of the HTA is an interconnected hierarchical structure with the primary goals represented at the top of the hierarchy and tasks/subtasks represented below. Each task analysis generates a task template that provides
392 Zotov
information regarding a user’s higher and lower-level goals, conditions neces- sary for specific actions to occur, average time required for task execution, and operations to perform. In the context of the armoured vehicles drills, the overall goals express the battle task objectives of each drill, along with the supporting tasks and subtasks that required to achieve the overall goals.
HTA analysis was performed for each crew position and for interactions among crew members (Cain, 2018). The top goals and related subgoals and tasks represent a local network model for each drill, representing a TNM. The task modeling step was carried out by converting of each task’s HTA in a discrete event simulation package called Performance Modeling Environ- ment (IPME; MA&D, 2015). All TNMs were then integrated into a runtime simulation model of the LAV crews. The communication between human and synthetic teammates in HBR-LAV included a voice interaction module, based on the Microsoft Speech Platform (Microsoft, 2022) for speech recognition, and the Sphinx Naturally Speaking platform (https://cmusphinx.github.io) for speech production. Eventually, the HBR-LAV is intended to be inte- grated with the virtual cockpits. Nevertheless, the integration part of this development is still in process, and, as such, it is not covered in this work. The follow-up step in developing HBR-LAV as a training platform was the development of a virtual instructor for LAV crew members.
INTELLIGENT TUTORING SYSTEM FOR HBR-LAV An intelligent tutoring system (ITS) is a computer-based application that provides AI-enabled customized and immediate feedback to trainees (e.g., Nwana,1990). It incorporates some form of artificial intelligence that allows it to be autonomous. The aim of any ITS is to reduce the training burden on instructors by using AI algorithms that can interact with students and provide them with corrective feedback with minimal or no human intervention, allo- wing instructors to focus on higher-level skill development. ITSs have been successfully used in multiple academic, industrial, and military settings (for reviews, see Kulik & Fletcher, 2016). Nevertheless, many of these applicati- ons were designed to train knowledge-based content —such as mathematics, economics or computer science (e.g., Steenbergen-Hu & Cooper, 2014) and vast majority of these tools are designed for individual training. The type of ITS presented in this report is a prototype of training software that can become a tool for team training of motor-cognitive skills.
The ITS module is an extension of the HBR-LAV model that adds a synth- etic agent to monitor actions of GNR and CC and provides them with verbal feedback, thus acting as a drill instructor. The first step in developing an ITS module for HBR-LAV was to develop a prototype with a limited scope which addresses a typical LAV training procedure. For this purpose, a “Power-Up” drill was selected due to its well-defined sequence of standard tasks that does not require a crew to respond to any external environmental event, as would be necessary in some other, battle-related drills (e.g., “Engage” and “Misfire” drills). The “Power-Up”drill is typically performed at the start of the operati- onal day as part of the Opening Up Procedure or after a temporary shutdown (e.g., for resupply of ammunition).
Army Crew Training: Coaching with ITS 393
Similar to the development of HBR-LAV model, the ITS module for Power-Up drill was developed as an autonomous TNM called COACH. The COACH TNM was developed in two stages: 1) identification and analy- sis of the LAV supervisor’s actions pertinent to the Power-Up drill, and 2) development of an IPME-based TNM.
Development of COACHs The process of developing a network for the virtual coach position was simi- lar to developing networks for other crew positions: previously developed tasks for the Power-Up drill network were validated and extended. The ori- ginal sequence of tasks was derived from the operating manuals and was reported in the initial work for developing LAV synthetic crew (Cain, 2018). The updated list of tasks for the Power-Up drill was instantiated with the corresponding IPME tasks that were added to the original network of the drill. These tasks typically contain executable expressions as well as task per- formance information. Figure 1 shows one of the drill’s tasks, “Order Turret Power-Up drill”. As the figure shows, the task includes a number of default parameters such as mean time of task execution, release condition, beginning and ending effects, and assigned operator. The characteristics and values of parameters were obtained from reviewing LAV operating values and in con- sultation with CAF Armoured Vehicle subject-matter expert. Figure 2 shows the task diagram that includes COACH, CC, and GNR TNMs.
The COACH network consists of three tasks in addition to the Power- Up drill TNM: “Initiate Instructor”, “Supervise Power-Up Drill”, and “After-Action Review”. The “Initiate Instructor” task instantiates a synth- etic COACH process in IPME and triggers the onset of the task sequence for COACH. It starts with voice instructions from COACH for the CC and GNR to begin the Power-Up drill.
For the next task, “Monitoring”, the synthetic COACH observes the crew tasks and communications (CC verbal commands to GNR, and GNR replies). Task monitoring is performed by comparing a pre-set sequence of tasks to be executed and the observed sequence of task execution as performed by the CC and GNR. If there is any omission or discrepancy in the trainee execution of the tasks, the COACH records a task ID, a time stamp, and an operator ID to the feedback array in IPME. All CC commands to the GNR (recognized by the speech interaction module) are recorded to the feedback array as a text string along with the time stamp, task ID, and GNR replies/acknowledgments. The “Monitoring”task terminates with the completion of the Power-Up drill, and it is followed by the “After-Action Review” (AAR) task.
The AAR task includes the feedback that the synthetic COACH provides to the CC and GNR about the drill execution, pointing to any error that might have happened during the drill. For this purpose, the COACH extracts stored values from the feedback array and notes any omissions and discre- pancies in the way the drill was executed. Currently, the feedback consists of a simple voice feedback confirming that all tasks were completed, and no errors were observed. Once, the integration of IPME and ICGS is completed,
394 Zotov
Figure 1: IPME task template for order turret power-up drill.Figure 2: A sample of composite LAV crew model.
the details of the drill execution, including errors and mistakes, will be part of that feedback.
Army Crew Training: Coaching with ITS 395
Testing ITS for HBR-LAVs The testing step included running a human-in-the-loop scenario, using a human CC and two synthetic players: a COACH and a GNR. This scena- rio evaluated how the synthetic gunner would interact with a human crew member and tested the ability of the COACH to monitor and record the actions of the crew. The tested Power-Up module included IPME’s TNM and speech processing software as a plug-in to IPME. To start the human- in-the-loop scenario, an IPME run-time environment was instantiated with a “Turret Power-Up”task, which activated both IPME TNM and a speech pro- cessing module. As a result, a synthetic GNR and COACH were instantiated. The synthetic COACH initiated the Power-Up drill by saying “Crew Com- mander, proceed with the Power-Up drill”. A human CC commanded the synthetic GNR to proceed with its tasks and then both CC and GNR carried out their respective tasks, as outlined in the Power-Up TNM. As mentioned above, in the absence of connection to the ICGS LAV simulation package, there was no opportunity for CC and GNR to execute tasks (e.g., pressing buttons, turning switches, flipping levers, checking dials, etc.), thus they were carried out automatically by both positions regardless of whether it was pla- yed by human or AI. In order to get any feedback on performed activities, the GNR provided verbal confirmation of the executed task (e.g., “Radio is ready”). During the drill’s execution, a synthetic COACH monitored the actions of the crew. As was mentioned above, in the absence of connection to a simulation package there was no mechanism for the crew to activate any LAV controls, thus the COACH notes were empty by the end of the drill. Once all tasks in the drill were completed, the COACH initiated the After- Action Review, which, as was mentioned earlier, consisted of a successful confirmation of task completion. Nevertheless, to analyze the training effe- ctiveness of HBR-LAV it needs to be re-tested with the simulation package fully connected.
CONCLUSION
The work presented here aimed to extend earlier DRDC work on synthetic teammates for training by adding a synthetic instructor. The prototype LAV- ITS is a step in the development of a comprehensive application to simulate the roles of a drill instructor for training a LAV crew. This work demonstra- ted the viability of developing simple HBR models of CA combat vehicle crew members that can interact with one or more human crew members through voice commands, and which stands ready to be leveraged in future research efforts. Considering this effort of modeling crew members for a specific trai- ning application against the wider research on HBR modeling for training has demonstrated an example of HBR-based ITS that could be applied to a number of other applications of relevance to CAF training. These applica- tions revolve in some way around using HBR models to facilitate instances where team and collective training may be impeded by insufficient training resources, challenges with the availability of training audience members, or limitations of live training environments.
396 Zotov
ACKNOWLEDGMENT
The author would like to thank Brad Cain of the C3 Human Factors Inc. who initiated this work and developed many ideas and concepts of HBR modeling for DRDC. The contribution of MWO Darrell Bush, the Armoured Vehicle subject-matter expert who helped to refine the overall architecture of the LAV-ITS module was extremely valuable.
REFERENCES
Annett, J. (2003). Hierarchical task analysis. In Handbook of cognitive task design (pp. 41–60). CRC Press. Army Training Authority (2015). Canadian Army Simulation Strategy Version 2.0.
Canadian Army policy document. Cain, B. (2018). Light Armoured Vehicle Crew Model Development. DRDC-RDDC- 2018-C155, CAE. Cain, B., Magee, L., & Kersten, C. (2012). Validation of virtual environments
incorporating virtual operators for procedural learning DRDC Toronto TM 2011–132. Demir, M., McNeese, N. J., Cooke, N. J., Ball, J. T., Myers, C., & Frieman, M.
(2015). Synthetic teammate communication and coordination with humans. In Proceedings of the human factors and ergonomics society annual meeting (Vol. 59, No. 1, pp. 951–955). Sage CA: Los Angeles, CA: SAGE Publications. DND (2016). 25mm Turret drill manual (No. B-GL-393-014/FP-001) (p. 82). CTC Gagetown, NB: Chief of the Defence Staff, Department of National Defence, Army IG team. Eddowes, E. E., & Waag, W. L. (1980). The Use of Simulators for Training In-flight
and Emergency Procedures. NATO Advisory Group for Aerospace Research & Development Report AGARD-AG-248. Gunzelmann, G., Gaughan, C., Huiskamp, W., van den Bosch, K., de Jong, S., Alexander, T., & Tremori, A. (2014, December). In search of interope- rability standards for human behaviour representation. In Proceedings of the Interservice/Industry Training, Simulation and Education Conference (I/ITSEC) 2014. Kulik, J. A., & Fletcher, J. D. (2016). Effectiveness of intelligent tutoring systems: a
meta-analytic review. Review of Educational Research, 86, 42–78. MA&D (2015). Integrated Performance Modelling Environment (IPME) User Guide, Version 4.6. Alion Science and Technology. Microsoft Speech Platform (2022). http://docs.microsoft.com/ Pew, R. W., & Mavor, A. S. (1997). Representing human behavior in military simu-
lations: Interim report. National Academy Press. (Pew & Mavor, 1997), The Synthetic Cognition for Operational Team Training (SCOTT, 2001) for naval team training. Steenbergen-Hu, S., & Cooper, H. (2014). A meta-analysis of the effectiveness of intelligent tutoring systems on college students’ academic learning. Journal of Educational Psychology, 106, 331–347. Zotov, V & Jarmasz, J. (2019). Embedded Intelligent Tutoring System for Mark-
smanship Training: a Case Study. DRDC-Toronto, DRDC-RDDC-2018-R304.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Applied Human Factors Research
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-43-8
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional