Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Optimizing Human Capital Performance: Influence of Simulation

1 MOVES Institute, Naval Postgraduate School Monterey, CA 93940, USA ↗
2 U.S. Army Combat Capabilities Development Command Analysis Center Aberdeen Proving Grounds, MD 21005, USA ↗
3 US Army Research Laboratory Aberdeen, MD 21005, USA ↗
Publication year
Digital Object Identifier10.54941/ahfe1001495

Abstract

Simulations have been employed to train people and provide novel environments topractice and test new skills as well as experiment with new concepts and procedures.The US Department of Defense (DoD) spends millions of dollars each year to provideboth live and virtual training to military personnel. Realizing that simulations offer aplethora of opportunities, the DoD is now spending millions of dollars to design anddevelop what it believes will be the optimal versions of synthetic trainingenvironments to train its workforce. Each of the military services has a slightlydifferent view of how simulation will or should support them in the future. This paperaims to provide readers with insights about the needed human requirements and thepath that the services are on to achieve their future visions with respect to simulation.It will briefly discuss historical, functional, and future views of how simulations havebeen, are being, and are envisioned to support the optimizing of human performance.

INTRODUCTION

World War II prompted the analog and digital simulation era. Since then, simulators have been employed to train people as well as to provide envi- ronments for them to practice and test new skills, and experiment with new concepts and procedures. The individual military services within the US Department of Defense (DoD) spend millions of dollars each year to design and develop what they believe will be the optimal synthetic environments to accomplish these activities. Each of the military services has a slightly diffe- rent view of how simulation will or should support them in the future. The information that follows will provide readers with insights about the path that the military services are on to achieve their future visions with respect to simulation. Discussion of historical, functional, and future views of simu- lation are provided. This paper provides insights for the human factors and human systems integration professionals as to where their support may be most needed and useful.

© 2022. Published by AHFE Open Access. All rights reserved. 84

Optimizing Human Capital Performance: Influence of Simulation 85

THE OBJECTIVE: OPTIMIZING PERFORMANCE Optimizing the potential of the human capital within the DoD first requires identifying the critical and supporting human levers that must be adjusted and then determining how far those levers need to be moved for optimal perfor- mance to occur. Simple, right? The US Army has 481k active-duty personnel alone; adding government civilians and supporting industry teammates to the mix only ensures that the situation is extremely complex. Researchers conti- nue to search for answers to and debate the following human performance questions: 1. What is optimal performance? 2. What conditions/elements are necessary for optimized human performance? 3. How do we measure opti- mal performance? 4. How to do we use the data we have and the future data we will gather to improve performance?

The US military has investigated these questions for decades. While rese- arch must adjust to meet the demands of ever changing leadership, research titles, equipment, and adversaries the focus remains the same. We need to ensure that we have the right people in the right positions with the right abilities, knowledge, skills, and competencies to dominate any adversary at any time and in any location. This non-material work has been poorly fun- ded compared to research and development of material solutions. Little of this research is coordinated, nor is the data made available across research domains to aid in optimizing performance. Despite this situation, research is being conducted to determine the best ways to address and manipulate the variables impacting performance to ensure success. For example, the human dimension work initiated in the early 1970s has focused on under- standing how the physical, social, and cognitive areas of human capital could be optimized.

Holistic Performance Measurement: An Identification Problem Human performance metrics have been derived from decades of observa- tional research in the areas of personality (e.g. coping, efficacy), stress (as a stimulus and as a response), workload, neurophysiological, and psycho- physiological measures (e.g. heart rate variability) (Matthews et. al., 2000). Personality traits are stable over time, while states are temporary. Traits can be used for personnel selection, while states might be used to determine if an intervention could relieve stress (Napier, 2021). States as well as psycho- physiological measures have the potential to predict performance outcomes before they happen. For example, past research shows that self-efficacy and heart rate variability correlate with shoot-don’t-shoot decision making per- formance in military and police simulations when temporal demand is high (Patton, 2014; Napier, 2021). These results could be used to predict poten- tial hazards to the warfighter or police while being monitored during training (Napier, 2021).

Although there are general human performance metrics they are not neces- sarily easy to measure in the moment, for example, speed or accuracy. In addition, those metrics vary by individual and have complex interactions with other metrics. For example, the emotional stress response (or coping response) to a potential threat (or stressor) is multifaceted and it is well

documented that performance is impacted by the stress response to a stressor (Napier, 2021, Patton, 2014). “The individual’s personality traits, emotional states, physiology, coping abilities, and the characteristics of the task itself all combine to elicit different coping responses” (Napier, 2021) to a stressor and “makes it difficult to define exact linkages between stress and perfor- mance “(Napier, 2021). Simulators have the potential to help by allowing researchers to quickly and inexpensively replicate results and introduce subtle nuances to research that might produce changes in the way those multifaceted processes interact. Simulators can also be used to introduce novel situations that could radically change how individual differences interact and impact performance.

The simulated or virtual environment offers a bridge between the lab and field studies. Controlled lab situations may be confounded by limiting perti- nent factors, while the field research may be confounded by its lack of control. Simulation and virtual reality environments now provide researchers with the ability to study phenomenon in silico that previously were out of reach (Wei- sberg, 2013) and to assess physical and cognitive impacts of new equipment, how and where to implement AI/unmanned teaming, stress mitigation, and many other new areas (Napier, et al., 2016; Johnston, et. al, 2017; Pat- ton, 2014). However, in many cases how to measure performance in silico becomes an issue. The answer to that question may come from the training community and their use of simulations.

Human Performance: A Measurement Problem The military focuses on skills training, physical fitness tests, and unit trai- ning as their primary means to determine if individuals and teams are best prepared to accomplish their assigned missions. Measuring human perfor- mance has been studied by the training community for many years, and simulations have been used successfully to support the training of various military skills by individuals and teams (Johnston, et. al, 2015). While some skills like shooting accuracy might be easy to measure, skills involved in team processes (such as leadership or supporting behaviors) are not as obvious.

Methodologies developed can often be modified for use in modeling efforts and in designing new equipment (Napier, et al., 2016) and should be used to gauge performance in new areas such as AI/human teaming. Hall, Dwyer, Cannon-Bowers, Salas, & Volpe, (1993) have suggested several steps and ideas for identifying and measuring performance in simulators. These four steps, summarized here, can also be viewed as a guide to study performance without the use of simulation technologies. (1) Identify the skills, compete- ncy desired, or concept to be trained. Front-end analysis techniques such as the work done for the Integrated Training Environment Assessment Metho- dology (ITEAM) (Hodges, 2014; Hodges 2016), concept mapping, protocol analysis, cognitive task analysis, the critical decision method, cognitive netw- ork tasks (Hall, et al., 1993) and Systematic Team Assessment of Readiness Training (START) (Napier et al., 2016) are used in this step. (2) Design an event-based scenario vignette with help from soldier subject matter experts

Optimizing Human Capital Performance: Influence of Simulation 87

(SMEs). Event based vignettes are operationally relevant and have defined catalyst incidents that trigger the trainee to engage in the desired behavi- ors (Fowlkes, et. al., 1994). (3) Establish the performance standards using Mission Essential Task Lists (METLs) (Hall, et al., 1992). These standards should be observable and measurable and linked back to the scenario. They can also reflect skills that take place in the individual’s head. For exam- ple, leadership and situational awareness could be demonstrated by offering appropriate guidance. (4) In the case of new technology, a fourth step is added to the process. Soldier SMEs, technology designers, and human factors pra- ctitioners need to collaborate to describe how the technology and tasks will work together. For example, in the case of automation or AI there needs to be consensus on the tasks the AI/automation technology and crew will perform in the same events. This process can take considerable time, relies heavily on soldier SMEs, and requires skill to facilitate. Yet, these early interactions will yield a more robust system that has accurate measures of performance.

SIMULATION: A BRIEF OVERVIEW Simulation is defined by the Department of Defense as “a method for imple- menting a model over time” (DMSE, 2014). Generally, the history of analog simulation, and in particular games, can be traced back to a time before writ- ten history (Perla, 1990). More recent, yet still historical examples, are the first flight simulator (Link Box Trainer) (McFadden, 2018) developed to train pilots, the Corps Battle Simulation System (CBS) (Mertens, 1993) designed to represent conflict at the operational level of war to support training and expe- rimentation, and the Multiple Integrated Laser Engagement System (MILES) (MILES, 2000) used to represent individual weapons effects using lasers ver- sus lead bullets. Today, due to the power and capability of game engines, game environments are viewed by many as synthetic training environments of the future force. Gaming Simulation is defined as “the simulation of the effects of decisions made by players who assume the roles and represent the interests of real-life actors” (Klabbers, 2009).

The Live, Virtual, and Constructive Paradigm The military generally views simulation from three different perspectives: Live, Virtual, and Constructive. A Live simulation is one involving real people who operate real systems. An example of this would be a real tank and its crew executing gunnery on a live fire range. Virtual simula- tion involves real people operating simulated systems. An example of this would be a tank crew in the Advanced Gunnery Skills Training simulator (AGST) practicing the execution of a gunnery exercise BEFORE firing live ammunition. Finally, Constructive simulations involve simulated people ope- rating simulated systems. These types of simulation may run without any human intervention or may have real humans making inputs via input devices (computer keyboard/mouse) to modify the activities of the simulated people or equipment (DMSE, 2014).

MAXIMIZE SIMULATION: THE GOAL Ultimately, the Department of Defense would like the ability to maximize the use of simulations to achieve training, testing, experimentation, acquisi- tion, and analysis objectives to preserve resources (money), reduce risks, and minimize the wear and tear on people and equipment. However, most resea- rchers, scientists, and training experts agree that not everything can be done in a simulated environment…yet Accomplishing this goal requires the virtual environment to be further assessed to determine deficiencies that must be addressed. Although engineering efforts have focused on the technical aspe- cts of simulators it is also important to consider the various aspects of the human within the training environment. The START and ITEAM methods mentioned earlier are ways to link the human, simulator, and training goals by focusing on defining the tasks to be completed, the criticality of the task, what cues are needed by the human to trigger the task, and to what extent the simulator can correctly represent the task and the environment (Napier, et al., 2016; Hodges, 2014; Hodges, 2016). Using START or ITEAM early allows simulation developers to identify where best to use their limited resources.

Whether or not there is a good strategy for incorporating human needs into simulators each of the military services has initiated efforts to modernize and increase the use of simulation. As the services increase their reliance on simulations, they should pay attention to how the systems evolve overtime to improve training. The following paragraphs present some of these efforts that are specifically focused on improving human performance.

Military Service Initiatives The US Army has initiated The Synthetic Training Environment (STE). STE’s goal is to provide a collective, multi-echelon training and mission rehearsal environment. This environment should be capable of supporting the ope- rational, institutional, and self-development training domains. STE brings together live, virtual, and constructive environments into a single environ- ment at the Point of Need (PON) for Army Active and Reserve Components as well as civilians. STE will train all Warfighting Functions and the human dimension across all echelons including Joint and Unified Action Partners within the context of Unified Land Operations (https://asc.army.mil/web/p ortfolio-item/synthetic-training-environment-ste).

The United States Marine Corp is starting from scratch with their Live, Virtual and Constructive-Training Environment (LVC-TE). “LVC-TE is envi- sioned as a transformational capability that will federate diverse training and exercise programs to meet individual, unit, and collective warfighting requirements to maintain relevancy, agility, and adaptability.” (USMC, N.D.) The United States Air Force is heavily reliant on simulators. The Air Force’s “2035 Flight Plan” calls for developing a training environment that has the “right level of fidelity for the missions that airmen are preparing for”(Harper, 2020). Unlike the other services the United States Navy does not have a singular unified vision or program for how it will employ models and simulati- ons in the future. The Navy’s plan includes three simulated environments:

Optimizing Human Capital Performance: Influence of Simulation 89

(1) The Architecture Management Integration Environment (AMIE). “The AMIE Standard provides an enterprise non-proprietary method to integrate models and simulations across the acquisition lifecycle (DASN RDT&E, 2019); (2) The Joint Simulation Environment (JSE) a government off the shelf immersive virtual simulation that is a flexible, reusable, tailorable, and cost-effective solution for developmental, operational and interoperability testing (NAWCAD, 2018a); (3) The Next Generation Threat System (NGTS), that will continually evolve, provide a synthetic environment generator of threat and friendly targets and interactions in a realistic theater environment (NAWCAD, 2018b).

To the best of our knowledge, none of these efforts are being designed, developed, or delivered considering the others or with interoperability in mind.

CONCLUSIONS AND DISCUSSION Simulations will continue to be developed and used, therefore, if optimi- zing human performance is the next objective and maximizing the use of models and simulations to achieve that objective is the goal, then an orga- nized, researched, and well led program to get there is required. This paper discussed some appropriate methods, such as START and ITEAM that should be incorporated at the earliest stages of design and development to ensure that the simulations are robust and meet the needs of the mission. This paper has presented information on human performance and simulation and the context of their past, present, and future uses. We hope that by providing this perspective we have exposed some challenges and opportunities with employing models and simulations in the support of maximizing human per- formance specifically for the military. We hope that readers will contemplate how improvements to models and simulations may help us move past our current challenges and reach our objective.

DISCLAIMER AND ACKNOWLEDGEMENT The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein. This document is approved for public release.

REFERENCES

DASN(RDT&E), (2019). Department of the Navy Modeling and Simulation Architecture Management Integration Environment (AMIE) Standard. Navy Modeling and Simulation Office (NMSO). Defense Modeling and Simulation Enterprise (DMSE). (March 19, 2014). The DoD

Modeling and Simulation Glossary. https://www.msco.mil/MSReferences/Glossa ry/MSGlossary.aspx

Fowlkes, J., Lane, N., Salas, E., Franz, T., and Oser, R., (1994). ‘Improving the measurement of team performance: The targets methodology,’ Military Psychology. 6(1), pp 47-61. Hall, J.K. Dwyer, D.J., Cannon-Bowers, J.A., Salas, E., & Volpe, C.E. (1993). ‘Tow-

ard assessing team tactical decision making under stress: The development of a methodology for structuring team training scenarios’, Interservice/ Industry Trai- ning Systems and Education, Proceedings of the 15th Annual Interservice/Industry Training Systems and Education Conference Washington, DC: National Security Industrial Association. pp. 87–98). Harper, J. (2020) Air Force Locking on to New Training, Simulation Tech. National Defense Magazine, NDIA accessed 5FEB22. https://www.nationaldefensemagazine.org/articles/2020/8/28/air-force-locking-on-t

o-new-training-simulation-tech Hodges, G., (2014). A Novel Approach to Determine Integrated Training Environment Effectiveness. Proceedings of Interservice/Industry Training, Simulation, and Education Conference. Hodges, G., (2016). Using an Analytical Approach to Understand Integrated Training

Environment Utility. Proceedings of MODSIM 2016 Conference. Johnston, J., Burford, C., Napier, S., Henry, S., Ross, W. (2017). Implications of training effectiveness best practices for army soldier systems engineering archi- tecture. Proceedings of Interservice/Industry Training, Simulation and Education Conference. Johnston, J., Napier, S., & Ross, W. (2015). Adapting Immersive Training Envi-

ronments to Develop Squad Resilience Skills. Proceedings of Human Computer Interaction International. Klabbers, J.H.G. Terminological Ambiguity: Game and Simulation. Simulation & Gaming. 2009;40(4):446-463. doi:10.1177/1046878108325500 Mcfadden, C. (August 21, 2018) The World’s First Commercially Built Flight Simu-

lator: The Link Trainer Blue Box. https://interestingengineering.com/the-worlds- first-commercially-built-flight-simulator-the-link-trainer-blue-box Matthews, G., Davies, D., Westerman, S., & Stammers, R. (2000). Human performance: Cognition, stress and individual differences. Hove and New York: Psychology Press. Mertens, S. (1993). “The Corps Battle Simulation for Military Training,”Proceedings

of 1993 Winter Simulation Conference - (WSC ‘93), 1993, pp. 1053–1056, doi: 10.1109/WSC.1993.718358. Napier, S., (2021). A breathing intervention to enhance cardiac regulation and mitigate stress in police cadets. Unpublished doctoral dissertation. University of Central Florida. Napier, S., Best, C., Patton, D., & Hodges, G. (2016). ‘Using an augmented training

event to collect data for future modeling purposes’, In proceedings of Human Computer Interaction International, 2016. Toronto, Canada. Naval Air Warfare Center Aircraft Division (NAWCAD), 2018a. Joint Simulation Enterprise. www.navair.navy.mil/nawcad NAVAIR Public Release 2017–1012. Naval Air Warfare Center Aircraft Division (NAWCAD). 2018b. Next Generation

Threat System. NAVAIR Public Release 2018-930. www.navair.navy.mil/nawcad Patton, D. (2014, June). How real is good enough? Assessing realism of presence in simulations and its effects on decision making. In International Conference on Augmented Cognition (pp. 245–256), Springer, Cham. Perla, P. (1990). The Art of Wargaming. Annapolis: Naval Institute Press.

Optimizing Human Capital Performance: Influence of Simulation 91

Multiple Integrated Laser Engagement System (Accessed 15 FEB 22) Instrumentable- Multiple Integrated Laser Engagement System (I-MILES) - PEO STRI (army.mil) Weisberg, M. (2013) Simulation and Similarity: Using Models to Understand the

World. Oxford: Oxford University Press. upGrad KnowledgeHut. (February 9, 2022) Project Management Tutorial: Motivation Theories. https://www.knowledgehut.com/tutorials/project-management/mo tivation-theories USMC (N.D.) LVC. https://www.tecom.marines.mil/Units/Divisions/Range-and-Trai

ning-Programs-Division/LVC/ Accessed 5FEB22.

Cover of the proceedings volume Human Factors and Simulation

Explore the complete digital volume

Human Factors and Simulation

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-06-3

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →