INTRODUCTION
Cognitive overload is defined by the American Psychological Association (https://dictionary.apa.org/cognitive-overload) as the situation in which the demands placed on a person by mental work (the cognitive load) are greater than the person’s mental abilities can cope with. This construct is not new and has been well studied, yet we still do not know when it applies for many Army technologies. In many cases, artificial intelligent agents provided to support Soldier performance can be the cause of this performance degradation. This is the focus of this report.
Kirsh (2000) identifies four primary causes of cognitive overload: 1) too much information supply, 2) too much information demand, 3) constant multi-tasking and interruption, and 4) inadequate work-related infrastru- cture to help mitigate these causes. Though all four causes may apply to Army technologies, we expect that multi-tasking and interruption are the most prominent. The effects of multi-tasking (in the form of task switching) have been well established, producing both increases in response time and drops in performance accuracy (Rubinstein, Meyer, & Evans, 2000).
© 2023. Published by AHFE Open Access. All rights reserved. 108
A Proposed Methodology to Assess Cognitive Overload 109
For the sake of this study, we will focus on cognitive overload as the result of conducting multiple concurrent or sequential discrete tasks. This differen- tiates cognitive overload from other types of task difficulty. For example, multiplication is considered more difficult than simple addition, but that difficulty does not result from the introduction of concurrent or sequential discrete tasks.
The concept of cognitive overload is often interchanged with cognitive fati- gue or resource depletion as seen in explanations of the vigilance decrement (Helton & Warm, 2008; Rubinstein, 2020). Vigilance decrement expects one’s ability in detecting or attending to important information to falter the longer the attention is needed, and resource depletion expects working memory to falter when one overloads its cognitive processing. While resource depletion may coincide with cognitive overload, it is not a defining fea- ture. We define cognitive overload operationally as a decline in performance, with or without co-occurring cognitive fatigue or resource depletion. As an example of cognitive overload without any co-occurring resource depletion consider an operator given two tasks simultaneously. As seen in much of the dual-task literature, the secondary task would have to be put on hold while most of the primary task is being processed (Rubenstein, 2001; Schumacher et al., 2001). In this situation, performance degradation for one of the tasks was caused by task postponement and not any form of cognitive fatigue or resource depletion.
Army technological advancements provide Soldiers with more tools (e.g., sensors, targeting aids, maps, communications) across multiple technology platforms. Examples of these platforms include the Android Team Awareness Kit (ATAK), Infantry Visual Augmentation System (IVAS), Next Generation Combat Vehicle (NGCV), or the Next Generation Intelligent Fire Control system (NGIFC). These advancements are designed to improve capabilities across multiple domains (e.g., air, ground, sea, cyber) Despite the propo- sed benefits, these additional technologies create the potential for increased cognitive load and reduced Soldier performance. Additionally, the Army does not have the information needed to represent such overload effects on Soldier performance for modeling and simulation.
The Android Team Awareness Kit (ATAK) is a networked SA/communi- cations system used by ground combat Soldiers during forward operations. Currently, little to no human performance research has been conducted using ATAK. Therefore, this important research will provide a needed assessment of how the proliferation of add-on tools enhances or degrades Soldier per- formance. All reports thus far are focused on the technical processes and capabilities of the software (Usbeck, Gillen et al., 2015; Metu, 2014; Sadler & Metu, 2017), but not how they interact in an applied setting.
Some issues expected to be relevant are how many technologies can be deployed at the same time; how many apps can be used at the same time; how does the user know when information is updated; does the technology impede external awareness; etc. This paper will discuss how we plan to study this problem space by identifying the combinations of technologies that pro- duce cognitive overload, to be incorporated into a simulation environment (e.g., Soldier and Squad Trade Space Analysis Framework (SSTAF), Infantry
110 Patton and Rubinstein
Warrior Simulation (IWARS), One Semi-automatic Forces (OneSAF)). These existing human models do not incorporate real human data but rather use an algorithm of percentages to change behavior. The addition of real human data is imperative to more accurately model and simulate human behavior to best predict a potential outcome.
ATAK is a product of the US Air Force Laboratory, BBN Technologies, and Raytheon, and is an Android smartphone geospatial infrastructure and military situation awareness app. It allows for precision targeting, surroun- ding land formation intelligence, situational awareness, navigation, and data sharing. This Android app is a part of the larger Team Awareness Kit (TAK) family of products.
The applications or tools within ATAK include Blue Force Tracker, tacti- cal video, communications (voice and text), maps, annotations, point/icon placement, route and drawing tools, planning and measurement, range and bearing, route planning, jump master skydiving, collaboration tools, external storage media, networking, geospatial data and file sharing, sensor tasking and control, laser range finders, notification devices (e.g., smart watches), external GPS, and more. Details of these can be found in Usbeck et al., 2015.
The US Special Operations Command (USASOC) were issued ATAK in 2016. Anecdotal observations indicate that it is possible for a Soldier engaged with ATAK to lose sight of their immediate surroundings, requiring another Soldier to act as a guide on the real scene and keep them from running into unexpected objects. USSOC has expressed interest in understanding when the system is over engaged, creating cognitive overload, and have joined forces with DEVCOM Analysis Center, Human Systems Integration Division to better understand this phenomenon.
METHOD
We propose a methodology for evaluating cognitive overload effects in Sol- diers’ use of ATAK. We propose to work with the Special Operations Forces (SOF) to develop relevant scenarios that will require the use of ATAK and vary the number of applications needed to conduct the mission.
The mission scenario will require the squad to react to Scenario-Test Events (STE) in the form of planned events on the ground (e.g., detecting an enemy squad) and directives from command and on-the-ground Observer- Controllers (e.g., change route, take a casualty, lose a partner force). The STEs will be used to create real-world Army situations with which we assess the performance impact of multiple technology tools while using ATAK. The tem- poral proximity of these STEs, and thus the need to use the ATAK tools, will be manipulated to create two levels of task overlap and interference (spread out and overlapping).
We propose to use a 2 × 2 repeated measures design in which we compare 1) Soldier performance with vs. without ATAK, and 2) STE proximity (spread out vs. overlapping). For the without-ATAK condition, pre-ATAK procedures and technologies will be used (i.e., wrist Garmin, paper map overlays, Radio, SAT voice). The primary independent variables - ATAK vs. Radio, and STE proximity will result in 4 experimental conditions. The dependent measures
A Proposed Methodology to Assess Cognitive Overload 111
will include performance accuracy (across multiple mission tasks), time to complete the mission (and subparts), and measurable stress levels. We will also collect baseline stress, possibly trait stress, demographic data on all par- ticipants to include their time in service, military occupational specialty, and familiarity with the technologies that will be used in the study.
Twenty USASOC Soldiers will participate in this study. Squads comprising 10–12 Soldiers will engage in pre-scripted react-to-contact ground missions. Each mission will last approximately 30–40 minutes. Ten squad missions for each of the 4 conditions will be conducted, resulting in 40 squad missions. Each squad leader (who operates ATAK) will be randomly assigned to each of the four experimental conditions. Note that not all of the participating Soldiers will have the opportunity to serve as the squad leader. The order of the 4 conditions will be counterbalanced across the experiment.
Hypotheses: As the number of STEs (and thus ATAK tool use) occur closer in time, the following results are predicted:
1. performance accuracy decreases and response time increases, 2. stress levels will increase – as measured by sub-scales of frustration and depression.
CONCLUSION
Outcomes of this research will be Soldier performance data on how ATAK impacts Soldier cognitive overload, specifically when operational conditions mandate using ATAK tools in close temporal proximity. Based on which ATAK tools, or combinations of ATAK tools, cause the most cognitive over- load, mitigation strategies will be proposed. For example, Task Collapsing is the restructuring of a workspace or system to integrate several different tasks into a single technology tool (Kirsh, 2000). This would allow Soldiers to activate a single ATAK tool to complete multiple tasks that used to require two or more separate tools. It might even be possible to collapse multi-step operations into a single step if the associated mission events often co-occur. A cognitive task analysis could be performed to identify any patterns of ATAK tasks that co-occur frequently.
Another possible mitigation strategy could involve establishing a priority matrix to manage command expectations. Squad SA and communications requirements could be prioritized such that certain low-value tasks (e.g., asking for a sitrep) could be automatically postponed by ATAK when in close temporal proximity to high-value tasks (e.g., informing squad of problems with a partner force). Command-level ATAK interfaces (e.g., at a TOC) could also provide guidance of when the squad leader is not overwhelmed and is ready for low-value tasks.
The human data obtained from this effort will be available for use in Army human performance models. The current human models do not contain real data; they use theoretical data and percentages. For instance, if the model adds stress to an avatar, the model may slow the speed of the avatar by say 20%. The actual performance decrease may in fact be more or less than
112 Patton and Rubinstein
20% depending on the cause and actual amount of the stress. Therefore, understanding and gathering actual data for performance under stress will provide more valid values and more accurate modelling. This data will be provided to the Army M&S community for incorporating into models like SSTAF, IWARS, and OneSAF. The addition of this data will allow for more realistic human behavior for better model prediction of Soldier survivability.
REFERENCES
Helton, W. S., Warm, J. S. (2008). “Signal Salience and the Mindlessness Theory of Vigilance”, Acta Psychologica, 129, 18–25. http://dx.doi.org/10.1016/j.actpsy .2008.04.002, 2008 Kirsh, D. (2000). “A Few Thoughts on Cognitive Overload”, Intellectica, Volume 1
No. 30: 19–51. Metu, S. (2014). “Map-Based Querying for Multimedia Database”, ARL-MR-0874, Aberdeen Proving Ground, MD 21001. Rubinstein, J. S., Meyer, D. E., Evans, J. E. (2001). “Executive Control of Cogni-
tive Processess in Task Switching”, Journal of Experimental Psychology: Human Perception and Performance, 27, 763–797. http://dx.doi.org/10.1037/0096-1523. 27.4.763 Rubinstein, J. S. (2020). “Divergent Resposne-Time Patterns in Vigilance Decrement Tasks. Journal of Experimental Psychology: Human Perception and Performance, Vol. 46 No. 12, 1058–1076. ISSN: 0096–1523. http://dx.doi.org/10.1037/0096- 1523.27.4.763 Sadler, L. C., Metu, S. (2017). Intelligent Command and Control Demonstration
Setup and Presentation Instructions. ARL-TR-8244, Aberdeen Proving Ground, MD 21001. Schumacher, T. L., Seymour, J. M. Glass, D. E., Fenscik, E. J., Lauber, Kieras, D. E., Meyer, D. E. (2001). Virtually perfect timing sharing in dual-task performance: Uncorcking the Central cognitive bottleneck. Psychological Science Vol. 12 No. 2, 101–108. Usbeck, K., Gillen, M., Loyall, A., Gronosky, J. Sterling, Kohler, R. Hanlon, K., Scally,
A., Newkirk, R., Canestrare, D. (2015). Proceddings Volume 9456, Sensors, and Command, Cojntrol, Communications, and Intelligence (C3I) Technologies for Homeland Security, Defense, and Law Enforcement XIV; 94560R. https://doi.org/ 10.1117/12.2180014