Michael Jaye, Patrick J. Driscoll, Donnie Choe, Victor Diaz, Anthony Keller (2019). Soldier Narrative Analysis as Part of a Rapid Fielding Program. In: Advances in Cross-Cultural Decision Making. AHFE Open Access, vol. 6. https://doi.org/10.54941/ahfe100183
The Warfighter Technology Tradespace Methodology (WTTM) employs technology acceptance and decision analysis theories, applying them to systems for developers to garner accurate understanding of end user expectations and concerns. WTTM applies a tradespace construct that is reliant upon three primary components as input – technical performance, logistic supportability, and human factors. The intent of this tradespace approach is two-fold: (1) to understand and assess the current state of systems as solutions to combat outpost challenges; and (2) to provide information to system developers participating in the activity as to how to tailor implemented design elements to better align with warfighter expectations, thereby maximizing the likelihood of technology acceptance upon fielding. To inform that process via a feedback mechanism, we gathered and analyzed narratives from soldier end users. We collected and tagged soldier sense-making items regarding newly fielded equipment, creating metadata for quantitative research and analysis. Here, we present the collection process as well as the results of sense-making analyses of narratives obtained from soldiers who worked with several pieces of newly fielded equipment. The intent is to highlight user dispositions as well as those factors influencing acceptance or rejection of the newly fielded equipment.
The Deployable Force Protection Technology Focus Team (DFP TFT) was created in 2010 at the direction of the Director, Defense Research and Engineering (DDR&E) to focus on force protection needs for troops operating in small, expeditionary bases (less than three hundred personnel), including at outposts that might be integrated with local communities. Since its inception, efforts have been directed to enhance remote operations at small bases with emphasis on capabilities that enable troops to detect, assess and defend against threats. Among the issues under investigation include holding off attackers at small bases, increasing the ability to identify shooters, return fire, and reduce response time, preventing perimeter breach, protecting critical assets, reinforce buildings to protect warfighters from blast and ballistics, assess changes to the operational environment, etc.
As a sub-organization of the DFP-TFT effort, the DFP Adaptive Red Team (ART) instituted scenario-based Technical Support and Operational Analysis (TSOA) live experiments to identify, immerse, and assess systems in operationally relevant, rigorous environments. A core component of all TSOA activities is a focused warfighter Cross-Cultural Decision Making (2019) engagement with all the technology systems that participate. This warfighter engagement, which takes place on a continuous basis over several days, is intended to uncover strengths and weaknesses related to field performance and usability as well as system vulnerabilities. TSOA events afford an early look at warfighter-defined critical capabilities at a point in the system development lifecycle where changes in design and implementation are welcome and affordable.
The ART’s research and development group is the academic underpinning of TSOA. Its purpose is to develop, identify, and adapt best practices, tools, and techniques from systems engineering, logistics systems integration, human systems integration, and modeling & simulation in order to identify vulnerabilities that could threaten successful fielding, use, and operational effectiveness of potential DFP technologies. The WTTM assessment framework is a product of this group’s effort. It supports comprehensive, validated, quantitative and qualitative assessments to identify new systems’ vulnerabilities and risks. Structurally, it functions as an accurate cognitive mapping from observational data to quantitative scales with supporting text-based comments. What is absent in the current WTTM process, is a means of directly leveraging and analyzing the human narratives initiating this mapping sequence. The approach presented in this paper, which relies upon a sense-making construct, attempts to close this gap in a complementary fashion that exposes valuable insights currently implicit while retaining the WTTM effectiveness that has been successful over time.
HUMANS AS STORY TELLERS Fisher’s narrative paradigm offers an approach to assessing human communication which is “fundamentally stories, as interpretations of aspects of the world occurring in time and shaped by history, culture, and character” (Fischer, 1984). Fisher claims that humans are at their essence story telling beings, homo narrans. Others such as Franzosi suggest that stories constitute the basic form of communication through which ideas, concepts, past experience, and interpretations are conveyed (Franzosi, 1998).
Storytelling is a uniting and defining component of all individuals and communities (Snowden, 1999). Storytelling helps the individual to make sense of personal experience and individual perspective. This facilitates the transfer of knowledge and learning – sensemaking, and it is the fundamental mechanism in understanding narrative (Snowden, 1999). Sensemaking, a method in which humans integrate experiences, insights, etc., into distinct stories or narratives, often takes a complex problem or idea and deconstructs it in a way that is understandable. Sensemaking might also be thought of as a problem solving process, either collaborative or personal, that depends on perspective and interpretation (Kolko, 2010). Sensemaking has been applied to multiple fields such as: communications, intelligence, organizational studies, mathematical modeling, artificial intelligence and informatics pattern recognition (Martin & Sturmberg, 2013).
Soldier Narratives – a Novel Means to Inform the WTTM Process We proceed by accepting that humans communicate narratives to influence one another, to capture events, and to understand and express simple and complex ideas to each other.
Yet the role that storytelling and narratives play in various aspects of warfare, especially how they shape soldier attitudes and behavior, is an understudied phenomenon. Soldiers’ stories reflect their deconstruction of complex problems, and they constitute a rich source of information. The information embedded within a soldier’s narrative is predominantly qualitative, yielding access to subjective experiences, thereby providing insights into new conceptions and opening up new perspectives pertinent to equipment and system assessment.
As such, soldier narratives and stories have a capacity to illuminate the sense that they have made of their experiences, including their work and dealings with new equipment. Moreover, this information naturally represents multiple perspectives, diverse experiences, and creative insights regarding a particular piece of equipment. This expands the aperture of pertinent and available information. In addition to providing equipment manufacturers specific areas for equipment improvement, narratives can serve to corroborate and validate WTTM findings and/or identify WTTM shortcomings.
Collection Methodology
Using in-depth/unstructured interviews to collect narratives or stories enables a researcher to isolate attention toward a specified population with similar life history experiences or oral history. Story collecting can be executed in a variety of ways, but the most ideal when attempting to attain sensible and self-produced data is through written stories. Story telling through writing does not adhere to the traditional pattern of a narrator telling a story from the beginning to the end, but it enables the story teller to provide a spontaneous, uninterrupted description without external complementing or sometimes aborting influence (Czarniawska, 2004).
Consequently, we conducted a series of in-depth and unstructured interviews with soldiers from several military units (in compliance with IRB protocol and approval). We collected their written stories and then asked them to modify their stories through a series of signifying questions. This resulted in a body of empirical and qualitative data to be used for analysis and feedback to the equipment fielding process.
We interviewed 279 soldiers from three units that were chosen based on their availability within the Rest- Training-Deployment cycle. Participants were selected based on a stratified random sampling of each battalion’s demographic; the main criterion for inclusion in the research was the participant’s current assignment to the unit and experience with a variety of equipment. Data collection was conducted in separate engagements with three different groups from May to December 2013. There were no discriminating factors to disqualify potential test subjects from participating in the research such as education levels, rank or military experience, or military occupational specialty (MOS). As long as the test subjects were part of the units sampled and were willing to volunteer, they were free to participate. Similarly, the test subjects were informed that they could discontinue their participation at any time, even after they had provided their input. To protect the test subjects’ privacy, all personal information was removed from the collected data in accordance with U.S. government institutional review board (IRB) protocol. Additionally, any information identifying specific unit, location, and mission was removed in accordance with the Department of Defense operational security policy.
The collection instrument consisted of a PowerPoint file that consisted of 25 slides, five of which were instructional. The remaining slides were designed for conveying, then signifying and amplifying the narrative. In order for the interview participant to feel more at ease and comfortable in the unscripted environment, grammatical and spelling errors were ignored. Font size, theme, and color were not prescribed (the intent being the creation of a comfortable environment that might afford free-thinking). Immediately after entering their narrative, the participant was asked to give a title of his or her story. This was intended to reinforce personal ownership of the narrative.
The following prompt was used to elicit all participant narratives: Describe an event with the newly fielded equipment that you were a part of, that you have witnessed, or that you know about. Talk about one day, one incident, or a personal experience. What happened? What came out of this event? What would you tell others who would be using that equipment in the future? What would you tell the equipment developers that best conveys your satisfaction or dissatisfaction with the new equipment? What do they need to know? How easy was it to adapt to the new equipment? What were the particularly memorable aspects of the new equipment?
Once participants had entered their narrative and given it a title, they next answered a series of multi- dimensional signifier prompts known as triads and diads. Triads are triangulated spaces with specified vertices pertinent to a prompt that allow for amplification of the respondent’s narrative in dimensions of interest. They can focus in detail on certain qualities of the service member’s experience that may not have been illuminated in their narrative, and they can give the soldier the ability to reinforce certain ideas (Snowden D., 2013). Diads are similar but have one less specification vertex.
Participants were prompted with a question related to their narrative and then asked to drag a circle to a location within the triangle (or place along a scale in the case of diads). This afforded participants an opportunity to amplify his or her narrative: positioning the circle within the triad (or diad) allowed them to signify a response that best described their view relative to the prompt and the story just told. The closer to a corner, the stronger that description is taken to match the context of their story. A bubble located in the center signifies that all three aspects are equally present in their story. If the participant believed that there was no associated relationship, he or she was given the option to move the circle outside of the triangle, stating that the triad (or diad) was non-applicable to his or her narrative. Cross-Cultural Decision Making (2019)
The participants’ stories spanned a broad spectrum of operational system and equipment use. As such, narrative analysis offers a source of valuable information that can be used to identify potential capability gaps and technological solutions to fill those gaps. One limitation is worth noting. Because relatively few systems have been fielded directly from the TSOA/WTTM process, we were able to identify only a small subset of participants (11) who had worked with TSOA-assessed systems. Nonetheless, as we demonstrate in what follows, those narratives are quite informative despite their small sample characteristic.
NARRATIVE ANALYSIS We analyzed participant stories using Cognitive Edge’s SenseMaker ™ and SenseMaker Explorer™. From the collected narratives and their amplifications/significations (through Triads and Diads), we created a metadata file consistent with formats used by SenseMaker Explorer™. Then SenseMaker Explorer™ was used to investigate and interpret the data in a variety of ways (e.g. the individual respondent, the entire data set, a specific category, or specific item). SenseMaker Explorer™ provides various investigative options allowing for the separation of the data by filters or themes as well as their sub-components in text form. Each of the themes is associated with a Triad or Dyad, and they can indicate quantity of stories related to that specific Triad or Dyad. One can investigate specific narratives associated a particular response and we could also read each story if categorized by theme and sub- components.
It should be noted, however, that the SenseMaker suite of programs does not perform semantic analysis of the narratives. Consequently, narrative processing was not automatic - we had to read and understand participant stories in addition to whatever analysis SenseMaker might provide.
RESULTS
Figures 1 through 5 display the summary results of participant input and signification in response to narrative interview prompts. Figure 1 depicts an example of the information produced by SenseMaker Explorer™, in this case it summarizes participants’ signification (or amplification) of their narratives in response to the prompt: “Based on your story, what aspect of the equipment could be improved the most?” Data displayed in Figure 1 reflect responses from over 200 participants. Each data point is associated with an individual participant’s narrative and his/her interpretation of how the equipment they described in their story fit the prompt. Because the equipment described in those narratives was so diverse, one might expect the resulting signification to be spread somewhat uniformly among the three corners of the triangles vertices, as observed in Figure 1.
Figure 1. An example of SenseMaker Explorer output gathered from a particular narrative signification question, "What aspect of the new equipment could be improved the most?" In this example, signifiers from all narrative entries were considered in the sample space.Figure 2. SenseMaker Explorer output from Figure 2, but now with narrative fragments depicting
soldier commentary associated with various signification concentrations.
Figure 2 depicts the identical information as Figure 1, but now with selected or representative participant commentary identified with its associated signification (or amplification) concentrations within the triad. Portions of the narrative fragments relating to specific equipment, names, functionalities, etc., are redacted yet still convey soldier intent. The ability to mine the data set to obtain this type of representation offers insight into the data and its meaning that would not be obtained through typical surveys that employ more traditional queries such as Likert scales.
Figure 3 is another example of triad signification of participant narratives. Figure 3 displays signification and representative responses to “The benefit to your unit provided by the equipment in your story was in…” (upper figure) and “The equipment in your story failed to improve…” (lower figure).
Figure 3. Participant signification and representative responses to the signifying prompts "The benefit to your unit provided by the new equipment was in…" and "The new equipment in your story failed to
improve…"
Figure 4 conveys the results from the subset of participants who had worked with a piece of equipment that has been subjected to the TSOA/WTTM assessment process. The equipment’s basic functionality is providing security to units engaged in combat operations, and this is corroborated by the massed location of the soldier significations as well as in their narratives.
Figure 4. Participant signification and representative responses to the signifying prompts "The benefit to your unit provided by the new equipment was in…" and "The new equipment in your story failed to improve…"
Another dimension that the participants considered in amplifying their narratives regarding this piece of equipment was how it could be improved the most. Resulting signification and associated narrative fragments are depicted in Figure 5. Interestingly, not only is the relative weighting of the groupings consistent with TSOA/WTTM findings (i.e. training requirements are too burdensome, mostly due to the ill-conceived user interface; logistic and technical performance are, relatively, much better), but many of the narrative fragments are consistent with TSOA assessments and commentary as well. Moreover, while the results here are not intended to be conclusive, they do provide valuable indicators regarding potential success upon fielding – narratives appear to indicate support for acceptance based on performance if the barrier associated with burdensome training requirements could be overcome.
CONCLUSIONS
Obtaining and analyzing narratives based on participant experiences with fielded systems and equipment provides valuable insights into performance expectations that can inform both capability and design choices. In addition, significations/amplifications of these narratives combined with associated narrative fragments facilitates a deeper and richer understanding than that accessible through traditional elicitation approaches such as Likert-based surveys. Preliminary findings as reported in this paper appear to indicate that, in the case of at least one newly fielded piece of equipment, the WTTM process is capturing information that is consistent with end user experiences with that equipment as expressed in their stories and amplifications/significations. Thus, narrative analysis can serve as both a stand-alone process or as a complementary approach against which validation and consistency checking can occur.
REFERENCES
Czarniawska, B. (2004). Narratives in Social Science Research. Los Angeles: SAGE Publications Inc. Fischer, W. R. (1984). “Narration as a human communication paradigm”, in: Communications monographs, vol 51, pp. 1-22. Franzosi, R. (1998). “Narrative Analysis-Or Why (And How) Sociologists Should be Interested in Narrative”. Annual Reviews, vol 24, pp. 517-554.
Kolko, J. (2010). “Sensemaking and Framing: A Theoretical Reflection on Perspective in Desgin Synthesis”, in: Design Research Society; Creative Commons. Snowden, D. (1999). “Story telling: an old skill in a new context”. Business Information Review, vol 16 No. 1, pp. 30-37.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Advances in Cross-Cultural Decision Making
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-4951-2095-4
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional