Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Using the Decision Ladder to Understand Human Decision-Making Processes During a UAV-Equipped

1 Faculty of Engineering, University of Bristol, UK ↗
2 Thales, Research, Technology and Innovation, Reading, UK ↗
Publication year
Digital Object Identifier10.54941/ahfe1004123

Abstract

The uptake of Uncrewed Aerial Vehicles (UAVs) within the Search and Rescue (SAR) application is becoming ubiquitous owing to the way in which UAVs can support and extend SAR responses. During a SAR mission, UAVs are deployed to offer a birds-eye perspective of the search space which is captured by the onboard payload sensors and transmitted to human operators in real time. Currently, this data stream is processed manually by a Payload Operator to identify any hazards or signs of human life on the ground. However, the task of identifying and extrapolating information using the display technology is highly challenging. This is primarily due to the high levels of cognitive effort which must be expended over time to detect sightings that are likely to be camouflaged or obscured by the surrounding terrain. For this reason, system engineers are looking to develop image classification modules capable of autonomous object detection and labelling to streamline the information acquisition process from the UAV. When looking to introduce novel functionality, such as an image classification module, it is important to consider how decision-making processes are currently undertaken by the team of human operators. In doing so, a greater understanding is yielded on how the current ways of working could be supported through further design intervention. The current work aims to capture these processes using the representational medium of the Decision Ladder. Rasmussen (1974) developed the Decision Ladder to define the different information processing activities undertaken by a decision-maker when identifying an appropriate course of action within a given situation. In order to develop the Decision Ladder for the UAV-equipped SAR scenario, knowledge elicitation activities were conducted using interviews with SAR personnel.The final amalgamated Decision Ladder was populated using the responses from the interviews. The decision model demonstrated the complexity of utilising UAVs to support a SAR mission due to the regulatory and technological constraints associated with the UAV. In addition, the importance of a validation activity was emphasised by the SAR personnel which would be conducted to determine the accuracy of any information presented by the automated system. Here, the decision ladder was able to identify the broad set of information aspects that would be reviewed by the human-UAV team. The subsequent knowledge obtained would be used to identify the relevance of a sighting and determine the most appropriate response using the limited resources available within the SAR environment. This insight provided from the Decision Ladder was used to propose a set of novel design recommendations that could extend the capabilities of the image classification module within the SAR context. Therefore, this work advocates the use of a user-centred design approach to support the development of technologies based on the tasks and cognitive processes of the end-user.

INTRODUCTION

The operation of Uncrewed Aerial Vehicles (UAVs) in goal-orientated con- texts such as Search and Rescue is complex. Throughout the UAV operation, the human operators are separated from the air vehicle but must maintain visual line of sight. As a result, the Human Machine Interface (HMI) serves as the primary information resource for ensuring the UAV is operating safely, efficiently, and incompliance with regulations that set operational boundaries for the airborne parameters of the vehicle (e.g., Civil Aviation Authority; CAA, 2023). The HMI must present the required information to the human operators at the right time to ensure decision-making does not become erro- neous or ineffective (Riley et al., 2010; Lynch et al., 2022). However, all too often there is a tendency to design UAV systems using technical requirements that fail to consider who is interacting with the system, and more importantly how (Steane et al., 2023 in press). This tendency has given rise to HMIs that are complex and rely on the skill and technical knowledge of the human

© 2023. Published by AHFE Open Access. All rights reserved. 133

operator to exploit the capability afforded by the UAV technology (Golightly et al., 2022). In doing so, a high reliance is placed on the human operators responsible for controlling the UAV, and any benefits of autonomous func- tionality may be lost. To manage this, novel algorithmic developments and improved sensor technologies are being designed to improve the level and quality of information available to the human-UAV team (Goodrich et al., 2007; Parnell et al., 2022). The goal of adding this functionality is to assist users in reducing workload, maintaining situation awareness and reducing the risk of errors.

It is during these early design stages that Human Factors Integration (HFI) methodologies can be leveraged to transfer the user’s needs into system requirements for use by system architects and software engineers (Bruseberg, 2008). The current work aims to demonstrate the utility of exploiting Human Factors (HF) methodologies to extrapolate such insight. Specifically, the use of the Decision Ladder was examined. We used the case study of SAR due to the recent interest in integrating image classification algorithms within UAV systems to detect and label targets within the terrain. It is anticipated that such functionality would reduce the workload placed on the human opera- tor responsible for interpreting the payload data received from the UAV (i.e., the Payload Operator). However, when designing these support mechanisms, it is important to consider where they can be extended to present the informa- tion needed to manage the UAV and support the goal of the wider SAR team. In other words, the goals and tasks encompassed within the work domain should be considered when designing automated decision aids (O’Neill et al., 2020). The Decision Ladder investigates key tasks in complex sociotechnical systems (Vicente, 1999) whilst also analysing the activities that take place in “decision-making terms” (Rasmussen et al., 1994, p. 58). As such, the Decision Ladder is employed in the current study to support the integration of automated functionality by eliciting insight into why, who, when, where, what and how a decision aid would be used in the SAR application (Steane et al., 2023 in press).

A template of the decision ladder is shown in Figure 1. The model is comprised of two types of node: rectangle nodes represent the information processing activities conducted by the decision-maker and circular nodes rep- resent the resultant states of knowledge (McIlroy and Stanton, 2015). The Decision Ladder begins with an alert in the environment which prompts the decision-makers to assess and respond to a situation. The left-hand side of the decision ladder shows the subsequent situational assessment made by the human actors through a means of gathering information and identifying the options available to respond to the initial alert (Banks et al., 2020). Once the optimal option is identified through the application of knowledge-based processing, the right-hand side of the ladder displays the planning and execu- tion of this response. Although the Decision Ladder is shown as a sequential framework, a decision-maker equipped with a high level of expertise could shortcut aspects of the ladder to enable faster decision-making and more deci- sive action (McIlroy and Stanton, 2015). This is represented using two cog- nitive shortcuts. Firstly, a leap is a shortcut between two states of knowledge and is shown by the dashed line (see Figure 1). This shortcut occurs when a

Using the Decision Ladder to Understand Human Decision-Making Processes 135

state of knowledge triggers another; for example, by understanding the sys- tem’s state, the decision-maker may already know what task to perform based on past experiences. The second shortcut is referred to as a shunt and occurs when the decision-maker requires further information from the environment in order to carry out a task. It is shown as a solid line between an information processing activity and state of knowledge (see Figure 1).

Figure 1: Decision Ladder template adapted from Parnell et al. (2021).
Original PDF Figure 1:

METHOD

In order to populate the Decision Ladder, semi-structured interviews were conducted with operationally active members of SAR units across England, Scotland and Wales. The structure of the interview was designed using the Schema World Action Research Method (SWARM; Plant and Stanton, 2016). This methodology was developed to measure the perceptual processes of aeronautical pilots. However, Plant and Stanton (2016) envisaged its use being extended beyond the domain of crewed aviation. The full SWARM repository comprises 95 cognitive prompts. Each prompt was designed based on the sub-types of the Perceptual Cycle Model (PCM; Neisser, 1976). Plant and Stanton (2016) recommended down-selecting and adapting the SWARM prompts based on the objectives of the study. As such, 30 and 33 SWARM

prompts were selected for use in the current work for the traditional ground search and UAV-equipped interviews, respectively.

Participants A total of five participants were recruited from Mountain Rescue Teams across England, Scotland and Wales to take part in the operator inter- views. Participants were currently operational SAR volunteers with varying amounts of operational experience. Within the participant sample were indi- viduals with experience of working as a ground searcher, UAV Pilot and/or Search Manager. Those who had experiencing of UAV piloting held the rel- evant qualifications required by the CAA for operating UAVs within civil operations. The study received ethical approval from the University of Bristol Ethics Committee (Reference 10785).

Procedure Prior to beginning the interview, participants read through the participant information sheet and, once they were happy to take part, signed the consent form. At the beginning of the interview, participants were shown a hypothet- ical SAR scenario involving an individual reported as missing in a National Park. Once the scenario was presented, the interviewee went through the set of SWARM prompts selected for inclusion. The participants described the approach that would be taken to respond to the missing person case when equipped with or without a UAV. In order to understand how the decision- making processes of the UAV team could be supported by a decision aid that displays image classification information, at the end of the interview, the following questions were asked:

“If the UAV could provide a confidence estimate for the image classifica- tion…

1. How would you interpret this information? 2. Would it be a useful piece of information? 3. Is there any other information that would be useful in assisting with image classification.”

Each interview was recorded using a Dictaphone to enable for data transcription and subsequent analysis.

DATA ANALYSIS AND MODELLING The Decision Ladder was developed iteratively following a period of data collection. Two Human Factors researchers were involved during the devel- opment of the Decision Ladder. Once the researchers reached a point of agreement, the final amalgamated model was sent to an independent Subject Matter Expert which served as a validation measure to ensure the models accurately depicted the practice of SAR teams.

Decision Ladder for a UAV-equipped SAR Response The final amalgamation of the Decision Ladder for a UAV-equipped SAR response is shown in Figure 2 and begins when the Payload Operator identi- fies a potential sighting on their display. It was recognised that during both

Using the Decision Ladder to Understand Human Decision-Making Processes 137

a UAV-equipped response and a traditional ground search, the goal of the system would always be to exploit any new information received from the environment to adapt the current plan in a way that increases the likelihood of locating the missing person (MISPER). Indeed, any information presented from the UAV would likely lead to further human involvement through the re- allocation of resources by the Search Manager responsible for controlling and directing the search effort from a control vehicle. Yet, prior to re-allocating any assets or teams, it is important that the finding identified on the dis- play is subject to a validation process to establish its reliability and relevance to the search. Whilst an object found on the ground would also be subject to a similar assessment, the validation process used during a UAV-equipped response was more complex due to the limited availability of information on the display.

The sparsity of resources in the SAR environment implicates the need to gather evidence to facilitate informed decision-making by ensuring the data collected by the UAV corroborates the information already known about the MISPER. For example, the last known location of the MISPER and the cur- rent location of the UAV may be reviewed in tandem to calculate the distance travelled between sightings. If the MISPER had travelled a respectable dis- tance but was an elderly individual, the sighting may be ruled as irrelevant under the assumption that they would be incapable of carrying out such a journey due to their physical condition. Other information aspects that could be attended to include the environmental surroundings shown in the imagery. Both the terrain and its environmental stimuli (e.g., trees) provide a basis on which to infer the sighting’s size, shape and movement. These characteris- tics provide further contextual information that aid in the classification and initial validation of the sighting.

It was also found that the Payload Operator could support the Pilot by reviewing the status and health of the UAV to ensure the vehicle is safe during its deployment and can continue to be used as part of the response. This involves monitoring parameters such as the battery life to determine the time left until the battery reaches a critical level. Finally, information would be reviewed to understand the availability of resources in the SAR environment that could be used to aid the response. The combined understanding of the sighting’s relevancy, the state of the SAR system, and the health of the UAV would be used to identify a set of options that could be used to respond to the finding. This decision falls to the Search Manager; an individual considered to hold a significant amount of experience to effectively coordinate the search effort. For this reason, the Search Manager may intrinsically recognise the optimal course of action to validate the UAV data once the system state is diagnosed. This shortcut is displayed as a leap from the ‘system state’ node to the ‘task’ node that resides on the right-hand side of the Decision Ladder (see Figure 2).

The procedure shown on the right-hand side of the ladder outlines the processes conducted when re-tasking the UAV to investigate a sighting. Ordi- narily, the UAV would land or be manoeuvred closer to the location where the sighting was originally identified. Subsequently, the Pilot would safely navigate the UAV within the search region whilst the Payload Operator mon- itors the display for the initial sighting. During the vehicle’s deployment the

Figure 2. Condensed version of the Decision Ladder. Note. Text in blue indicates information used specifically for UAV-equipped missions.

Figure available in the original PDF paper.

Using the Decision Ladder to Understand Human Decision-Making Processes 139

UAV team could refer back to information on the left-hand side of the lad- der. For example, the location of the sighting may be reviewed to maintain awareness of where to navigate the UAV and allocate attention when looking at the aerial imagery. This referral is depicted as a shunt between the ‘plan procedure’ processing activity and the ‘information’.

It is also critical for the UAV team to maintain awareness of the systems state, such as its parameters, to ensure the vehicle is adhering to regula- tions stipulated by the CAA. These regulations currently prevent the UAV being flown beyond visual line of sight and more than 120 metres from the earth’s surface (CAA, 2023). The continuous monitoring of these parameters is shown as an additional shunt between the ‘plan procedure’ activity and ‘system state’ information node.

In the context of UAV-equipped SAR missions, it is important to demon- strate the flexibility of SAR responses. This is because new information is used to shape and guide the direction of a search effort, meaning the planned procedure is always subject to change. To encapsulate this within the Deci- sion Ladder, a link was placed between the ‘plan procedure’ and ‘execution’ nodes to highlight the way in which a plan could be altered following the reception of new intelligence or changing circumstances in the environment (see Figure 2). An additional link was also embedded between the ‘execu- tion’ and ‘activation’ nodes to demonstrate the cyclical nature of SAR (see Figure 2). Fundamentally, SAR involves continuously adapting and carrying out the plan using information from the search teams until an outcome is reached. Such outcomes could see the MISPER being rescued or the mission being abandoned when the safety of the SAR team is at risk.

DISCUSSION

The design recommendations identified using the Decision Ladder are shown in Figure 3 with the potential cognitive shortcuts that could be enabled fol- lowing their integration. First, the implementation of automatic functionality for imagery analysis (i.e., object detection, classification) could manifest in a shortcut between the ‘alert’ to ‘task’ node. This is because the presenta- tion of a detected sighting may trigger an intrinsic understanding of the task needed to respond to the potential finding. Nevertheless, SAR operators indi- cated that the findings of an automated system would likely not be taken on face value. Instead, further information would be gathered to corrobo- rate the relevance of the sighting and determine an appropriate response. In order to support these information gathering activities, the decision aid could utilise the information aspects used to review a sighting’s relevancy (e.g., size, motion, speed, distance travelled) by presenting these to the Payload Operator alongside the automatic detection notification. This support may facilitate the second shortcut between the ‘information’ and ‘task’ nodes.

The support mechanisms presented thus far would not enable an intrinsic recognition of the required procedure as no SAR mission involves the same type of MISPER. For this reason, the procedure used to respond to a sight- ing must be reviewed on a case-by-case basis to determine the most suitable response. For instance, it may not be appropriate to navigate a UAV within the vicinity of a vulnerable individual as the noise generated from the vehicle

may invoke feelings of fear in the MISPER, causing them to conceal them- selves from the SAR team. However, a decision aid presenting the options available to the Search Manager could help determine which resource to allo- cate by displaying the set of resources available for re-tasking. These options could be displayed in order of their appropriateness based on factors such as the remaining UAV battery, mission timing requirements and the type of MISPER profile being dealt with during a mission. In doing so, a shortcut may be enabled between the ‘options’ and ‘procedure’ nodes.

CONCLUSION

The uptake of ‘off-the-shelf’ UAVs for SAR missions has provided an invalu- able technological tool for aiding human responders operating on the ground. Nevertheless, the generic design of these systems has resulted in UAV teams adopting strategies to manage the technological constraints associated with the technology. These strategies extend to the analysis of the aerial imagery, navigation management, status and health monitoring, and the communi- cation of information with search management personnel. It should also be recognised that these strategies evolve quickly, with operators finding adap- tive techniques and refining their Standard Operating Procedures to manage emergent issues.

The design recommendations proposed in the current work aim to pro- vide support mechanisms that could help manage each of these aspects. By identifying how work is currently done using the Decision Ladder, it became possible to consider: (i) why intervention is need; (ii) who would utilise the support; (iii) when the support mechanism would be used; (iv) what it would be used to achieve; and (v) how this would be embedded or modified to fit current practice (Steane et al., 2023 in press). Even so, it is important that these models are viewed as representations of SAR practice as opposed to a fully fledged descriptions of UAV-equipped SAR missions. For this reason, further validation of the Decision Ladder is required using empirical methods to ensure the assumptions from this work accurately represent SAR practice (Banks et al., 2020; Parnell et al., 2021). Moreover, future work could extend the current research effort by modelling the practice of SAR teams for more specific UAV mission vignettes to encapsulate the broad range of techniques used to embed UAVs within SAR operations (see Table 1 for a set of mission vignettes developed using insight from UAV operators).

Table 1. UAV mission vignettes.

Table available in the original PDF paper.

Description UAV mission vignette

Sound sweep Use an onboard speaker to broadcast messages to possible victims on the ground. Region search Investigate a region of land using a formalised search strategy. Localised search Investigate a localised geographical feature (e.g., gorge) using a formalised search strategy. Parallel search Use the UAV to guide ground search teams to a potential sighting on the ground. Terrain mapping Use the UAV to supplement knowledge of the terrain and inform the development of a search plan.

Using the Decision Ladder to Understand Human Decision-Making Processes 141

ACKNOWLEDGMENT

This work was funded and delivered in partnership between the Thales Group and the University of Bristol, and with the support of the UK Engineering and Physical Sciences Research Council Grant Award EP/R004757/1 entitled ‘Thales-Bristol Partnership in Hybrid Autonomous Systems Engineering (T-B PHASE)’.

REFERENCES

Banks, V. A., Plant, K. L. and Stanton, N. A. (2020) ‘Leaps and shunts: designing pilot decision aids on the flight deck using Rasmussen’s ladder’, Contemporary ergonomics and human factors. Bruseberg, A. (2008) ‘Human views for MODAF as a bridge between human fac-

tors integration and systems engineering’, Journal of Cognitive Engineering and Decision Making, 2(3), pp. 220–248. Civil Aviation Authority. (2023). The Drone and Model Aircraft Code (CAP2320).

CAA. https://register-drones.caa.co.uk/drone-code/the_drone_code.pdf Golightly, D., Chan-Pensley, J., Dadashi, N., Jundi, S., Ryan, B. and Hall, A. (2022) ‘Human, Organisational and Societal Factors in Robotic Rail Infrastructure Maintenance’, Sustainability, 14(4), p. 2123. Goodrich, M. A., Cooper, J. L., Adams, J. A., Humphrey, C., Zeeman, R. and Buss,

B. G. ‘Using a Mini-UAV to Support Wilderness Search and Rescue: Practices for Human-Robot Teaming’. 2007 IEEE International Workshop on Safety, Security and Rescue Robotics, 27–29 Sept. 2007, 1–6. Lynch, K. M., Banks, V. A., Roberts, A. P. J., Radcliffe, S. and Plant, K. L. (2022) ‘Maritime autonomous surface ships: can we learn from unmanned aerial vehicle incidents using the perceptual cycle model?’, Ergonomics, pp. 1–19. McIlroy, R. C. and Stanton, N. A. (2015) ‘A decision ladder analysis of eco-

driving: the first step towards fuel-efficient driving behaviour’, Ergonomics, 58(6), pp. 866–882. Neisser, U. (1976) ‘Cognition and Reality WH Freeman’, New York. O’Neill, T., McNeese, N., Barron, A. and Schelble, B. (2020) ‘Human–Autonomy Teaming: A Review and Analysis of the Empirical Literature’, Human Factors, 64(5), pp. 904–938. Parnell, K. J., Fischer, J., Clarck, J., Bodenman, A., Galvez Trigo, M. J., Brito, M.

P., Divband Soorati, M., Plant, K. and Ramchurn, S. (2022) ‘Trustworthy UAV relationships: Applying the Schema Action World taxonomy to UAVs and UAV swarm operations’, International Journal of Human–Computer Interaction. Parnell, K. J., Wynne, R. A., Plant, K. L., Banks, V. A., Griffin, T., GC and Stanton, N. A. (2021) ‘Pilot decision-making during a dual engine failure on take- off: Insights from three different decision-making models’, Human Factors and Ergonomics in Manufacturing & Service Industries. Plant, K. L. and Stanton, N. A. (2016) ‘The development of the Schema World

Action Research Method (SWARM) for the elicitation of perceptual cycle data’, Theoretical Issues in Ergonomics Science, 17(4), pp. 376–401. Rasmussen, J., Petjersen, A. M., & Goodstein, L. P. (1994) Cognitive Systems Engineering. Wiley. Riley, J. M., Strater, L. D., Chappell, S. L., Connors, E. S. and Endsley, M. R. (2010)

‘Situation awareness in human-robot interaction: Challenges and user inter- face requirements’, Human-Robot Interactions in Future Military Operations, pp. 171–192.

Using the Decision Ladder to Understand Human Decision-Making Processes 143

Steane, V. A., Hart, S. G., Oakes, J., Palmer, S. & Chattington, M. (2023 in press).

‘Using EAST to inform systems architecture design’. Considerations relating to the use of UAVs In search and rescue missions. AHFE 2023 International Conference. Vicente, K. J. (1999) Cognitive work analysis: Toward safe, productive, and healthy computer based work. CRC Press.

Cover of the proceedings volume Human Systems Engineering and Design (IHSED 2023): Future Trends and Applications

Explore the complete digital volume

Human Systems Engineering and Design (IHSED 2023): Future Trends and Applications

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-88-9

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →