Mario Simões-Marques, M Filomena Teodoro, Isabel L. Nunes (2022). Decision-making in disaster operations - Intuition vs Intelligent System support. In: Human Factors and Systems Interaction. AHFE Open Access, vol. 52, pp. 1–8. https://doi.org/10.54941/ahfe1002132
1 CINAV, Escola Naval, Instituto Universitário Militar, Base Naval de Lisboa 2810-001 Almada, Portugal ↗
2 CINAV, Escola Naval, Instituto Universitário Militar, Base Naval de Lisboa 2810-001 Almada, Portugal; CEMAT - Center for Computational and Stochastic Mathematics, Instituto Superior Técnico, Lisbon University, Avenida Rovisco Pais, n.1, 1048-001 Lisbon, Portugal ↗
3 NOVA School of Sciences and Technology, NOVA University Lisbon, Caparica, 2829-516 Lisbon, Portugal ↗
The paper presents a study where human decision-making is benchmarked against IS recommendations in a disaster management context. Data collection was done in tabletop exercise sessions where the participants played the role of disaster managers, engaged on decisions scenarios of increasing complexity. Initially, participants were asked to make assignment decisions without any IS advice. Later they were exposed to the advice of the an IS to assess if participants accepted the solutions proposed by the IS as satisficing, considering the explanations provided by the IS. Results suggest that decision-makers tend to rely increasingly in intuition as complexity increases, and welcome the recommendations of IS as satisficing, considering the decision-making process easier with this type of support.
Simões-Marques and Figueira (2019) discussed the role of Intelligent Systems (IS) to support decision-makers in the context of disaster management (DM), noting that Cognitive Science frequently addresses thematic such as bounded rationality, decision fatigue, or impulse decision and discussed the impa- cts and the burden of decision-making on humans. They recognized that this burden is exacerbated when the decision process occurs under com- plex and stressful situations, processing big volumes of information, often shadowed by uncertainty, such as DM in catastrophes, a context where decision-makers are faced with the assessment and prioritization of con- flicting lines of action and difficult trade-offs for selecting and assigning resources in response to disaster crises. Narrowing the number of reasona- ble decision options is a significant contribution to reducing the scope and number of decision situations humans must face, hence reducing their burden and fatigue.
Intelligent Systems (IS) attempt to reach, in a specific domain, a level of analysis and performance comparable to and desirably better than human experts. In fact, IS can engage in complex inference processes, neces- sary for evaluating alternative options and offering high-quality conclusions and advice, as well as explanations about the rationale that led to such conclusions. The use of IS can contribute to circumventing some of the limitations of human decision-makers, therefore being a very promising means to support their decision-making tasks. DM is quite demanding for decision-makers because, since coordinating the variety of actors implies dea- ling with large amounts of uncertain, incomplete, and vague information (Simões-Marques, 2017).
The paper presents a study where human decision-making (mostly intui- tive) is benchmarked against IS recommendations in a DM context. Data collection was done in tabletop exercise sessions where the participants played the role of disaster managers, engaged on decisions scenarios of increa- sing complexity. Initially, participants were asked to assign resources without any IS advice. At a second stage they receive advice from the disTributed Holi- stic Emergency Management Intelligent System (THEMIS) (Correia et al., 2021) to assess if participants accepted the IS proposed solutions as satisfi- cing, considering the explanations provided. For this purpose, the inference process of THEMIS offered non-dominated solutions (considering multiple objectives) and advised regarding the assignment of available resources in support of disaster managers engaged in disaster relief activities.
METHODOLOGY
The test was performed using four fictitious decision scenarios, reflecting the activity of decision-makers in the context of DM after the occurrence of a natural disaster. The test involved Subject Matter Experts (SME) that were asked to perform simulated DM activities in a tabletop exercise, without and with decision support.
The test design process, which is illustrated in Figure 1(a), used a common scenario created based on a virtual simulation, and considered the type of scri- pts used by the Portuguese Navy on their DM training activities. The decision scenarios reflect the assignment of ten disaster response elements, that can be used individually as single-element units or grouped in two-element units, to combine specific skills. The number of incidents considered was twenty.
The script was designed with the goal of exposing SME to the same DM activity performed, in a first stage, without advice and, in a second stage, with advice. The purpose of the first stage was collecting data regarding: (i) the type of reasoning strategy (ranging from totally intuitive to totally objective) adopted while solving the problem of assigning resources to incidents; (ii) the degree of success achieved by SME; and (iii) the SME’s opinion regarding the easiness of the decision process without advice. The purpose of the second stage was collecting data regarding: (i) whether SME were satisficed with the advice based on a IS proposed solution; and (ii) the SME’s opinion regar- ding the easiness of their decision process with advice, which was supported
Decision-Making in Disaster Relief Operations - Intuition vs Intelligent System Support 3
Figure 1: The test design process and the tabletop setup.
by graphical, tabular and natural language means of explaining the propo- sed solution to the disaster manager. The script was used in a pilot test, and after a revision it was validated. The script considers the different decision situations, based on the same geographic context. For each of these situa- tions the SME has to decide the ‘Unit-to-Task’ assignment, without advice and with advice (as previously explained). The script was complemented by a Microsoft PowerpointTM presentation containing hyperlinks that allowed navigating the slides, simulating the user interaction with the IS.
Considering the data collection stage, the test was performed in a class- room. SME interacted with the presentation of the scenario (displayed in a 21-inch monitor) using a mouse, as illustrated in Figure 1(b). The first author played the role of facilitator, clarifying any doubts presented by the SME while performing their DM tasks. The script was applied in individual sessi- ons with a duration of approximately one hour. The participants were Navy officers with training and/or participation in disaster relief operations; the- refore, they were individuals holding higher education degrees and having an adequate level of knowledge regarding the requirements associated with DM. The universe of participants considered in this study was composed by 12 individuals (9 male and 3 female). The average age was 37.9 years old (Std. Deviation = 8.8). Half of the participants had already participated in two or more DM activities. Only one participant had no prior experience.
DECISION SCENARIOS As mentioned, Scenario 1 is characterized by the existence of ten response elements and twenty incidents. Each participant plays the role of a disaster manager that has to decide the assignment of the elements to perform the
response tasks. In this situation the only decision criteria is response time, and the goal of the decision-maker is to minimize the total response time. The participant is faced with a map (where the location of the elements and inci- dents is presented) and also with a table (showing the estimated time for each element to reach the location of the incident), as illustrated in Figure 1(b).
In the Scenario 2 the ten response elements offer different skill sets, while the twenty incidents which are characterized by location and response skills required. The goal of the assignment is to minimize the total response time while ensuring the highest match of the skills of the elements that execute the response tasks. The participants were faced with a map where the location and the skills of the elements and incidents are presented and also with tables showing the estimated time for each element to reach the location of the incident, as well as skill matching information.
In Scenario 3 the ten response elements are the same as before, but the twenty incidents are now characterized by location, skills required and response priority. The goal of the assignment is to minimize the total response time while ensuring the highest match of the skills of the elements that per- form the priority response tasks. The participants were faced with a map where the location, the skills of the elements and incidents and the priority of the tasks are presented and also with tables showing the estimated time for each element to reach the location of the incident as well as skill matching and priority information.
In Scenario 4 the ten response elements offer different skill sets and diffe- rent availability. The incidents are the same as in scenario 3. In this situation the goal of the assignment is to minimize the total response time while ensu- ring the highest match of the skills of the available elements that perform the priority response tasks. The participants were faced with a map where the location, the skills of the elements and incidents, the availability of the elements, and the priority of the tasks are presented and also with tables sho- wing the estimated time for each available element to reach the location of the incident, as well as skill matching and priority information.
TABLETOP SESSIONS RESULTS Regarding Scenario 1, at a first stage, participants were asked to take the assignment decision, using the available information (i.e., the graphics on the map and the tabular estimated response time) and, without advice, to fill a response form with their decision. No time limit was set; however after two minutes the participants were asked to switch the pen they were writing with by one of another color. The intention was to create some pressure on the decision-makers regarding the need to produce the decisions in a reasonable amount of time, inducing some stress in the individuals.
Figure 2(a) presents the results of the decisions made by the participants. The X-axis corresponds to the time spent by participants to complete the decision process. The Y-axis corresponds to the ‘total response time cost’ of the assignment solutions. The horizontal dotted line corresponds to the ‘total response time cost’ of the non-dominated solution proposed by the
Decision-Making in Disaster Relief Operations - Intuition vs Intelligent System Support 5
Figure 2: Weighted costs of participants’ decisions (points) compared with model non- dominated solution (horizontal dotted line) in the context of four decision scenarios.
IS. Each point in the graph reflects the solution of an individual partici- pant. It is possible to observe that some ‘participants’ solutions’ are close to the ‘IS solution’, but the majority presents total response time costs much higher than this value. It is not clear that the quality of participants’ decisi- ons improves with the time spent for making the decision. After finishing this task, the participants were asked to characterize the approach they adopted in solving the problem (intuitive vs. based on objective information). This answer was conveyed based on a 4-points Likert scale (Likert, 1932). Appro- ximately 42% of the participants made their decisions exclusively based on the visual perception of distances (i.e., intuitive approach), while only 8% refer making their decisions exclusively based on objective data. Participants were also asked to express their opinion (based on a 6-points Likert scale) regarding how easy the decision-making process was (refer to Table 1).
At a second stage, participants were faced with advice. The participants were asked to express their opinion if the solution was satisficing (using a 6-points Likert scale). The participants could also reject the proposed solu- tion, justifying the reason and presenting an alternative solution. At the end, the participants were asked to express their opinion regarding how easy the decision-making process supported by advice was (using a Likert scale of six points). The aggregated results are also presented in Table 1.
The process was repeated for Scenario 2. Figure 2 (b) presents the parti- cipants’ decisions, which are close to the ‘IS solutions’, despite several are more costly than this value. As before, the participants were asked to cha- racterize the approach they adopted in solving the problem (intuitive vs.
‘Unit-to-Task’ assignment decision without advice is easy
6 – Totally agree 1 – Totally disagree
6 6 6 6
The proposed solution is satisficing.
6 – Totally agree 1 – Totally disagree
6 6 6 6
‘Unit-to-Task’ assignment decision with advice is easy.
6 – Totally agree 1 – Totally disagree
objective). The distribution of the answers (Table 1) reflects that two thirds of the participants made their decisions intuitively, while only 8% made their decisions exclusively based on objective data. Further to this, participants were asked to express their opinion regarding how easy the decision-making process was. Then, participants were faced with advice (a non-dominated solution that minimized the total response time while respecting the skill matching objective) and asked to express their opinion whether such solu- tion was satisficing, and if the decision-making process supported by advice was easy The results of Scenario 3 are presented in Figure 2(c). In this case the Y-axis corresponds to the weighted cost (‘response time’, ‘skills matching’ and ‘task priority’). Once again ‘participants’ solutions’ tend to be much more costly than the ‘IS solution’. Answers regarding the approach adopted reflect that 75% of the decisions were made intuitively, and none was made exclusively based on objective data (cf. Table 1).
Finally, the results Scenario 4 are presented in Figure 2(d). The Y-axis corresponds to the weighted cost (‘unit availability’, ‘response time’, ‘skills matching’, and ‘task priority’) of the decisions. Once more just a few ‘par- ticipants’ solutions’ are close to the ‘IS solution’. The distribution of the participants’ answers regarding the approach adopted for solving the pro- blem is shown in Table 1, which reflects that 75% of the decisions were intuitive, while only 8% were exclusively objective.
Decision-Making in Disaster Relief Operations - Intuition vs Intelligent System Support 7
Figure 3: Participants opinion regarding the easiness of decision-making in scenarios 1 to 4.
The results of the second stage of each session (Table 1) show that parti- cipants are satisficed with the “IS solutions” and totally agree (answer 6 in the 6-points Likert scale) that decision-making in this context with advice is easy. This has to be compared with the results of decision-making with- out advice, where the opinions of the participants regarding decision-making easiness concur that DM is incrementally more difficult from scenario 1 to scenario 4. The box and whisker graphs presented in Figure 3, reflect the opinion regarding the easiness of decision-making in the four scena- rios, where the graph (a) corresponds to the decision process performed without advice, and the graph (b) to the decision process performed with advice. A conclusion that can be extracted is that the test participants syste- matically consider the decision with advice easier than the decision without advice.
CONCLUSION
The paper presented the procedure employed and the results obtained on table top exercises involving decision-makers, with the purpose of validating the multi-objective assignment model in the context of disaster management, namely considering fictitious decision scenarios that simulate the engage- ment of SME in disaster relief operations. As test findings, it was observed that participants perceived the decision tasks without advice as incrementally more difficult, and their problem-solving strategy was increasingly based on intuition rather than objectivity. It was also observed a significant number of decisions were far from optimality. Participants agreed that the proposed IS non-dominated solutions were satisficing and that they would accept the recommendations, and, finally, that having an IS to support decision-making in disaster management activities is extremely important. To reinforce the conclusions, a detailed Statistical analysis using tests for paired small samples (Jonckheere, 1954; Scheffé, 1959) will be described in an extended version of the present paper.
REFERENCES
Correia, A., Simões-Marques, M., Água, P. (2021) “Event Generation for Emergency Scenarios Simulation”, in: Information Technology Applications for Crisis Response and Management, Beard, Jon (Ed.), pp. 128–149. IGI Global. Jonckheere, A. (1954) “A distribution-free k-sample test again ordered alternatives”,
in: Biometrika, 41, pp: 133–145 Likert, R. (1932) “A technique for the measurement of attitudes”, in: Archives of Psychology, 22(140), pp. 5–55 Scheffé, H. (1959) “The Analysis of Variance”, Wiley, New York Simões-Marques, M. (2017) “Facing Disasters—Trends in Applications to Sup-
port Disaster Management”, in: Advances in Intelligent Systems and Computing, volume 497, Nunes I.L. (Ed.). pp. 203–215. Springer. Simões-Marques, M., Figueira, J.R. (2019) “How Can AI Help Reduce the Burden of Disaster Management Decision-Making?”, in: Advances in Intelligent Systems and Computing, volume 781, Nunes I.L. (Ed.). pp. 122–133, Springer.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors and Systems Interaction
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-28-5
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional