Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

The impacts of multi-agent quantity, type and transparency on mental workload, situation awareness and human out-of-the-loop

Department of Industrial Engineering, Tsinghua University, Beijing, China ↗
Publication year
Digital Object Identifier10.54941/ahfe1003964

Abstract

With the development of artificial intelligence, more autonomous systems, robots, unmanned aerial vehicles, and unmanned ground vehicles that can adapt and learn, independently determine goals, and allocate resources to perform specific tasks independently are changing human life. These devices are collectively referred to as autonomous systems. Although these autonomous systems can reduce labor demand, expand human capabilities, and improve human security, they will not be able to complete tasks independently in the foreseeable future and need a human to monitor and cooperate with them. Work organization between autonomous systems and humans is essential in the collaboration process. The ideal way is to maximize the agents' autonomy to expand human ability without causing humans mental overload. This study puts forward three objectives to explore a suitable cooperation mode between human and autonomous systems. First, to examine the impact of the multi-agent quantity on human mental workload, situation awareness (SA), and human out-of-the-loop (OOTL) degree. The second is to study the influence of multi-agent type on the above dependent variable. Thirdly, based on the completion of the first two objectives, a new variable, transparency, is introduced to explore ways to improve the cooperation between humans and autonomous systems. We designed a task scenario that simulated humans working with multiple autonomous agents. Two types of agents were adopted and redesigned: an intelligent assistant that helps operators assign unmanned vehicles, and a semi-autonomous dynamic positioning system for vessels. This research conducted two experiments. The first conducted a 2 (multi-agent quantity) × 2 (multi-agent type) within subjects experiment. The second investigated the effect of multi-agent type and transparency. The subjective mental workload, SA, OOTL degree, and performance were measured. According to the experimental results, the following conclusions can be drawn. The more the quantity of agents, the higher the cognitive mental workload, the lower the SA, and the more serious the OOTL degree. The more complex the types of autonomous systems are, the higher the mental workload. When humans interact with multiple autonomous systems, the heterogeneous agents reduce SA compared to homogeneous agents. The higher the transparency of the autonomous systems, the lower the mental workload, the higher the SA, the lower the degree of OOTL, and the better the experimental performance.The results of this study have significant theoretical and applied value. It can improve the existing theories of work organization between autonomous systems and humans, provide a new perspective from the perspective of mental workload, and improve the current research on multiple agent transparency. In terms of practical application, it can help enterprises and individuals decide which work organization mode to adopt in the human and multi-agent teams. It also has important reference value in the design of multiple autonomous systems. When only one aspect can be improved, priority can be given to the tradeoff on the quantity of agents. When it is inevitable to make humans interact with multiple agents, the transparency of the autonomous systems can be improved to increase the explainability and enhance the mental workload and task performance.

INTRODUCTION

Artificial intelligence’s development, more autonomous systems, robots, unmanned aerial vehicles, and unmanned ground vehicles that can adapt and learn, independently determine goals, and allocate resources to perform specific tasks independently are changing human life (Kaber, 2018). These machines are collectively referred to as autonomous systems. Although these autonomous systems can reduce labor demand, expand human capabilities, and improve human security, they will not be able to complete tasks indepen- dently in the foreseeable future and need a human to monitor and cooperate with them (Xiong et al., 2022).

In general, humans mainly monitor the agents’ work and verify the comple- tion of their tasks. As a result, the mental workload is significantly reduced, and humans have more free time. Sometimes one operator can monitor mul- tiple agents simultaneously, saving limited resources and reducing labor costs

© 2023. Published by AHFE Open Access. All rights reserved. 40

The Impacts of Multi-Agent Quantity, Type and Transparency 41

for society. However, increasing the number of agents cooperating with peo- ple may overload the human mental workload, thus reducing the operational performance level of the human-agent team (Donmez et al., 2010). At the same time, it may also cause mistakes in human information acquisition and analysis and decision-making errors, leading to accident symptoms and accidents (Cummings et al., 2007). Work organization between autonomous systems and humans is essential in the collaboration process. The ideal way is to maximize the agents’ autonomy to expand human ability without causing humans mental overload.

In addition, when people cooperate with multiple agents, there are pro- blems related to the types of agents besides the quantities of agents (Mina et al., 2020). In a real-world scenario, people may need to simultaneously monitor agents of the same kind, but they may also monitor different types of agents simultaneously (Donmez et al., 2010). We named agents with the same functions and principles as homogeneous agents and called agents with different functions and principles heterogeneous agents. How to orga- nize work is a problem that needs to be solved for collaboration between humans and multi-agents. It needs to be clear whether there is a significant difference between monitoring multiple homogeneous agents and multiple heterogeneous agents from the perspective of mental workload. What design should be adopted to reduce the mental workload problems encountered in a human multi-agent team? These are essential questions that still need to be thoroughly studied.

The design that considers transparency in a multi-agent team may pro- vide an improved approach to the above problems. As autonomous agents become more and more complex and independent, agents’ intentions, beha- viors, and reasoning processes behind them, as well as expected results and other aspects of information, become very important. Humans must acquire effective situational awareness of multiple agents, properly calibrate trust for agents and make appropriate decisions, and achieve high performance in a human multi-agent team. Access to this information depends on the design of transparency for the multiple agents. Transparency refers to the visibility of human or agent information processing processes and decision-making processes. The higher the transparency, the higher the visibility (Chen et al., 2018; Kraus et al., 2020). The level of transparency is usually influenced by interface design and the information content (Guznov et al., 2020).

Based on the above definitions, Chen et al. (2018) proposed a situation awareness-based transparency (SAT) model to assess the level of transparency for autonomous agents, guiding the design of autonomous agent transpa- rency. In this model, transparency can be divided into three levels. An autonomous agent with SAT level 1 (SAT 1) transparency provides essen- tial information, such as current operating status, decision goal, and actions taken, to help humans to perceive agents’ behavior. An autonomous agent with SAT level 2 (SAT 2) transparency needs to provide the logical reasoning process of its decisions and the constraints considered in action, to help peo- ple understand the behavior of autonomous agents. An autonomous agent with SAT level 3 (SAT 3) transparency predicts the future operating status, decision consequence, likelihood of success or failure, and the uncertainty of

the above judgments to help people project the future outcome of the current task.

This study puts forward three objectives to explore a suitable cooperation mode between humans and multiple autonomous agents. First, to examine the impact of the multi-agent quantity on human mental workload, situation awareness (SA), and human out-of-the-loop (OOTL) degree. The second is to study the influence of multi-agent type on the above dependent variable. Thirdly, based on the completion of the first two objectives, a new varia- ble, transparency, is introduced to explore ways to improve the cooperation between human and multiple autonomous agents.

METHOD

Participants In Experiment 1, 12 undergraduates (7 males and 5 females) were recruited as participants. In Experiment 2, 22 undergraduates (17 males and 5 females) were recruited to participate.

Design Experiment 1 adopted a 2 (multi-agent quantity: 1, 2) × 2 (multi-agent type: homogeneous, heterogeneous) within-subjects design. The condition of quan- tity 1 means that operators can only switch to the other agent after interacting with one agent and cannot manipulate multiple agents simultaneously. In other words, the operator only works with one agent at a time but can alter- nately manipulate the other agent. The other agent may be of the same type (homogeneous) or a different type (heterogeneous). The total amount of work in the experiment was controlled and was the same in different experimental conditions. Therefore, in one case, the agent continues to interact with the same type of agent (homogeneous). After finishing one group, the other type of agent is switched to complete a set of tasks. The other case is switching between agents of different types (heterogeneous), switching after comple- ting one trial, which means alternately manipulating agents of different types. The condition of quantity 2 means that the human monitors two agents at a time. Under the homogeneous condition, two agents of the same type are monitored simultaneously. Under the heterogeneous condition, two agents of different types are monitored. In the four experimental conditions, the total amount of work performed was the same, meaning that the total number of tasks performed was the same. The task scenario diagram of Experiment 1 is shown in Figure 1.

Experiment 2 adopted a 2 (multi-agent type: homogeneous, heterogene- ous) × 2 (transparency: SAT 1, SAT 2) within-subjects experiment design. The design details of transparency are covered in the next section.

Simulation Environment and Tasks Two types of autonomous agents are adopted and redesigned to simulate the scenario of human interaction with two types of agents. The two agents have different functions. One is an intelligent assistant that helps operators assign

The Impacts of Multi-Agent Quantity, Type and Transparency 43

Figure 1: The illustrative diagram shows monitoring one agent (left) and two agents (right) at a time.
Original PDF Figure 1:

unmanned vehicles, and the other is a semi-autonomous dynamic positio- ning system for pre-defined path tracking of vessels, forming a multi-agent team with humans. The former is a multi-unmanned vehicle management assistant designed by Bhaskara et al. (2021) to study the autonomy tran- sparency design to improve the accuracy of using automation. The latter is adapted from an automatic change identification tool designed by Rick van der Kleij et al. (2018) for dynamic vessel positioning to study the impact of automatic change identification on situational awareness recovery in the case of an emergency takeover. The interfaces of the two types of agents are shown in Figure 2.

In Experiment 1, in collaboration with the intelligent assistant (agent 1), the operator assigns appropriate unmanned vehicles to the destination to complete the task according to the task’s requirements. According to four attributes of the vehicle (including speed, distance to the search area, fuel efficiency, and search steps), the intelligent assistant calculates the score of each unmanned vehicle’s ability to complete the current task (including three dimensions: time to the search area, search time and fuel consumption). A weighted score of vehicle capability is then calculated.

Based on the scores, the intelligent assistant recommends two alternative vehicles (plan A and plan B). Although the calculations of the assistant are reliable, the recommendations are only sometimes optimal because the ope- rator will obtain some up-to-date information, which may interfere with the recommendations. There were three categories of additional information: 1)

Figure 2: The interface of the intelligent assistant for assigning unmanned vehi- cles (left) and the interface of the semi-autonomous dynamic positioning system for vessels (right).
Original PDF Figure 2:

information that had no effect on recommendations; 2) information that affe- cted the score of the recommended vehicle but had no effect on the ranking of the scores; 3) information that had an impact on the score of the recom- mended vehicle and the ranking. The operator needs to make decisions based on the recommendation of the assistant and the latest information.

On the other hand, the operator monitors the job of Agent 2 and ensures the vessel’s navigation. When the system is abnormal, the operator can timely intervene appropriately and take over the agent’s work if necessary. There are four parameters in the system. The operator’s feedback rules are as follows: 1) Four parameters are normal, the operator will give the feedback “OK”; 2) The change amount of 1–2 parameters are abnormal, feedback “Monitor” ; 3) One parameter exceeds the threshold, or the change of 3–4 parameters is abnormal, feedback is “Caution”; 4) One parameter exceeds the thresh- old and some parameters change abnormal, feedback is “Partial takeover”; 5) Two or more parameters exceed the threshold, feedback “Full takeover”. In Experiment 2, the task was similar to Experiment 1, with some dif- ferences in the transparency design. For Agent 1, in the low transparency condition (SAT 1), the “Recommendation” information will only show the recommendation priority without the calculation process. In the condition of high transparency (SAT 2), the “Recommended” indicates the recommenda- tion and shows the reasoning process, namely the calculation of two score items. Participants were shown the formulas for the two points in both SAT 1 and 2 conditions. The difference between the two levels of the transparency interface is shown in Figure 3.

For Agent 2, the system prompts only the abnormal value in SAT 1 con- dition. In the SAT 2 condition, the agent further analyzes values to assist in diagnosis and displays them on the interface. The interfaces of the two transparency designs are shown in Figure 4.

Measures The mental workload was measured by the NASA-TLX Scale. Mental requi- rements, physical requirements, time requirements, operational performance, effort degree, and frustration degree were assessed. Situation awareness was measured by the Situational Awareness Rating Technique (SART) questi- onnaire, which includes ten questions. Situation awareness was measured

Figure 3: The Recommendation part of the two transparency designs for agent 1.
Original PDF Figure 3:

The Impacts of Multi-Agent Quantity, Type and Transparency 45

Figure 4: The interfaces of the two transparency designs for agent 2.
Original PDF Figure 4:

from three dimensions: attention demand, attention supply, and understan- ding of the situation (Endsley et al., 1998). OOTL degree was measured by the OOTL self-rating scale proposed by Jamieson & Skraaning (2020). The scale consists of four questions to investigate the problems that operators encounter when working with agents in the following aspects: (a) keeping track of the program execution, (b) finding the components needed to run the program, (c) knowing the impact of the program steps, and (d) knowing whether the program introduces certain deviations. In addition, experiment 2 also analyzed performance measured by the accuracy rate.

Procedure The participants first practiced the tasks and got the correct rate feed- back. After the practice, the subjects completed the formal experiment. A total of eight unmanned vehicle assignment tasks and eight vessel dyna- mic positioning tasks were included in each experimental condition. After the tasks of each condition, participants completed NASA-TLX, SART and OOTL questionnaires. The participants filled in the questionnaires four times, corresponding to four experimental conditions.

RESULTS

The dependent variables passed the tests of normality and homogeneity of variance. ANOVA was used to analyze the influence of independent variables. Bonferroni adjustment was used for post-hoc analysis. The effect size was measured with partial eta-squared (ηp2).

Experiment 1 The Means and standard deviations of the dependent variables are summari- zed in Table 1. The ANOVA results of mental workload (Table 2) indicated significant main effects of multi-agent quantity (F (1, 11) = 51.58, p < 0.001, ηp2 = 0.824) and type (F (1, 11) = 6.20, p = 0.03, ηp2 = 0.360). The mental workload of monitoring two agents working together (Mquantity = 2 = 48.33) is higher than monitoring only one agent at a time (Mquantity = 1 = 39.79), even though the total number of tasks is the same. The mental workload of monitoring heterogeneous agents (Mheterogeneous = 45.42) is higher than monitoring homogeneous agents (Mhomogeneous = 42.71).

Table 1. Means and standard deviations of the measures.

Table available in the original PDF paper.

Measures Quantity 1 Quantity 2 Homogeneous Heterogeneous Homogeneous Heterogeneous Mean (SD) Mean (SD) Mean (SD) Mean (SD)

NASA-TLX 38.58 (7.80) 41.00 (8.48) 46.83 (10.18) 49.83 (4.09) SART 24.67 (7.61) 25.25 (6.70) 18.17 (5.20) 15.08 (6.64) OOTL 12.33 (7.38) 13.00 (5.33) 15.92 (5.60) 15.92 (5.53)

Table 2. Results of ANOVA for NASA-TLX.

Table available in the original PDF paper.

F df1 df2 p ηp2 Quantity 51.58 1 11 < 0.001 0.824 Type 6.20 1 11 0.030 0.360 Quantity × Type 0.04 1 11 0.848 0.003

The ANOVA results of SART (Table 3) indicated a significant interaction effect between multi-agent quantity and type (F (1,11) = 5.64, p = 0.037, ηp2 = 0.339). The post-hoc results indicated that when collaboration with two agents at a time, heterogeneous agents resulted in a more significant loss of situational awareness than homogeneous agents (Mhomogeneous = 18.17, Mheterogeneous = 15.08, t (11) = 2.564, p = 0.026). When operators moni- tored only one agent at a time, situational awareness had no significant difference between a heterogeneous agent and a homogeneous agent. Mea- nwhile, the main effect of quantity was significant (Mquantity = 1 = 24.96, Mquantity = 2 = 16.63, F (1, 11) = 18.31, p = 0.001, ηp2 = 0.625), indica- ting a significant loss of situation awareness when operators monitored two agents at a time.

The ANOVA results of OOTL degree (Table 4) indicated significant main effects of multi-agent quantity (F (1, 11) = 34.22, p < 0.001, ηp2 = 0.757). The OOTL degree of monitoring two agents working toge- ther (Mquantity = 2 = 15.92) is higher than monitoring only one agent at a time (Mquantity = 1 = 12.67), even though the total number of tasks is the same.

Table 3. Results of ANOVA for SART.

Table available in the original PDF paper.

F df1 df2 p ηp2 Quantity 18.31 1 11 0.001 0.625 Type 2.24 1 11 0.163 0.169 Quantity × Type 5.64 1 11 0.037 0.339

Table 4. Results of ANOVA for OOTL.

Table available in the original PDF paper.

F df1 df2 p ηp2 Quantity 34.22 1 11 < 0.001 0.757 Type 0.693 1 11 0.423 0.059 Quantity × Type 0.141 1 11 0.715 0.013

The Impacts of Multi-Agent Quantity, Type and Transparency 47

Experiment 2 The Means and standard deviations of the dependent variables are summari- zed in Table 5. The ANOVA results of the four measures indicated significant main effects of agent transparency (Table 6–9). Working with multiple agents with high transparency will reduce mental workload, improve situational awareness, reduce OOTL, and improve performance.

Table 5. Means and standard deviations of the measures.

Table available in the original PDF paper.

Measures SAT 1 SAT 2 Homogeneous Heterogeneous Homogeneous Heterogeneous Mean (SD) Mean (SD) Mean (SD) Mean (SD)

NASA-TLX 40.64 (8.28) 43.50 (7.48) 35.23 (7.86) 37.64 (7.46) SART 18.45 (8.59) 16.50 (7.34) 23.23 (6.75) 21.91 (7.38) OOTL 13.55 (4.83) 14.09 (5.11) 11.91 (5.51) 11.68 (4.86) Performance 0.75 (0.14) 0.74 (0.15) 0.83 (0.09) 0.81 (0.11)

Table 6. Results of ANOVA for NASA-TLX.

Table available in the original PDF paper.

F df1 df2 p ηp2 Transparency 17.82 1 21 < 0.001 0.459 Type 3.12 1 21 0.092 0.129 Transparency × Type 0.05 1 21 0.829 0.002

Table 7. Results of ANOVA for SART.

Table available in the original PDF paper.

F df1 df2 p ηp2 Transparency 18.01 1 21 < 0.001 0.462 Type 1.92 1 21 0.180 0.084 Transparency × Type 0.07 1 21 0.800 0.003

Table 8. Results of ANOVA for OOTL.

Table available in the original PDF paper.

F df1 df2 p ηp2 Transparency 7.45 1 21 0.013 0.262 Type 0.06 1 21 0.804 0.003 Transparency × Type 0.20 1 21 0.660 0.009

Table 9. Results of ANOVA for performance.

Table available in the original PDF paper.

F df1 df2 p ηp2 Transparency 13.65 1 21 0.001 0.394 Type 0.59 1 21 0.451 0.027 Transparency × Type 0.03 1 21 0.862 0.001

DISCUSSION

Goodrich et al. (2005) raised that different mental models are required for different types of agents. Arrington & Logan (2004) proposed that in the cooperation between humans and multi-tasks, people must choose when to switch from one task to another, which will also affect people’s mental wor- kload. Wong & Seet (2017) also raised that the competition for attention resources caused by multiple agents affects mental workload. Monitoring heterogeneous agents have two different mental models compared to homo- geneous agents. Monitoring multiple agents simultaneously increases the attention resource competition load and task switching load compared with monitoring one agent. Therefore, this paper explored the effects of the quan- tity and types of agents on the dependent variables such as mental workload, situation awareness, OOTL degree, and performance when people interact with multiple agents.

The results showed that the number of agents would significantly impact mental workload, which is the consensus of previous studies. It shows that cooperation with multiple agents will bring problems with attention alloca- tion and increase the mental workload. According to the SART and OOTL scores, the increase in the number of agents significantly reduced the situa- tion awareness and increased the degree of OOTL. Secondly, the agents’ types also significantly impact mental workload, an influential factor that has not been studied before. It is worth noting that, from the perspective of mental workload, the influences of type and quantity are not the same (by compa- ring the effect size). When the number of agents monitored at a time is only one, the type has no effect on situation awareness. Only when the number of agents increases to two the situational awareness will be affected by type. However, there was no significant relationship between the type of agent and OOTL.

In the second experiment, transparency design was introduced to improve human and multiple agents interaction. Some studies proposed that impro- ving transparency will bring new information and enhance people’s situa- tional awareness (Endsley, 2017). But some studies revealed that adding information will increase the amount of information needed to display, which may require additional cognitive efforts (Chen et al., 2018). The results of previous experiments showed no consistent conclusion on whether enha- ncing transparency can affect performance. After reviewing some empirical studies on transparency, Rajabiyazdi & Jamieson (2020) conclude that the correlation between transparency and performance is still uncertain. In this paper, we found that transparency can reduce mental workload, improve situation awareness, reduce OOTL degree, and improve team performance. From an application perspective, a higher level of transparency would be a cost-effective option for improving agent teams. According to the SAT model, from level 1 to level 2 only requires displaying the reason for the decision, and from level 2 to level 3 requires predicting the future outcome. Transpa- rency design is more actionable than purely thinking about improving the intelligence and autonomy of agents.

The Impacts of Multi-Agent Quantity, Type and Transparency 49

CONCLUSION

This study has significant theoretical and applied value. It can improve the existing theories of work organization between autonomous systems and humans, provide a new perspective from the perspective of mental workload, and improve the current research on multiple agent transparency. In practical application, it can help enterprises and individuals decide what work orga- nization mode to adopt in the human multi-agent team. It also has essential reference value in the design of multiple autonomous systems. Priority can be given to the trade-off on the quantity of agents. When it is inevitable to make humans interact with multiple agents, the transparency of the autonomous systems can be improved to increase explainability, reduce mental workload and enhance task performance.

ACKNOWLEDGMENT

The authors would like to acknowledge. This work was funded by the Natural Science Foundation of China [Grant No. 72171130 and 71771134].

REFERENCES

Arrington, C. M., & Logan, G. D. (2004). The Cost of a Voluntary Task Switch.

Psychological Science, 15(9), 610–615. Bhaskara, A., Duong, L., Brooks, J., Li, R., McInerney, R., Skinner, M., Pongracic, H., & Loft, S. (2021). Effect of automation transparency in the management of multiple unmanned vehicles. Applied Ergonomics, 90, 103243. Chen, J. Y. C., Lakhmani, S. G., Stowers, K., Selkowitz, A. R., Wright, J. L., & Barnes,

M. (2018). Situation awareness-based agent transparency and human-autonomy teaming effectiveness. Theoretical Issues in Ergonomics Science, 19(3), 259–282. Cummings, M. L., Nehme, C. E., Crandall, J., & Mitchell, P. (2007). Predicting Operator Capacity for Supervisory Control of Multiple UAVs. In J. S. Chahl, L. C. Jain, A. Mizutani, & M. Sato-Ilic (Eds.), Innovations in Intelligent Machines—1 (pp. 11–37). Springer. Donmez, B., Nehme, C., & Cummings, M. L. (2010). Modeling Workload Impact in

Multiple Unmanned Vehicle Supervisory Control. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 40(6), 1180–1190. Endsley, M. R. (2017). From Here to Autonomy: Lessons Learned From Human– Automation Research. Human Factors: The Journal of the Human Factors and Ergonomics Society, 59(1), 5–27. Endsley, M. R., Selcon, S. J., Hardiman, T. D., & Croft, D. G. (1998). A Comparative

Analysis of Sagat and Sart for Evaluations of Situation Awareness. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 42(1), 82–86. Goodrich, M. A., Quigley, M., & Cosenzo, K. (2005). Task Switching and Multi- Robot Teams. In L. E. Parker, F. E. Schneider, & A. C. Schultz (Eds.), Multi- Robot Systems. From Swarms to Intelligent Automata Volume III (pp. 185–195). Springer Netherlands. Guznov, S., Lyons, J., Pfahler, M., Heironimus, A., Woolley, M., Friedman, J.,

& Neimeier, A. (2020). Robot Transparency and Team Orientation Effects on Human–Robot Teaming. International Journal of Human–Computer Interaction, 36(7), 650–660. Jamieson, G. A., & Skraaning, G. (2020). The Absence of Degree of Automation Trade-Offs in Complex Work Settings. Human Factors, 62(4), 516–529.

Kaber, D. B. (2018). A conceptual framework of autonomous and automated agents.

Theoretical Issues in Ergonomics Science, 19(4), 406–430. Kraus, J., Scholz, D., Stiegemeier, D., & Baumann, M. (2020). The More You Know: Trust Dynamics and Calibration in Highly Automated Driving and the Effects of Take-Overs, System Malfunction, and System Transparency. Human Factors, 62(5), 718–736. Mina, T., Kannan, S. S., Jo, W., & Min, B.-C. (2020). Adaptive Workload Allocation

for Multi-Human Multi-Robot Teams for Independent and Homogeneous Tasks. IEEE Access, 8, 152697–152712. Rajabiyazdi, F., & Jamieson, G. A. (2020). A Review of Transparency (seeing-into) Models. 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 302–308. van der Kleij, R., Hueting, T., & Schraagen, J. M. (2018). Change detection support

for supervisory controllers of highly automated systems: Effects on performance, mental workload, and recovery of situation awareness following interruptions. International Journal of Industrial Ergonomics, 66, 75–84. Wong, C. Y., & Seet, G. (2017). Workload, awareness and automation in multiplerobot supervision: International Journal of Advanced Robotic Systems. Xiong, W., Fan, H., Ma, L., & Wang, C. (2022). Challenges of human—Machine col-

laboration in risky decision-making. Frontiers of Engineering Management, 9(1), 89–103.

Cover of the proceedings volume Social and Occupational Ergonomics

Explore the complete digital volume

Social and Occupational Ergonomics

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-82-7

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →