Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Do Not Adjust Your Set! How a Visual Alert Reduced Unnecessary Human Intervention in an Automated Vehicle

Research Centre for Future Transport and Cities, Coventry University, Coventry, UK ↗
Publication year
Digital Object Identifier10.54941/ahfe1007124

Abstract

Human remote operators, teaming with Automated Vehicles (AV), will need to be provided information from the AV to make decisions on how, or when, to intervene in the AV operations. As an extension to research into how many AVs an individual can successfully monitor, the authors designed and implemented a human machine interface (HMI) that provided key information on the probability that an AV might need an intervention from a remote operator. A key element of that interface was the provision of feedback indicating if a vehicle was stationary, provided in the form of a timer, and a visual alert given 10s after the vehicle came to a halt. An experiment was conducted into the efficacy of this visual alert, by on occasion removing it from use. It was expected that the absence of the visual alert would lead to more incidents where a remote operator missed a requirement to intervene. However, the results indicated that in the absence of the alert the remote operator was more likely to intervene. This paper examines how elements of the HMI design affected the participants decision to intervene in AV operations, and concludes that by offering transparent system-state feedback, the HMI effectively counters the innate psychological pressure to act, reassuring the operator that inaction can be an appropriate and system-approved response.

INTRODUCTION

There can be no doubt that the arrival of the automated vehicle (AV) is extremely imminent, with trials of integrating AVs into regular traffic being implemented worldwide (eg, Waymo, 2024; Wayve, 2024; Solihull Metropolitan Borough Council, 2025). Whilst the findings of most of these trials does indicate that AVs are able to negotiate a significant proportion of their Operational Design Domain (ODD) (see BSI 2020 for an explanation of ODD in the United Kingdom), many vehicles are still operating at Society of Automotive Engineers (SAE) Level of Driving Automation 4 (SAE 2021), meaning there is, at present, a residual requirement for human operators to provide support as an in-vehicle safety officer in situations that the AV cannot manage (see BSI 2021 for requirements of a safety operator in the United Kingdom). There is an expectation that in the longer-term these safety officers will be relocated to a remote operations central (ROC) where they provide support using teleop eration technologies and techniques (Le Large et al., 2025).

658 Bogg and Birrell To support this expectation, commercial and scientific research is currently underway on defining the scope of tasks expected of humans conducting remote operations (SAE 2021, BSI 2024, Parr et al., 2024) and the solutions used to undertake those tasks (Neumeier et al., 2019, Brecht et al., 2024). Whilst commercial standards have focused on categorising and defining types of remote operation (BSI 2025, SAE 2021), much of the scientific research being conducted aims to provide knowledge on the plausibility and practicality of conducting remote tasks (Majostorovic and Diermeyer, 2023, Zhao et al., 2024). More recently there has been an increase in the design and testing of human machine interfaces (HMI) used by remote operators to interact with AVs (Schrank et al., 2024, Tener and Lanir, 2025). This study was designed to provide information relevant to this latter research goal of providing design advice for the development of remote operation HMI.

METHODOLOGY

An experimental study was conducted to evaluate the optimum number of AVs that an individual could safety and successfully supervise when the scope of remote operation was reduced to remote monitoring only (with no remote assistance or driving demand, see BSI 1887 for a definition of remote moni toring). The general methodology and findings of the study are discussed in Bogg and Birrell (2025).

As well as investigating the capacity for remote monitoring, the study was also used to evaluate a novel user-centric design for a remote opera tor HMI. During analysis of the participants performance, it was observed that some facets of that HMI design; specifically the information from the AV to indicate its current operational state had an effect on the participant performance. This paper provides and discusses those findings.

EXPERIMENTAL DESIGN The experiment study was a simulation of a remote operator (RO) in a ROC monitoring a small fleet of AVs (see Figure 1). Participants were recruited to role play the remote operator position. They were asked to manage four different sized fleets (3, 5, 7 and 9) of AV, each fleet size set as a condition tested over a separate 15 minute trial. Thus, participants would undertake four trials of 15 minutes. In the simulation each AV was represented by a video playback. Each video featured a number of situations identified as “events” where the AV would temporarily come to a halt for an average of approximately 20s (M = 19.7s SD = 18.0s).

The participant monitored the AVs with an aim to identifying which AVs were either disengaged from the driving task or had become “stuck” and unable to manoeuvre and needed human intervention. The participants, having identified an AV in need of assistance, were then asked to (theoret ically) assign the AV to a separate remote operator, who would take over responsibility for further interactions with that AV.

To promote participant engagement, some events, identified as Call Dilemma’s, were designed to stimulate a participant to intervene. Call

Do Not Adjust Your Set! How a Visual Alert Reduced Unnecessary Human Intervention 659 Dilemmas were events longer than 20s (m = = 51.1s SD = 26.9s) where either: the AV positively asked for assistance; the AV stayed in place after a traffic light changed to green or vehicles in front of it moved; the vehicle was blocked at a junction by other stationary or moving vehicles; or, where the AV simply stopped and gave no reason for stopping (Bogg & Birrell 2025). Call Dilemmas were presented at the rate of approximately one Call Dilemma per 10 minutes per AV.

Figure 1: Remote monitoring study human machine interface.
Original PDF Figure 1:

To carry out the monitoring task, participants were provided with an HMI that provided information and cues that they could use to assess whether any of the AVs had become “stuck” or disengaged. These included: assessments made by the AV itself on its current status given as a change to the video feed border colour (Red, Amber or Green) to represent a “risk assessment” (see Table 1 for explanation of risk states and colours); a text message from the AV giving a reason for it stopping (eg “Stopped In Congestion”); a stop- watch timer that indicated if and for how long an AV had been stationary; and, the forward looking video feed from the AV (see Figure 2).

Figure 2: AV communication devices and remote driver engagement control.
Original PDF Figure 2:

660 Bogg and Birrell The participants were told that if the AV had calculated it needed assis tance, then it would add an extra statement in the Text Message on why it had stopped. For example, when the AV thought no assistance was needed the message could be “Stopped At Traffic Lights”. When assistance might be needed the message could be “Stopped In Congestion - Possible Assistance Needed”. When assistance was definitely needed the AV was clear and the message could be “Stopped At Accident - Assistance Needed”.

Routinely the AV risk assessment was limited to low and high (Green and Red) as per Table 1. However, in addition to those two states, the AV would also provide an intermediary risk “Alert” (Amber) to draw the attention of the participant remote monitor. The risk alert was issued when the AV detected that it had been stationary for 10s or longer which could indicate it was more likely to become “stuck”. Approximately 68% of events were designed trigger a change in the border colour to orange (Bogg & Birrell, 2025).

Table 1: Alert colours and associated risk category.

Table available in the original PDF paper.

Risk Colour Instructed Interpretation Low Green Not Stopped. AV identifies it is not stationary.

Medium Amber Stopped but AV estimates it is likely temporarily (eg at traffic lights) High Red Stopped possibly Permanently. AV is unable to determine if it will be able to move (eg pedestrians in road)

It had been hypothesised that the introduction of this 10s risk alert would assist the participants detect events where an intervention was likely to be required. To test this hypothesis in each of the four condition trials the final event of one of the AVs being monitored was modified so that the AV remained stationary for much longer than 10s but the border colour did not change. Thus, the independent variable was the provision of a visual alert to a risk and the two conditions were Alert and No Alert.

For each participant, half of the modified events were “Call Dilemmas”, a situation designed to stimulate a participant to intervene. The other half of the modified events were designed as distractors that required no interven tion and should be ignored. Thus, it would be possible to observe for changes in behaviour to events that participants were expected to respond to and also events that were expected to be ignored. As Call Dilemma events in the main study were on average approximately 51s, for equity of experience and effect, the final distractor events were of a similar duration.

Human Machine Interface Design The remote monitoring HMI solution finally proposed and implemented was designed as adaptation of a CCTV workstation following the principles of user-centred and task focused design (eg Schrank et al., 2024). The HMI was provided on a single 50-inch screen at UHD (3840 × 2160) divided into equal sized sections, the number of sections depending upon the number of AVs being monitored (see Figure 1). Participants watched the screen and

Do Not Adjust Your Set! How a Visual Alert Reduced Unnecessary Human Intervention 661 monitored the AVs as they drove around a simulation of the area around Coventry, UK.

A total of 24 participants were recruited to meet the minimum Latin Square and Power Analysis requirement for a study with four conditions (the Power Analysis was conducted using G*Power 3.1.9.4 (Faul et al, 2009) with an expected small to medium effect size of f = 0.25 ([9]), and error probability of 0.05, and a power of 0.8).

RESULTS

The rate at which participants responded to a Call Dilemma with No Alert was compared to the overall rate at which participants responded to all other Call Dilemmas with an Alert. However, as long duration distractor events had been selected for the No Alert condition, the rate of response of No Alert distractors was compared only those Alert distractors of a similar duration. This ensured that statistical analysis of behaviour was being made between events of a similar duration and appearance.

Analysing the participant interventions to the regular Call Dilemmas with an Alert in the main study and the final Call Dilemma with No Alert using the Wilcoxon signed-rank test, significant variance was found between the percentage of interventions to Call Dilemmas with an alert against those without an alert (Z = −2.858, p = .004). The profile plots indicate that par ticipants were more likely to make an intervention when there was No Alert (Figure 3).

Figure 3: Difference in rate of intervention between regular events and final events (statistically significant effects are indicated by *).
Original PDF Figure 3:

The analysis of response to the distractor events using the Wilcoxon signed-rank test also indicated significant variance between those distractors with an Alert compared to those with No Alert (Z = −3.166, p = .002) with the profile plots again indicating an increase in the number of interventions when there was No Alert present (Figure 3).

These findings, whilst a partial match for the hypothesis, are not entirely as expected. Whilst the results indicate that adding an Alert will reduce the number of unnecessary interventions to a distractor event, they also indicate

662 Bogg and Birrell that they can reduce the number of expected (and potentially desired) inter ventions to the Call Dilemmas.

The number of unnecessary interventions for the distractor events in Figure 3 also appeared to be higher than expected. Investigating this phe nomenon further, an examination of the general spread of unnecessary inter ventions to all distractor events in all trials was made (see Table 2).

Table 2: Spread of unnecessary interventions to distractor events.

Table available in the original PDF paper.

Distractor Type % of All Unnecessary Interventions Made

Average Event Duration Passenger Drop Off/Pick Up 53.7% 45.2s

Traffic Lights 22.4% 21.0s Congestion 17.4% 21.7s

Pedestrians on road 6.4% 12.3s The results also showed that the distractor events for which an interven tion was requested were generally longer in duration (M = 30.4s SD = 23.0s) than the average distractor event (M = 18.9s SD = 16.7s). Thus, it seemed there might be a correlation between duration of event and chance of the human intervening.

To test this hypothesis, a Pearson’s r product-moment correlation was run to calculate the relationship between the duration of an event (time the vehi cle was stationary) and the percentage chance of the event having an inter vention. There was a strong positive correlation (r = .828, p<.001) between event duration and likelihood of intervention, with the scatter graph plot (Figure 4) indicating that the longer the event the more likely it was that the participant would carry out an intervention.

Figure 4: Correlation between duration of event and percentage chance of participant intervention.
Original PDF Figure 4:

DISCUSSION

The results for the use of a changing border colour to communicate the automated vehicles (AVs) assessment of the risk of an event does appear to demonstrate that the provision of this alert can affect the decision-making of the human remote monitor. However, that effect was not entirely as hypoth esized; the results indicate that in situations where the participants were not provided with a colour changing alert they expected, they were more likely to call for an intervention irrespective of whether it was needed or not.

This compulsion for the human to intervene unnecessarily or prematurely, for example when the AV is stationary at a green light waiting for out of sight congestion to move, can be explained by the powerful combination of the Zeigarnik Effect and Action Bias. The Zeigarnik Effect, explained through the lens of Lewin’s (1951) field theory of psychological tension systems, creates a cognitive itch; the vehicle’s inaction is an open loop that demands closure, generating tension and pulling the operator’s focus. This can be seen in the relationship between the time the AV is stationary and the percentage change of intervening. The longer the AV was stationary, the more likely it was that the participants would step in and make an interven tion, irrespective of whether the circumstances of the event warranted it. This finding does explain why in the post-study interview many participants made positive comments about the presence of the timer, saying they found this information valuable in making their decision (e.g. “I wanted to see why the car stopped, and how long it had been stopped for. Those were the two main indicators as to whether I would call the driver”).

Concurrently, the Action Bias creates a strong preference for taking con crete steps over passive observation, making intervention feel more produc tive and responsible than waiting, even if the AV is just moments away from resolving the situation efficiently on it’s own. This psychological push (from the unresolved state) and pull (toward decisive action) can lead to unnec essary interventions from the remote operator. This paper presents initial findings to suggest that a visual alert on the HMI reduces this powerful drive to intervene, as the AV presents the illusion of being ‘aware’ of the situation it is in.

However, before concluding that the time for which a vehicle is stationary is the most important factor in identifying if there will be an intervention (whether needed or not), it must be observed that participant intervention was most strongly affected by the content of the message from the AV. If the AV asked for an intervention, then 92% of the time the participants provided it. If the AV did not ask for an intervention, then 94% of the time the par ticipants did not intervene. Most participants stated that they valued these SA messages and used them to make their decisions, which accords with the results of Ulahannan et al., (2021) that automation action explanations are vital to humans interacting with that automation.

However, many participants also observed that in longer events, in par ticular those involving the dropping-off or picking up of passengers, they had no knowledge of the success or failure of the progress of those events. These comments do lead to a possible deduction on why the majority of

664 Bogg and Birrell unnecessary interventions were for events involving passengers. In fact, one participant made it clear, explaining that as time advanced and no further information was provided the participant would start to doubt whether the stop was deliberate and was more likely to become convinced it was a system malfunction. Interestingly, some participants themselves offered a potential solution to this problem; they suggested that additional information, specifi cally on movement of passengers, should be provided to help maintain their SA. That information could be in the form of additional video feeds from inside the AV, or from additional update messages.

CONCLUSION

The results of the study do provide positive guidance for the design and implementation of Human Machine Interfaces (HMI) for humans engaged in a Human-Autonomy Team (HAT) with an automated vehicle (AV). Perhaps unsurprisingly, the results all indicate that participants were constantly reviewing the situation and their decisions, and that active communications from the AV, and the duration of an event was strongly correlated with the decision to intervene. These findings confirm that the inclusion of an event timer and text explanation messages from the AV are essential to assist remote operator decision-making.

The results further demonstrated that a visual ‘risk’ alert significantly influenced participant decision-making through a reduction in unnecessary interventions. In this paper we address a core human factors challenge in remote monitoring - the compulsion to intervene. This tendency, driven by psychological effects like Action Bias, can disrupt efficient operations. The visual alert functioned as a double-edged sword; while potentially detrimen tal during genuine system failures, it proved highly effective during planned, albeit slow-moving, tasks by providing a system-status rationale for the delay. Participant feedback supports this, suggesting that such alerts, par ticularly when paired with contextual updates, resolve operator uncertainty and maintain situation awareness. Thus, by offering transparent system-state feedback, the HMI effectively counters the innate psychological pressure to act, reassuring the operator that inaction is the appropriate and system-ap proved response.

These results and conclusions on the design of an HMI for remote oper ators are of direct relevance to groups designing and building AVs and AV systems. The provision of event time and the communication of AV action explanations and assessment of risk will strongly influence the decision-mak ing behaviour of remote operators tasked with monitoring and assisting such AVs. Giving a confirmatory message that the AV is performing as expected, but not giving any message when the AV is unsure should lead to more right decisions from the human remote operators.

FURTHER RESEARCH Further research should investigate how alert dynamics, specifically timing, presence, and absence, can shape operator decision-making and behaviour. The current findings, coupled with participant feedback, highlight a critical

Do Not Adjust Your Set! How a Visual Alert Reduced Unnecessary Human Intervention 665 gap in expanding alert taxonomies. Participants expressed a strong desire for more granular alert categories conveying contextual data. Future studies should therefore explore the efficacy and potential cognitive load of such information-rich alert systems to optimise the flow of information without leading to overreliance or overload.

ACKNOWLEDGMENT

The authors would like to acknowledge the Solihull & Coventry Automated Links Evolution (SCALE) Project funded by Innovate UK UKRI (project code 10040507).

REFERENCES

Bogg, A and Birrell, S. A. (2025) “Overloaded, Underloaded or in Control: How Many Automated Vehicles Can One Person Supervise?”. Computers in Human Behavior, p. 108690. Brecht, D., Gehrke, N., Kerbl, T., Krauss, N., Majstorović, D., Pfab, F., Wolf, M.M.

and Diermeyer, F., (2024). “Evaluation of teleoperation concepts to solve auto mated vehicle disengagements”. IEEE Open Journal of Intelligent Transportation Systems. British Standards Institution (2020) “Operational Design Domain (ODD) taxonomy for an automated driving system (ADS) – Specification”. BSI PAS 1883:2020. London: British Standards Institution. British Standards Institution (2021) “Safety operators in automated vehicle testing

and trialling. Guide.” BSI PAS 1884:2021. London: British Standards Institution. British Standards Institution (2024) “System aspects for remote operation of vehi cles. Guide”. BSI Flex 1886 v2.0:2024–09. London: British Standards Institution. British Standards Institution (2025) “Human factors for remote operations of vehi

cles. Guide”. BSI Flex 1887 v2.0:2025–01. London: British Standards Institution. Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G.(2009) “Statistical power analy ses using G*Power 3.1: Tests for correlation and regression analyses” Behavior Research Methods, 41, pp. 1149–1160. Le Large, N., Brecht, D., Poh, W., Pauls, J.H., Lauer, M. and Diermeyer, F., (2025).

“Human-Aided Trajectory Planning for Automated Vehicles through Teleoperation and Arbitration Graphs”. in 2025 IEEE Intelligent Vehicles Symposium (IV) (pp. 598–604). IEEE. Lewin, K. (1951). Field theory in social science (D. Cartwright, Ed.). New York: Harper. Majstorović, D. and Diermeyer, F., (2023). “Dynamic collaborative path planning for

remote assistance of highly-automated vehicles”. in IEEE International Automated Vehicle Validation Conference (IAVVC) (pp. 1–6). IEEE. Neumeier, S., Wintersberger, P., Frison, A.K., Becher, A., Facchi, C. and Riener, A., (2019). “Teleoperation: The holy grail to solve problems of automated driving? Sure, but latency matters”. in proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 186–197). Parr, H., Harvey, C., Burnett, G. and Sharples, S. (2024) “Investigating levels of

remote operation in high-level on-road autonomous vehicles using operator sequence diagrams”. Cognition, Technology & Work, 26(2), pp. 207–223. Schrank, A., Walocha, F., Brandenburg, S. and Oehl, M.. (2024) “Human-centered design and evaluation of a workplace for the remote assistance of highly auto mated vehicles”. Cognition, Technology & Work, pp. 1–24.

666 Bogg and Birrell Society of Automotive Engineers (SAE) (2021) “Taxonomy and definitions for terms

related to driving automation systems for on-road motor vehicles”. SAE J3016 2021–04. Warrendale: SAE Headquarters. Solihull Metropolitan Borough Council (2025) “CAV Trials SCALE” available from https://www.solihull.gov.uk/about-council/cav-trials/scale [21 October 2025]. Tener, F. and Lanir, J., (2025). “Guiding, not Driving: Design and Evaluation of a

Command-Based User Interface for Teleoperation of Autonomous Vehicles”. arXiv preprint arXiv:2502.00750. Ulahannan, A., Thompson, S., Jennings, P. and Birrell, S. (2021) “Using glance behaviour to inform the design of adaptive HMI for partially automated vehicles”. IEEE Transactions on Intelligent Transportation Systems, 23(5), pp. 4877–4892. Waymo (2024) Waymo One [online] available from <https://waymo.com>

[6 November 2024]. Wayve (2024) Wayve [online] available from <https://wayve.ai> [6 Novemeber 2024]. Zhao L, Nybacka M, Rothhämel M, Habibovic A, Papaioannou G, Drugge L. (2024)

“Driving experience and behavior change in remote driving: an explorative exper imental study”. IEEE Transactions on Intelligent Vehicles. 9 (2) pp 3754–3767.

Cover of the proceedings volume Intelligent Human Systems Integration (IHSI 2026): Disruptive and Innovative Technologies

Explore the complete digital volume

Intelligent Human Systems Integration (IHSI 2026): Disruptive and Innovative Technologies

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-964867-76-2

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →