Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

External HMI for Automated Vehicles: Adding a Communication Perspective for all Road Users

Eindhoven University of Technology, Eindhoven, The Netherlands ↗
Publication year
Digital Object Identifier10.54941/ahfe1001922

Abstract

In this paper, we address the need for a common communication framework for all road users when interacting with automated vehicles (AVs). Our vision is to work towards a unified, comprehensive, and efficient external human-machine interface (eHMI) that can be commonly used and understood by all. A unified concept is needed since an AV will drive in mixed traffic (coexistence of AVs, pedestrians, cyclists, manually driven vehicles and motorcycles). Since manually driven vehicles (MDVs) will be one of the main road user groups (e.g. due to the first applications on motorways), it is important to consider this target group in research. Although some researchers have begun to focus on AV-MDV interaction to fill the gap in recent years, existing studies are still limited. This paper will summarise existing studies of eHMI with a specific focus on MDV and add this to the large existing research field of AV-VRU interaction. Existing literature is classified and summed up around two main aspects: 1) locations and scenarios of encounters; 2) task requirements of AV-MDV interaction. They can describe and help us get a more comprehensive understanding of AV-MDV interaction. Based on the literature review, the existing research gaps are summarized to develop new approaches for a more unified eHMI.

INTRODUCTION

Automated vehicles (AVs) are often promised to benefit the traffic envi- ronment from many perspectives, such as improving efficiency, reducing accidents, and reducing drivers’ task load. To get these benefits, AVs need to be trusted and accepted not only by the people using them but also by the road users (RUs) confronted with AVs in daily traffic. While traditionally, many studies focused on in-vehicle interaction of AVs (e.g. Beggiato et al., 2015; Strömberg et al., 2018; Carsten and Martens, 2019; Feierle et al., 2020), more and more research is now focusing on AV-pedestrian interaction from the perspective of a pedestrian who is confronted with an AV (e.g. De Clercq et al., 2019; Walker et al., 2019; Dey et al., 2021; Madigan et al., 2021). Especially in AV-pedestrian research, various external HMIs (eHMIs) smoo- thening the interaction have been developed. However, in the next couple of decades, the traffic environment will be a mix of diverse traffic participants

© 2022. Published by AHFE Open Access. All rights reserved. 78

External HMI for Automated Vehicles: Adding a Communication Perspective 79

interacting, including AVs and conventional RUs such as pedestrians, cycli- sts, and manually driven vehicles (MDVs). Therefore, the interaction between conventional RUs and AVs will become one of the most important issues to establish safe and smooth traffic. In this paper, we will mainly focus on AV-MDV interaction, a topic that has been less studied than AV-pedestrian interaction.

Challenges of Developing AV-MDV Interaction There is already a large research field studying the interaction between AVs and other RUs. Existing studies mainly focus on the interaction between AVs and vulnerable road users (VRUs) such as pedestrians (e.g. Mahadevan et al., 2018; Faas et al., 2021) and cyclists (e.g. Bella and Silvestri, 2017; Berge et al., 2022). These studies established a relatively comprehensive set of study types. For instance, most studies focus on: 1) the interaction at a specific location, such as crosswalks (e.g. Dey and Terken, 2017) or intersections (e.g. Wang et al., 2021); 2) the interaction of a specific target group such as pedestrians (e.g. Walker et al., 2019; Faas et al., 2021) or cyclists (e.g. Hou et al., 2020; Ackermann et al., 2021); 3) the interaction with a specific design concept for an eHMI (e.g. Fridman et al., 2017; Chang et al., 2017; for an ove- rview of eHMI concepts for pedestrians see Dey et al., 2020). Based on this large amount of studies, there is extensive knowledge about how pedestri- ans behave in encounters with AVs, which factors influence the interaction, what are the preferred eHMI patterns, etc. However, the existing studies and concepts for pedestrians cannot simply be applied to the interaction with all human RUs.

Objective This paper has three main objectives: 1) Provide an overview of and sum- marise the existing studies of eHMI for manually driven vehicles; 2) Give guidance or point out topics that require further investigation about AV-MDV interaction; 3) Encourage researchers to develop holistic eHMIs to cater for the needs of all RUs.

METHOD

To structure the existing studies on AV-MDV interaction, we summarise the existing AV-MDV interaction studies. We came to two aspects that stress chal- lenges and gaps of AV-MDV interaction studies: scenarios of the encounters and the task requirements of AV-MDV interaction. Both aspects are summa- rised to describe AV-MDV interaction and help us get a more comprehensive understanding of various factors that may influence AV-MDV interaction. Moreover, some studies from other related research fields that may provide inspiration or contribution are also addressed.

The keywords in our search included “automated vehicle”, “driving behavior”, “mixed traffic”, “manually driven vehicle”, “human driver”, “cooperation”, “uncertainty”, “intention”, as well as combinations of these initial keywords. Citations and references of important publications were also reviewed.

80 Gao and Martens

RESULTS

After reviewing, about thirty publications turned out to be relevant to the topic “AV-MDV interaction”. The findings from the literature review are presented below.

Locations and Scenarios of Encounters The locations of AV-MDV interaction are more diverse than AV-pedestrian interaction. Traffic rules require most pedestrians to walk in designated areas (e.g. sidewalks or zebra crossings), different from where vehicles are driving. Therefore, the encounters of VRUs and AVs mostly happen in particular location-based scenarios. Existing studies mainly addressed these scenarios, such as crosswalks or zebra crossings (e.g. Rasouli et al., 2017) and unstructu- red intersections (e.g. Wang et al., 2021). Even in studies in which pedestrians are not in designated areas, it is always a distinct point in time where a pede- strian was asked to either cross the road or indicate their willingness to cross the road (e.g. Dey and Terken, 2017).

However, researchers cannot solely use these particular location-based scenarios when looking at studies on AV-MDV interaction. MDVs may encounter AVs all the time during driving. In the limited amount of studies that looked into AV-MDV interaction, researchers firstly paid attention to some particular location-based scenarios, such as the deadlock situation or T-intersections (e.g. Imbsweiler et al., 2018) and bottlenecks (narrow pas- sage) (e.g. Rettenmaier et al., 2020). In these scenarios, drivers usually need to negotiate the priority with the oncoming vehicles. Therefore, complex actions, cooperation, quick responses and anticipation will be required for drivers to avoid traffic accidents. However, AV-MDV interaction may happen everywhere on the road when they meet and interact, such as lane changing (Kauffmann et al., 2018) and overtaking (Ritchie et al., 2019), making it very different from AV-pedestrian interaction.

Task Requirements of Interaction As discussed before, AV-MDV interaction may happen everywhere in every- day traffic. This means that the information needs may not be restricted to isolated encounters. Moreover, the information need may be more complex and continuous for drivers to make manoeuvres for the next series of opera- tions since multiple factors may influence the interaction in situations such as overtaking and lane changing.

Differences of encounters and information needs may also lead to the differences of requirements for eHMI modalities. Some researchers have stu- died different eHMI modalities. For example, Rettenmaier and his colleagues (2020) designed a frontal display for bottleneck scenarios in a simulator expe- riment, using arrows and colours known from traffic signs, to show the AV’s states and intentions of yielding or insisting the right of the way. Besides displays, Papakostopoulos and colleagues (2021) used a light strip mounted on the windshield to show AV’s yielding intention in a four-way junction in a field experiment. Moreover, in 2019, Rettenmaier and colleagues also condu- cted a driving simulator study by using a frontal display (using arrows known

External HMI for Automated Vehicles: Adding a Communication Perspective 81

from traffic signs) and projection (project information in the form of arrows and lines on the road in front of the car). This study showed that the display improved the efficiency of interaction more significantly than the projection. The reason may be that the display may have better visibility. These interfa- ces positively influence interaction efficiency compared to situations without eHMI. Besides, many other potential interface modalities still remain to be studied, such as displays with symbols, anthropomorphic displays, auditory or multi-modal interfaces.

Moreover, timing is also essential since AV-MDV interaction may be more dynamic. For example, an earlier signal (signal at a longer time to collision [TTC]) and a longer waiting time after the AV has shown intentions have demonstrated to be more cooperative and unambiguous since human drivers have more time to decide and plan for the following operations (Kauffmann et al., 2018). However, an early signal may also be misunderstood by other RUs (Kauffmann et al., 2018). A longer TTC may be regarded as a sign of low criticality, reducing human drivers’ willingness to interact and cooperate with AVs (Stoll et al., 2020). So the ideal timing of interaction may be ‘early’ but ‘within a limited time frame’, which needs to be further investigated.

Other Studies That May Contribute to AV-MDV Interaction Similar research topics also appear in the research area of human-robot inte- raction. Phillips et al. (2011) point out that one of the significant changes in human-machine interaction is the transition in the human’s role from con- trolling machines to collaborating with intelligence agents (IAs). This change may make the interaction more likely to be peer-based. As IAs, AVs may have the same potential. Sorokin et al. (2019) offer a design approach to cre- ate characters for AVs and design them as social actors by analysing public expectations via online media. Therefore, AVs’ roles in traffic may be a new topic for eHMI study in mixed traffic.

CONCLUSION

Differences of eHMIs are summarised by looking into existing studies of AV- MDV interaction and comparing them with studies for AV-VRU interaction. The differences will influence eHMI requirement and show that eHMIs for AV-pedestrian interaction can not simply be used for AV-MDV interaction:

1) MDVs (hence human drivers) have a higher task load than VRUs since they need to process the information of the eHMI while performing a driving task, suggesting fewer resources are available to process the information. 2) The vehicles’ speed is often higher than pedestrians’, which will lead to a difference in encounters and the need to abstract information from vehicles being further away. 3) The mere fact that drivers sit in a vehicle may give different requirements for the eHMI. Factors such as glare of the window, viewing angle to the AV and noise in the car may produce different requirements regarding the intensity or design of signals.

82 Gao and Martens

4) The location of the interaction is often different. Where pedestrians usu- ally walk on the pavement and only cross the road when needed or in dedicated areas, MDVs drive on the same road stretch as AVs. 5) AV-MDV interaction is more complex and diverse than AV-pedestrian interaction. For example, in an overtaking situation, vehicles may need to merge into mixed traffic or move slightly sideways in an adjacent lane when a car moves slowly towards the other car. These are all manoeuvres that pedestrians never need to do. 6) From the human drivers’ perspective, an AV may feel more like a peer, a different road user category from VRUs. Moreover, this topic has not yet been studied.

Besides these differences, we identified the following potential research topics that need to be studied more extensively:

1) other scenarios in addition to the ones studied, such as merging and overtaking; 2) solutions to make eHMIs understood by multiple AVs and MDVs in diverse encounters; 3) information needs in addition to the common cues that are currently being used in MDVs (e.g. indicators, brake lights); 4) the timing (onset and duration) for various forms of information; 5) location at which the information needs to be shown (e.g. displays on the roof of AVs, front, all-around); 6) suitable modality or the need for a multi-modal approach or the design of more natural concepts such as anthropomorphic or self-explaining concepts.

Only when the research gaps are filled and more knowledge is gained under various traffic scenarios and situations with a multi-user perspective can we start to develop a more holistic eHMI concept. Therefore, we encourage more eHMI studies for AVs to consider all RUs in mixed traffic. Safe and efficient mixed traffic can only be accomplished when all RUs’ safety and efficiency are guaranteed. From the perspective of eHMI design, a good eHMI should not be provided by simply adding many different interfaces for each particular target group on an AV but an eHMI that can be accepted and understood by multiple RUs.

REFERENCES

Ackermann, C., Trommler, D. and Krems, J., 2021, June. Exploring Cyclist- Vehicle Interaction–Results from a Naturalistic Cycling Study. In Congress of the International Ergonomics Association (pp. 533–540). Springer, Cham. Beggiato, M., Hartwich, F., Schleinitz, K., Krems, J., Othersen, I. and Petermann-

Stock, I., 2015. What would drivers like to know during automated driving? Information needs at different levels of automation. In 7. Tagung Fahrerassi- stenzsysteme. Bella, F. and Silvestri, M., 2017, July. Driver–cyclist interaction under different bicycle crossroad configurations. In International Conference on Applied Human Factors and Ergonomics (pp. 855–866). Springer, Cham.

External HMI for Automated Vehicles: Adding a Communication Perspective 83

Berge, S.H., Hagenzieker, M., Farah, H. and de Winter, J., 2022. Do cyclists need HMIs in future automated traffic? An interview study. Transportation research part F: traffic psychology and behaviour, 84, pp. 33–52. Carsten, O. and Martens, M.H., 2019. How can humans understand their auto-

mated cars? HMI principles, problems and solutions. Cognition, Technology & Work, 21(1), pp. 3–20. Chang, C.M., Toda, K., Sakamoto, D. and Igarashi, T., 2017, September. Eyes on a Car: an Interface Design for Communication between an Autonomous Car and a Pedestrian. In Proceedings of the 9th international conference on automotive user interfaces and interactive vehicular applications (pp. 65–73). De Clercq, K., Dietrich, A., Núñez Velasco, J.P., De Winter, J. and Happee, R., 2019.

External human-machine interfaces on automated vehicles: effects on pedestrian crossing decisions. Human factors, 61(8), pp. 1353–1370. Dey, D., Habibovic, A., Löcken, A., Wintersberger, P., Pfleging, B., Riener, A., Martens, M. and Terken, J., 2020. Taming the eHMI jungle: A classification taxo- nomy to guide, compare, and assess the design principles of automated vehicles’ external human-machine interfaces. Transportation Research Interdisciplinary Perspectives, 7, p.100174. Dey, D., Matviienko, A., Berger, M., Pfleging, B., Martens, M. and Terken, J.,

2021. Communicating the intention of an automated vehicle to pedestrians: The contributions of eHMI and vehicle behavior. it-Information Technology, 63(2), pp. 123–141. Dey, D. and Terken, J., 2017, September. Pedestrian interaction with vehicles: roles of explicit and implicit communication. In Proceedings of the 9th international conference on automotive user interfaces and interactive vehicular applicati- ons (pp. 109–113). Feierle, A., Danner, S., Steininger, S. and Bengler, K., 2020. Information needs

and visual attention during urban, highly automated driving—an investigation of potential influencing factors. Information, 11(2), p.62. Fridman, L., Mehler, B., Xia, L., Yang, Y., Facusse, L.Y. and Reimer, B., 2017. To walk or not to walk: Crowdsourced assessment of external vehicle-to-pedestrian displays. arXiv preprint arXiv:1707.02698. Hou, M., Mahadevan, K., Somanath, S., Sharlin, E. and Oehlberg, L., 2020, April.

Autonomous vehicle-cyclist interaction: peril and promise. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1–12). Imbsweiler, J., Ruesch, M., Weinreuter, H., León, F.P. and Deml, B., 2018.

Cooperation behaviour of road users in t-intersections during deadlock situa- tions. Transportation Research Part F: Traffic Psychology and Behaviour, 58, pp. 665–677. Kauffmann, N., Winkler, F., Naujoks, F. and Vollrath, M., 2018. “What Makes a Cooperative Driver?” Identifying parameters of implicit and explicit forms of communication in a lane change scenario. Transportation research part F: traffic psychology and behaviour, 58, pp. 1031–1042. Madigan, R., Lee, Y.M. and Merat, N., 2021. Validating a methodology for under-

standing pedestrian–vehicle interactions: A comparison of video and field obse- rvations. Transportation research part F: traffic psychology and behaviour, 81, pp. 101–114. Mahadevan, K., Somanath, S. and Sharlin, E., 2018, April. Communicating awareness and intent in autonomous vehicle-pedestrian interaction. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (pp. 1–12).

84 Gao and Martens

Faas, S.M., Kraus, J., Schoenhals, A. and Baumann, M., 2021, May. Calibrating Pedestrians’ Trust in Automated Vehicles: Does an Intent Display in an External HMI Support Trust Calibration and Safe Crossing Behavior?. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (pp. 1–17). Papakostopoulos, V., Nathanael, D., Portouli, E. and Amditis, A., 2021. Effect

of external HMI for automated vehicles (AVs) on drivers’ ability to infer the AV motion intention: A field experiment. Transportation research part F: traffic psychology and behaviour, 82, pp. 32–42. Phillips, E., Ososky, S., Grove, J. and Jentsch, F., 2011, September. From tools to teammates: Toward the development of appropriate mental models for intelli- gent robots. In Proceedings of the human factors and ergonomics society annual meeting (Vol. 55, No. 1, pp. 1491–1495). Sage CA: Los Angeles, CA: SAGE Publications. Rasouli, A., Kotseruba, I. and Tsotsos, J.K., 2017. Understanding pedestrian beh-

avior in complex traffic scenes. IEEE Transactions on Intelligent Vehicles, 3(1), pp. 61–70. Rettenmaier, M., Pietsch, M., Schmidtler, J. and Bengler, K., 2019, June. Passing through the bottleneck-the potential of external human-machine interfaces. In 2019 IEEE Intelligent Vehicles Symposium (IV) (pp. 1687–1692). IEEE. Rettenmaier, M., Albers, D. and Bengler, K., 2020. After you?!–Use of external

human-machine interfaces in road bottleneck scenarios. Transportation research part F: traffic psychology and behaviour, 70, pp. 175–190. Ritchie, O.T., Watson, D.G., Griffiths, N., Misyak, J., Chater, N., Xu, Z. and Mouzakitis, A., 2019. How should autonomous vehicles overtake other dri- vers?. Transportation research part F: traffic psychology and behaviour, 66, pp. 406–418. Sorokin, L., Chadowitz, R. and Kauffmann, N., 2019, May. A change of perspective:

Designing the automated vehicle as a new social actor in a public space. In Exten- ded Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (pp. 1–8). Stoll, T., Lanzer, M. and Baumann, M., 2020. Situational influencing factors on understanding cooperative actions in automated driving. Transportation research part F: traffic psychology and behaviour, 70, pp. 223–234. Strömberg, H., Bligård, L.O. and Karlsson, M., 2018, August. HMI of autonomous

vehicles-more than meets the eye. In Congress of the International Ergonomics Association (pp. 359–368). Springer, Cham. Walker, F., Dey, D., Martens, M., Pfleging, B., Eggen, B. and Terken, J., 2019, May.

Feeling-of-safety slider: Measuring pedestrian willingness to cross roads in field interactions with vehicles. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (pp. 1–6). Wang, P., Motamedi, S., Qi, S., Zhou, X., Zhang, T. and Chan, C.Y., 2021. Pedestrian interaction with automated vehicles at uncontrolled intersections. Transportation research part F: traffic psychology and behaviour, 77, pp. 10–25.

Cover of the proceedings volume Ergonomics In Design

Explore the complete digital volume

Ergonomics In Design

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-23-0

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →