Ali Alhawiti, Valerian Kwigizile, Jun-seok Oh, Zachary D Asher, Obaidullah Hakimi (2024). Exploring External Human Machine Interface Design for Autonomous Vehicle to Pedestrian Communication: Insights from Discussions and Drawing Sessions. In: Intelligent Human Systems Integration (IHSI 2024): Integrating People and Intelligent Systems. AHFE Open Access, vol. 119, pp. 49–58. https://doi.org/10.54941/ahfe1004469
The development of an effective external Human-Machine Interface (eHMI) for autonomous vehicles (AVs) is crucial for safe interaction with their environment. However, most current eHMI concepts are designed for one-to-one interaction and fail to consider complex interactions with multiple road users. To address this need, we conducted two focus group discussions and drawing sessions to gather participants’ feedback about AVs communication in mixed traffic situations and to come up with design ideas for eHMIs. In focus groups, eleven participants expressed mixed opinions about AVs, some of which were optimistic about their potential benefits, while others expressed skepticism about their accuracy and safety. All of the participants expressed a desire for clear communication from AVs, including information about the AV's mode, intention, surroundings detection, and advising. The participants also discussed the criteria for designing effective eHMIs, emphasizing the importance of simplicity, clarity, visibility, and not directing messages to specific individuals. In drawing sessions, participants generated 31 designs, with symbol modality being the most common. Multiple modalities were used more frequently in the designs, with text and symbol being the most common combination. Radiator grilles and windshields were the most popular display locations. Advice to pedestrians was the most frequently provided information type in the designs. Participants preferred using fewer information types in their designs, with most designs utilizing just one or two information types per display. Overall, this work provides valuable insights and guidance for the design, standardization, regulation, and overall development of eHMIs that is currently ongoing.
Autonomous vehicle (AV) technology is becoming more and more common on roads. As modern vehicles continue to evolve to full automation, there are general expectations of reduced congestion, increased efficiency, and
improved safety (Fagnant & Kockelman, 2015). However, as AV technol- ogy becomes more popular, it is important to ensure that they can interact safely and effectively with other road users, particularly vulnerable road users like pedestrians. Lack of a clear Vehicle-to-Pedestrian (V2P) communication channel may create difficulties for pedestrians in interpreting the AV’s inten- tions, leading to potential safety issues (Taeihagh & Lim, 2019). One solution to overcome this issue is using external Human-Machine Interfaces (eHMIs) (Dey et al., 2020; Liu et al., 2020; Mahadevan et al., 2018).
While eHMIs have the potential to improve safety and efficiency on the road, the majority of the current concepts are designed only for one-to-one interaction, where the AV interacts with a single user at a time (Colley et al., 2020; Dey et al., 2021; Holländer et al., 2022; Verstegen et al., 2021). This approach fails to take into account the complex interactions that occur on the road, where multiple road users interact with each other simultaneously. In such situations, different road users may perceive the eHMI differently, leading to misunderstandings and even dangerous circumstances. Accord- ing to (Verstegen et al., 2021), there is a lack of research on the ability of eHMIs concepts to clearly convey messages to multiple road users. Thus, there is a pressing need to understand how to design eHMIs that are effective in scenarios involving multiple road users.
The goal of this study is to explore pedestrians’ perception of the interac- tion with AVs in mixed traffic situations and to produce several design ideas for eHMIs. While previous research has examined pedestrian interactions with AVs, our study differentiates itself by focusing on complex mixed traffic scenarios, an aspect not fully explored in current research. The study involved conducting two focus group discussions and drawing sessions with partici- pants. The study’s findings can help manufacturers and developers of AVs to create effective eHMIs concepts. Ultimately, this research can contribute to the creation of a safer and more efficient transportation system.
METHODS
The study involved conducting two focus group discussions and draw- ing sessions with participants. This approach facilitated a comprehensive exploration of the participants’ perspectives and preferences regarding AV- pedestrian communication and helped to generate eHMIs design ideas. The one-hour discussion involved four sections: general thoughts on AVs, impres- sions of eHMIs, interaction considerations with AVs in mixed-traffic scenar- ios (see Figure 1, left), and designing eHMIs. In the final section, participants sketched their eHMI design ideas on a sheet (see Figure 1, right), which they then shared.
The study employed audio recordings of focus group sessions, which were transcribed, anonymized, and analyzed using thematic analysis with MaxQda software, aggregating similar participant responses for pattern identification. Drawing analyses were conducted manually, examining eHMI modalities, locations, and information types for recurring themes and design intricacies. Participants were compensated $20 at the end. The study was approved by the Western Michigan University Human Subjects Institutional Review Board.
Exploring External Human Machine Interface Design for Autonomous Vehicle 51
Figure 1: Multi-user road scenarios shown to participants (left) and sketching sheet to draw eHMIs concepts (right).
RESULTS AND DISCUSSIONS The study recruited a total of 11 participants. In the first group, there were six participants, while the second group had five participants. The age groups were well-represented, with two participants in the 18–24 age group, five in the 25–34 age group, and four in the 35–44 age group. In terms of gen- der, seven participants were male, and four were female. Employment status was also diverse, with five participants employed, five students, and one unemployed.
Discussion Groups Findings In the group discussion on AVs, participants’ opinions varied from optimism about their potential to concerns over safety and technology readiness. While some saw AVs as a pathway to more efficient and safer roads, others, like Speaker 4, questioned their reliability by saying “I’m a little sceptical of the accuracy and safety of these vehicles, especially when it comes to image recog- nition.” Safety concerns were tied to the level of vehicle autonomy, and some participants felt safer with the idea of a human backup driver as Partici- pant 9 said “I would like to see a driver in the car.” Trust and confidence were deemed essential for the interaction with AVs. Communication issues were a key concern, Speaker 2 mentioned the importance of eye contact and being aware of each other’s presence, asking, “But how can you do that with an autonomous vehicle?”. The discussion reflected a mix of hope for the tech- nology’s promise and the need for further development and transparency to gain public trust.
Participants reviewed various eHMI concepts for AVs and generally agreed that eHMIs could enhance communication between AVs and pedestrians. They stressed the need for these systems to be inclusive, particularly for indi- viduals with disabilities. For example, Speaker 9 said, “I think eHMIs should be accessible to everyone, including people with disabilities.” Concerns were raised about the “creepiness”of some displays, like simulated “eyes”on vehi- cles, and the practicality of street projections in poor visibility or on uneven surfaces. The use of universally recognized symbols and colors was suggested to improve understanding. Overall, while supportive of eHMIs, participants called for designs that prioritize accessibility, familiarity, and clarity.
The study involved participants discussing a hypothetical scenario where they interact with an approaching AV and other road users. They were ques- tioned about factors they would consider before crossing the road in such a scenario. Participants acknowledged the complexity of the situation, and a variety of factors were considered by participants. They mentioned the impor- tance of observing other pedestrians, using them as a reference for when to cross the road, like a family crossing, as indicated by Speaker 5. Participants also focused on the AV’s behaviour, such as its speed and whether there was sufficient time to cross safely, as mentioned by Speakers 4 and 9. A key con- cern was the clarity of the AV’s intentions and ensuring that signals intended for them were distinguishable from those directed at others. Speaker 2 high- lighted the difficulty in discerning if the AV’s signals were meant for a specific individual or another party. The overarching sentiment was the need for clear communication between AVs and pedestrians to make safe decisions in traffic.
Participants discussed the type of information they would like the AVs to provide. Key information includes:
1. Mode Indication: They want AVs to clearly display whether they are in manual or autonomous mode to understand who is controlling the vehicle. Speaker 6 said, “I like an ‘M’ or an ‘A’ on the screen, telling it’s automatic or manual” 2. Detection Capability: Information about the AV’s ability to detect and be aware of its surroundings, including pedestrians, is considered crucial for safety. Speaker 8 explains, “I want to know if it can see me.” 3. Intentions of AV: Clear signals from AVs regarding their intentions, such as turning or stopping, are needed to help pedestrians make informed decisions. Speaker 2 noted, “Tell me whether it will stop”. 4. Stopping Parameters: Details on where the AV will stop and for how long, possibly through a countdown system, would assist in predicting the vehicle’s behavior. Speaker 9 said, “ I would prefer to have a line that it will stop here and also for how long it will stop.” 5. Guidance and Advice: Some participants expressed a desire for the AV to offer crossing advice, although there were concerns about the potential for confusion or danger in situations with multiple road users as noted by Speaker 7, “What if the other car hits me?”
This discussion underlines the necessity for transparent and effective com- munication to enhance pedestrian safety around AVs. These findings align with those made by (Schieben et al., 2019).
Lastly, participants in the discussion highlighted several key criteria for the design of effective interfaces, particularly for Autonomous Vehicles (AVs). They underscored the importance of using universal signs that are widely rec- ognized to transcend cultural and language barriers. Simplicity in design was emphasized to ensure the interface is easy to read and understand. Clarity and high visibility were also considered essential, with designs needing to be clear, sufficiently large, and bright enough for easy viewing. Accessibility for people with disabilities was mentioned, with suggestions like using blinking signals for color-blind individuals. Interfaces should avoid being overly distracting
Exploring External Human Machine Interface Design for Autonomous Vehicle 53
with excessive colors or text. Audio cues were recommended to assist those with visual impairments. Information provided by the interface should be comprehensive yet concise, avoiding information overload. Messages should not be directed at specific individuals to prevent confusion in crowded areas, and standardization across all AVs was desired to ensure consistency in communication.
Drawing Sessions Findings Participants came up with 31 different designs for conveying the yielding and non-yielding behavior of AVs (see Figure 2). Fourteen concepts were designed by the first group participants including 8 concepts for yielding AV and 6 designs for non-yielding AV. The second group proposed 17 concepts, 10 for yielding and 7 for non-yielding AV.
Figure 2: Examples of participants eHMIs design.
Design Modalities The designs utilized a blend of modalities, with symbols being the predomi- nant choice, featured in 28 designs. Text was included in 19, and audio cues were part of 14 designs. It appears that multiple modalities are used more frequently in designs. The chart (see Figure 3) provides a summary of the different combinations of modalities used in the designs. For both types of AV behavior, the most common modality combination is Text + Symbol, fol- lowed by Symbol + Audio, indicating that multiple modalities are often used together to convey information effectively.
In terms of text modalities, words used in the designs serve different pur- poses, with some focused on yielding behavior (e.g., “walk,” “cross,” “you may cross the road,” “go”) and others on non-yielding behavior (e.g., “do not cross,” “driving,” “do not walk,” “stop”). Some words, like “Auto,” are used in both yielding and non-yielding designs to indicate the vehicle’s automa- tion status. For yielding AV designs, the most used words were “Walk” in six designs and “Cross” in two concepts while “driving” and “stop” were found
in two non-yielding AV designs (see Figure 4). The majority of participants’ designs included only one text per display, and only two designs involved two texts or phrases in one display (e.g., “walk + car will move in sec”).
Figure 3: Modalities participants used in the eHMIs design.Figure 4: Most used text words (left) and symbols (right) in the eHMIs design.
Symbols were the most prevalent modality type, appearing in 28 designs. A variety of symbol modalities were used in participants’ design ideas to con- vey the yielding and non-yielding behavior of the AV. Some symbols were specific to either yielding or non-yielding designs, while others were used in both types. For example, the “countdown timer,” “walking man,” “green
Exploring External Human Machine Interface Design for Autonomous Vehicle 55
LED strip” and “car icon” were more prevalent in yielding designs, while the “red stop hand,” “no pedestrian crossing sign,” “red headlights,” and “red LED light” were more common in non-yielding designs. “Arrows,” “A sign,” and “robot sign” were used across both yielding and non-yielding designs. A summary of the most symbols used in yielding and non-yielding AV designs is presented in (see Figure 3). For yielding AV designs, the most frequently used symbols are the countdown timer (9 occurrences) and the walking man (7 occurrences), followed by the green LED strip (3 occurrences). These sym- bols are employed to indicate safe crossing conditions for pedestrians. On the other hand, for non-yielding AV designs, the red stop hand is the most com- mon symbol, appearing 7 times, and the no pedestrian crossing sign is used in 3 instances. Moreover, the A sign and robot sign are used in both yielding and non-yielding designs, with 3 and 2 occurrences for yielding designs and 3 and 4 occurrences for non-yielding designs, respectively. In terms of audio modality, a total of 14 designs, 9 yielding AV and 5 non- yielding AV, incorporate audio to convey information. No design relies solely on audio for either yielding or non-yielding AV. Audio is combined with other modalities, such as text and symbols. For example, an audible countdown timer is suggested to accompany a visual countdown timer, which informs pedestrians about the remaining time to cross safely. Also, a combination of the red stop hand symbol with an A sign is accompanied by a sound, alerting pedestrians of the non-yielding AV.
Design Location and Information Type A variety of display locations have been proposed in the designs including the windshield, side, radiator grille, projection, and top of the vehicle. The radiator grille was the most frequently used location, appearing in 20 designs. The windshield was used in 16 designs, the side in 7 designs, projection in 3 designs, and the top in 2 designs. The radiator grille is used in 11 yielding AV designs and 9 non-yielding AV designs. This location’s popularity could be due to its visibility. For the windshield, there were 11 instances of yielding AV designs and 5 instances of non-yielding AV designs. Regarding the side of the vehicle, this location was used in 4 yielding AV designs and 3 non-yielding AV designs. Although not as popular as the windshield and radiator grille, the side of the vehicle remains a viable option for displaying messages. It ensures that pedestrians on the sidewalks or at crosswalks can clearly see the information. The most common combinations of display locations used in the designs were combinations of radiator grille and windshield in 6 occurrences.
The information types in the designs were grouped based on the primary message they convey to pedestrians. They were categorized into four informa- tion types: Automated Mode, Intention, Awareness of the Surroundings, and Advice. The most frequently provided information type in the designs was advice, with 15 instances for yielding AVs and 10 instances for non-yielding AVs (see Figure 5). This information provides instructions for pedestrians on actions they should take, like “walk,” or “stop”.
The Automated Mode information, indicating whether the vehicle is self- driving, was equally used in yielding and non-yielding designs, highlighting the importance of communicating the vehicle’s control status to pedestrians.
Figure 5: Type of information used in the eHMIs design.
Moreover, the Intention information, revealing the vehicle’s next actions, was more common in yielding designs, suggesting participants felt it was crucial for pedestrians to understand a yielding vehicle’s behaviour. Aware- ness of surroundings was the least used, indicating other information types were prioritized for immediate pedestrian safety. In terms of the number of information used in a display, most designs utilized one or two types of infor- mation. The most common combinations were automated mode and advice in 8 designs, and advice and intention were in 7 designs.
CONCLUSION
This study aimed to explore the design of effective eHMIs for AVs through a combination of discussion groups and drawing sessions. The findings from both techniques revealed the importance of clear and accessible communi- cation between AVs and pedestrians. Incorporating familiar symbols, color coding, and multiple modalities (text, symbols, audio) in eHMI designs were important factors in enhancing understanding and facilitating effec- tive communication. Participants also emphasized the need for concise and non-distracting displays, highlighting the importance of presenting informa- tion in a manner that avoids overwhelming pedestrians and allows them to remain attentive to their surroundings. The findings from both methods also underscore the importance of considering specific information types in eHMI designs. Clear indications of the AV’s mode, its intentions, and specific advice for pedestrians were identified as valuable information types that can enhance pedestrians’ understanding and enable safer interactions with AVs. Addition- ally, the inclusion of a countdown timer in the eHMI design was identified as a valuable feature, providing pedestrians with a visual indication of the
Exploring External Human Machine Interface Design for Autonomous Vehicle 57
remaining time to safely cross the road. These findings emphasize the sig- nificance of providing pedestrians with timely and relevant information to support their decision-making and ensure their safety. Despite these valuable findings, the study had certain limitations. The sample size was relatively small, which may not fully capture the diverse perspectives and preferences of all road users. Additionally, the study focused mainly on theoretical concepts and ideas, without evaluating the practical implementation and effectiveness of the suggested eHMI designs. Future work should address these limita- tions by conducting studies with larger and more diverse participant groups to ensure broader representation. Moreover, the development and testing of prototype eHMI systems based on the insights from this study can provide valuable information on their real-world effectiveness and potential areas for improvement.
ACKNOWLEDGMENT
The authors would like to acknowledge the great guidance and assistance of Professor Valerian Kwigizile, Jun-Seok Oh, Zachary D. Asher, and my colleague Obaidullah Hakimi.
REFERENCES
Colley, M., Walch, M., & Rukzio, E. (2020). Unveiling the Lack of Scalability in Research on External Communication of Autonomous Vehicles. Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, 1–9. https://doi.org/10.1145/3334480.3382865 Dey, D., Habibovic, A., Löcken, A., Wintersberger, P., Pfleging, B., Riener, A.,
Martens, M., & Terken, J. (2020). Taming the eHMI jungle: A classification tax- onomy to guide, compare, and assess the design principles of automated vehicles’ external human-machine interfaces. Transportation Research Interdisciplinary Perspectives, 7. Scopus. https://doi.org/10.1016/j.trip.2020.100174 Dey, D., Van Vastenhoven, A., Cuijpers, R. H., Martens, M., & Pfleging, B.
(2021). Towards scalable eHMIs: Designing for AV-VRU communication beyond one pedestrian. Proceedings - 13th International ACM Conference on Automo- tive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2021, 274–286. Scopus. https://doi.org/10.1145/3409118.3475129 Fagnant, D. J., & Kockelman, K. (2015). Preparing a nation for autonomous vehicles: Opportunities, barriers and policy recommendations. Transportation Research Part A: Policy and Practice, 77, 167–181. https://doi.org/10.1016/j.tra.2015.04. 003 Holländer, K., Hoggenmüller, M., Gruber, R., Völkel, S. T., & Butz, A. (2022). Take It
to the Curb: Scalable Communication Between Autonomous Cars and Vulnerable Road Users Through Curbstone Displays. Frontiers in Computer Science, 4. https: //www.frontiersin.org/articles/10.3389/fcomp.2022.844245 Liu, Y., Lyu, Y., Böttcher, K., & Rötting, M. (2020). External Interface-based Autonomous Vehicle-to-Pedestrian Communication in Urban Traffic: Commu- nication Needs and Design Considerations. International Journal of Human– Computer Interaction, 36(13), 1258–1272. https://doi.org/10.1080/10447318. 2020.1736891 Mahadevan, K., Somanath, S., & Sharlin, E. (2018). Communicating Awareness and
Intent in Autonomous Vehicle-Pedestrian Interaction. Proceedings of the 2018
CHI Conference on Human Factors in Computing Systems - CHI ‘18, 1–12. https: //doi.org/10.1145/3173574.3174003 Schieben, A., Wilbrink, M., Kettwich, C., Madigan, R., Louw, T., & Merat, N.
(2019). Designing the interaction of automated vehicles with other traffic partici- pants: Design considerations based on human needs and expectations. Cognition, Technology and Work, 21(1), 69–85. Scopus. https://doi.org/10.1007/s10111- 018-0521-z Taeihagh, A., & Lim, H. S. M. (2019). Governing autonomous vehicles: Emerging responses for safety, liability, privacy, cybersecurity, and industry risks. Transport Reviews, 39(1), 103–128. https://doi.org/10.1080/01441647.2018.1494640 Verstegen, R., Dey, D., & Pfleging, B. (2021). CommDisk: A Holistic 360
eHMI
Concept to Facilitate Scalable, Unambiguous Interactions between Automated Vehicles and Other Road Users. Adjunct Proceedings - 13th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applica- tions, AutomotiveUI 2021, 132–136. Scopus. https://doi.org/10.1145/3473682. 3480280
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Intelligent Human Systems Integration (IHSI 2024): Integrating People and Intelligent Systems
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-95-7
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional