Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Implications of Human-Machine Interface for Inclusive Shared Autonomous Vehicles

1 Department of Design, Politecnico di Milano, Italy ↗
2 Department of Mechanical Engineering, Politecnico di Milano, Italy ↗
Publication year
Digital Object Identifier10.54941/ahfe1002488

Abstract

Autonomous Vehicles (AVs), also known as self-driving cars, driverless cars, or robot cars, can perceive their environment and drive safely with little or no human inputs. Under the support of engineering, information science, anthropology, AVs have developed rapidly at the technical level, demonstrating to reduce human error operations and lessening road usage to save time. In the future, users will always be more released from driving tasks Self-driving vehicles can be more advantageous and feasible in public transportation than in private ones. Therefore, shared buses and logistics vehicles have been trial operation in various countries nowadays. Moreover, this technology will enable road traffic for people who cannot drive due to various physical and cognitive impairments. Therefore, inclusivity is often cited as one of the main reasons for promoting user acceptance of this technology, and it is fundamental in its application. Inclusive design can be achieved by identifying and addressing as many barriers to the Human-Machine Interface (HMI) as possible, focusing on human factors inside and outside the self-driving vehicle. It can enable groups with different needs to interact comfortably with the AVs and the traffic environment. Specifically, within a shared autonomous vehicle, we focus on transferring vehicle control between multiple users, reducing human error, and improving system availability. In addition, for many non-driving-related tasks (NDRT) derived from the gradual improvement of autonomous driving, the design of interactive devices and processes suitable for different groups and their cognition, the usability and comfort of the system will be improved, as well as the user experience. Concerning the scenes outside the vehicle, the impact of different types of external HMIs on the user experience of other user groups (e.g., young, old, cognitive, or physical disabilities, Etc.) attracted more attention from scholars. At the same time, communication channels and interfaces established between users and AVs will become more prominent on an inclusive basis. Consequently, HMI is essential for functional and inclusive driving automation, guaranteeing an efficient and satisfactory interaction between the automated system and different users.This paper provides an overview of the HMI challenges in shared driving automation from an inclusive design perspective, summarizing existing research on the role of HMIs in shared autonomous vehicles. The authors identified the fundamental changes in the way the user interacts with the car in shared autonomous vehicles using a systematic literature review including the following four steps: 1) identification of purposes and research questions of the literature review 2) definition of a literature search strategy by identifying a combination of sequential and iterative search queries; 3) analyze the retrieved articles compiling a concept matrix for each of them. As a result, relevant literature related to the research topic was selected; 4) identifying research gaps and inconsistent research results to make tacit domain meta-knowledge explicit. The paper will contain the analysis and discussion of the obtained data. Finally, the paper will discuss the future challenges for promoting a deeper exploration of inclusivity of HMIs for autonomous vehicles, also proposing the research avenues practical to increase the user's acceptance of this technology.

INTRODUCTION

Autonomous Vehicles (AVs), also known as self-driving cars, driverless cars, or robot cars, can perceive their environment and drive safely with little or no human inputs (Hu et al., 2020; Lim and Taeihagh, 2018). Under the sup- port of engineering, information science, and anthropology, AVs have been rapidly developed at the technical level, demonstrating to reduce human error operations and lessening road usage to save time (Ahangar et al., 2021). In the future, users will always be more released from driving tasks. Self-driving vehicles can be more advantageous and feasible in public transportation than private ones. Therefore, shared autonomous vehicles (SAVs) (Wang et al., 2021), also called robot-taxis or shuttles, are frequently mentioned when shared buses and logistics vehicles have been trial operation in various coun- tries. It is a purpose-built vehicle designed to safely move people from point A to point B and share them among multiple users. It offers a flexible service

© 2022. Published by AHFE Open Access. All rights reserved. 542

Implications of Human-Machine Interface for Inclusive Shared Autonomous Vehicles 543

Figure 1: Literature review process.

Figure available in the original PDF paper.

using mobile device applications to summon vehicles rather than finding and walking to them.

Moreover, this technology will provide road transportation for those una- ble to drive due to various physical and cognitive impairments. Therefore, shared autonomous vehicles (SAV) have great potential to alleviate the exi- sting travel obstacles many special groups face. Inclusivity is often cited as one of the main reasons for promoting user acceptance of this technology and as the basis for its application. The application of inclusive design in SAVs represents the future of accessible transportation.

As the level of automation increases, the control of the vehicle will gra- dually shift from humans to machines, and their interaction methods will also undergo essential changes (Burns et al., 2019). As a medium for inte- raction and information exchange between the system and the users, the human-machine interface (HMI) handles the conversion of the internal infor- mation form of the computer language into the external information form that humans can receive. It can enhance the user’s trust and acceptance of autonomous vehicles by ensuring the proper communication between the vehicle and the user, including those inside the car and pedestrians outside the car.

According to the authors, HMI is strategic in researching and developing autonomous vehicles’ interactive design and inclusive design. This research focuses on user research on the human-machine interface of SAV. We explore the rationale and necessity of existing research areas and methodologies by aggregating them. Compare the innovation points of different research dire- ctions and the effectiveness of different methods to understand the impact of the HMI on inclusive shared autonomous vehicles. Finally, find valuable research directions in the future.

METHODOLOGY

In the following, the four steps of performing a literature review are descri- bed, shown in Figure 1.

PURPOSE OF THE LITERATURE REVIEW Since this literature review aims to provide ideas for new research, it is impe- rative to clarify the research question. The targeted research topic has been described and obtained according to the state-of-the-art in Section I. This review focuses mainly on other research outcomes. Several different theoreti- cal frameworks and classification methods of autonomous driving HMI have been sorted out through the literature summary to understand the applica- tion of different HMIs for inclusive shared autonomous vehicles. This paper summarizes the previous research work related to the research gap of HMI

inside and outside the SAVs and demonstrates the timeliness and relevance of addressing these issues.

INFORMATION SOURCES AND SEARCH STRATEGY A structured approach was used to identify the literature that could be sele- cted. First, develop a protocol to guide and document the search process. The search protocol provides context and rationale for a literature search and describes the search scope, including the keywords, phrases, databases, and related publications used for the search (Nordhoff et al., 2019). The rese- arched topic locates at the intersection of “Shared Autonomous Vehicles,” “Human-Machine Interface,”and “Inclusive design.”The search keywords in this paper were expanded on these three-word terms. The expressions for sha- red autonomous driving vary across the literature. The search process tried to summarize similar expressions, such as shared automated, connected, auto- mated, and electric (CASE) vehicles, robot-taxis, automated shuttles, or SAVs. Essentially, discussing the social inclusion of this technology is about under- standing the accessibility needs and diversity of different groups of users. As for HMI, it is a defined field. This article focuses more on the design and user research perspective. To cover the identified research areas, the authors used the following query string: ((Shared Autonomous Vehicles OR Robot- taxis OR Automated Shuttles, OR SAVs OR CASE vehicles) AND (Inclusive Design OR Universal Design OR Accessibility OR Human diversity) AND (HMI OR HCI OR Interaction Design OR User and AVs).

In addition, the search was limited to the last decade, 2012-2022, in terms of emerging technology. The review was conducted on the three primary data- bases for scholarly publication: Web of Science, Scopus, and Google Scholar. Within these databases, keyword searches were conducted between Septem- ber 2021 and February 2022. The initial search, including all types of studies, including journal publications, articles from conference proceedings, theses, reports, posters, and presentation slides, yielded 694 articles, excluding any duplicates.

QUALITY APPRAISAL AND DATA EXTRACTION The searched materials were screened for relevance and quality to determine which could ultimately be used for the literature review (Bornholt and Heidt, 2019). Most of the documents are mainly based on urban planning and tech- nology research, they are not in line with the research direction and field, so they should be deleted. By reading the abstracts, we determined inclusion and exclusion criteria, thereby selecting material from the remaining relevant lite- rature that was relevant and suitable for our final analysis. On the one hand, the following publications were retained: (1) Investigating studies affecting social inclusiveness of SAV, (2) The impact of HMI design inside and outside the car on users, (3) Completing or full scientific research papers written in English.

On the other hand, we excluded (1) related research focusing on diffe- rent types of autonomous driving technologies (e.g., autonomous underwater,

Implications of Human-Machine Interface for Inclusive Shared Autonomous Vehicles 545

micro air, mining, and urban land vehicles), (2) technical, legal, ethical, and policy-related aspects of SAVs. Ultimately, we removed 276 records that did not fulfill our search criteria. Instead, we employed the iterative search appro- ach, backward and forward searches, to dig deep into some key literature instead of only targeting them in sequential searches. Backward searching refers to collecting relevant publications by screening the reference lists of the papers retrieved from the keyword search (“time-reversal”). Forward search- ing refers to collecting appropriate publications that have cited these papers (“time-forward”) (vom Brocke et al., 2015). We also excluded review-based studies, which already discussed the results of some of the studies that met our eligibility criteria. Therefore, 179 records were thus retained in the qualitative analysis in the final stage.

ANALYSIS OF FINDINGS AND WRITING THE REVIEW The literature was organized and analyzed by summarizing different conce- pts from different authors to synthesize the literature and creating a concept matrix through an excel sheet. All materials are divided into two categories: (1) the development and application of inclusive design in SAVs (2) research on internal and external HMI in SAVs.

In the first category of related literature, the development process and shift in focus of social inclusion research in SAVs are analyzed, summarizing the coverage areas and existing research gaps. For the second category of related literature, the concepts to be tested, assessment methods, carrier platforms, and experimental results involved in the study are summarized and analyzed. The search, reading, classification, and interpretation of the selected publica- tions were completed through the literature search, selection, and synthesis phases described above. A glossary and bibliography were created by summa- rizing and analyzing the data. This literature review concludes with finding directions or continued research, expanding existing theories, or developing new ones. Finally, research results are presented and written to complete the paper.

RESULTS

The literature is analyzed and synthesized by summarizing different conce- pts of various authors through the methods mentioned above. This section presents the analysis results, and all materials are divided into following two categories.

APPLICATION OF INCLUSIVE DESIGN IN SAVS Inclusive design is a design process methodology in which a mainstream pro- ducts, services, or environments are designed to be usable by as many people as reasonably possible, without the need for specialized adaptions. Although inclusive design can be inspired by engagement with extreme users, who have specific needs, it differs from disability-specific design in its mainstream focus (John Clarkson and Coleman, 2015). It considers many different aspects of

human diversity, such as ability, language, culture, gender, and age, which means including and learning from people with diverse perspectives.

For autonomous driving scenarios, inclusion refers to meeting the needs of people with disabilities, the elderly, and potentially marginalized t users of the transportation system to benefit a broader population. Few relevant studies often address mobility barriers for the aging populations while pro- viding safe, flexible, and accessible transportation for groups constrained by current modes of transportation due to disability. The blind and visually impaired often attract more attention from scholars.

Due to the inherently high-tech and high-cost attributes of autonomous driving technology, its application to public transportation can be more soci- ally inclusive, so this paper pays more attention to the inclusive design of shared autonomous driving technology. It can benefit groups whose mobility needs are not met by the current transportation system, especially in rural areas, where public transportation does not accommodate point-to-point services, and low-income users cannot afford customized personal mobility solutions. As the implementation of automated transportation systems acce- lerates, automated shuttles could increase mobility accessibility and reduce costs for marginalized populations by offering rideshare, door-to-door shuttle services, and online journey scheduling.

This type of research is an interdisciplinary field and is mostly combi- ned with user research and public transit. Qualitative analysis methods such as co-design, journey maps, and user interviews are used to observe, inter- pret, and translate people’s needs into vehicle design points, leading to online remote control and offline real interaction scenarios for automated shuttles. For example, Riener, Andreas, et al. developed an in-vehicle and out-of- vehicle interaction system for booking and riding Automated Shuttle Buses to improve information communication (John Clarkson and Coleman, 2015). Service capacity, travel costs, and cultural differences are now neglected in such studies, apart from accessibility and safety. Special groups need to receive greater attention and services than ordinary ones, such as treating sudden diseases for the elderly, interacting with disabled people, automated shuttles, Etc. In addition, it should also develop from a qualitative analysis- based research method to a combination of quantitative and qualitative research methods.

HMI IN SHARED AUTONOMOUS VEHICLES This section categorizes the existing research on the human-machine inter- face (HMI) of shared autonomous vehicles and users inside and outside the vehicle.

Collaborative Construction of HMI Inside the SAVs Due to the continuous enhancement of the functions and forms of onboard electronic equipment in recent years, the development of multi-modal and multi-channel integration of the interaction interface in the in-vehicle space has become the direction of development (Chen and Xiao, 2019; Neßelrath

Implications of Human-Machine Interface for Inclusive Shared Autonomous Vehicles 547

and Feld, 2013). The amount of information received by drivers and passen- gers in the vehicle increases gradually, and many cognitive stimuli can distract the driver’s attention. However, users who become passengers have entirely gotten rid of driving behavior and engage in some non-driving-related tasks (NDRTs) in SAVs (Yang et al., 2018). At this time, the in-vehicle scenario is endowed with entertainment, office, and other activities. The transparency of the information conveyed by the automated operating system to the users in the car is essential (Pokam et al., 2019), which is a significant way for SAVs to gain trust.

Existing research on SAVs is gradually combined with service design to create a sense of safety and comfort for passengers, (Kettwich et al., 2021) such as providing accurate travel information, remote control in emerge- ncies, and recreational functions during the journey. This emerging field uses qualitative analysis methods such as interviews, focus groups, surveys, and field studies to target a broader range of users to provide equitable and inclusive opportunities. At the same time, it also enhances the user’s emotional experience through various multimodal and anthropomorphic interactions, thereby increasing public awareness and acceptance of SAVs (Mahdavian et al., 2021). On this basis, the rational distribution of interior space, the inclusiveness of interaction methods, and the transfer of control rights among multiple users have received less attention. In addition, after the COVID-19 outbreak, the communication and interaction in closed public spaces in SAVs should break through the previous fixed mode, and a few stu- dies have proposed methods such as gestures and voice interaction to replace traditional ones (Sun et al., 2020).

Information Exchange of HMI Outside the SAVs At present, research on External HMI (eHMI) has become a hot spot. The eHMI is responsible for interacting with external road users through auditory and visual media. The current design of the eHMI for SAVs focuses on ele- ments such as position, color, information type, and technology (Dou et al., 2021). Most research focuses on the communication between the vehicle and the external environment, the display of the vehicle’s intentions, and the perceived risks and behaviors of vulnerable road users (VRUs). Howe- ver, less involved signal communication, road structure, user acceptance, and social norms. Since autonomous driving is still in the concept stage, written questionnaires, direct interviews, traffic reports, observations, eye-tracking devices, naturalistic recording of traffic scenes (both videos and photos), and other traditional methods for reproducing natural scenes may not reflect the actual behavior of users. On this basis, it is also affected by observer bias to a certain extent. The setting of interviews and questionnaires may be inductive, and the interpretation of observed behaviors may also have deviated. There- fore, the analysis of eHMI obtained by these experimental methods is more subjective and lacks objective data support. More suitable methods for stud- ying autonomous driving scenarios have been employed in recent research, such as Wizard of Oz, computer simulation, and experimental methods com- bining virtual reality and natural venues. Numerous case studies have proved

that many factors affect pedestrians’ behavior, roughly divided into two cate- gories: personal (gender, culture, etc.) and environmental factors (vehicle characteristics, dynamics, and physical signals). For VRUs, predicting the driving behavior of autonomous vehicles is essential. It includes identifying the mode of vehicle driving (manual or automatic), the expression of vehicle information about its intentions, and providing alternative mechanisms for eye contact.

All in all, the information provided by the vehicle should be presentable to inform VRUs of the AV’s intentions, rather than advisory information used to guide the user’s behavior outside the vehicle. Nowadays, many in- depth studies have been conducted on the eHMI form’s location (None, Text, Front Brake Lights), color, and lighting to analyze the impact of diffe- rent types of interaction concepts on users. By summarizing example studies with objectives, evaluation methods, experiment platforms, conclusions, and remark points on HMI assessment outside the vehicle, authors found that some scholars believe that eHMIs may lead to people’s over-reliance, and the implicit information of the vehicle is equally important. Among them, Moore et al. think that the implicit information is sufficient for the communication between AVs and pedestrians (Moore et al., 2019).

CHALLENGES AND FUTURE WORKS In this paper, through a literature review, authors understand the research status of SAVs. Although existing research has shown a trend of integrating with service design and inclusive design, most of the research is limited to the planning of journey routes and the overall perception of information. In the future, it should be analyzed which factors will affect users’ acceptance of this technology from the perspective of different groups of people.

On the one hand, the distribution and conversion of control rights among different users will focus on in-vehicle HMI. On this basis, improving the interaction of passenger NDRT and user experience is also another critical point. On the other hand, many scholars believe that the interplay of fully automated vehicles and VRUs is the future direction for HMI outside the car. Attention should be paid to the information exchange between vehicles and VRUs in specific scenarios, such as campuses, (Riener et al., 2021) tourist attractions, industrial parks. The design of inclusive eHMIs for visual and hearing impairments is significant among them. Such research should focus on visual impairment caused by environmental conditions such as weather and adopt multi-modal interfaces to increase social inclusion. The relation- ship between different levels of abstraction in information communication and the subjective and cognitive perception of VRU is also another direction worthy of future research.

CONCLUSION

This research conducted a literature review on the HMI and inclusive design of SAVs. Inclusive design can be achieved by identifying and addressing as many HMI barriers as possible, focusing on human factors inside and

Implications of Human-Machine Interface for Inclusive Shared Autonomous Vehicles 549

outside the self-driving vehicle. It allows groups with different needs to interact comfortably with the AVs and the traffic environment. Specifically, within a shared autonomous vehicle, we focus on transferring vehicle con- trol between multiple users, reducing human error, and improving system availability. In addition, for many non-driving-related tasks (NDRT) derived from the gradual improvement of autonomous driving, the design of intera- ctive devices and processes suitable for different groups and their cognition, the usability and comfort of the system will be improved, as well as the user experience. Regarding off-vehicle scenarios, the impact of different types of external HMIs on the user experience of other user groups (e.g., young, old, cognitive, or physical disabilities, Etc.) has attracted more attention from scholars. At the same time, communication channels and interfaces establi- shed between users and AVs will become more prominent on an inclusive basis.

In brief, this research has comprehensively analyzed the effects of HMI on functional and inclusive SAVs. It aims to guide the design of SAVs to guarantee an efficient and satisfactory interaction between the automated system and different users.

REFERENCES

Ahangar, M.N., Ahmed, Q.Z., Khan, F.A. and Hafeez, M. (2021), “A survey of autonomous vehicles: Enabling communication technologies and challenges”, Sensors (Switzerland), MDPI AG. Bornholt, J. and Heidt, M. (2019), Association for Information Systems Asso-

ciation for Information Systems To Drive or Not to Drive-A Critical Review Regarding the Acceptance of Autonomous Vehicles Completed Research Paper, available at: https://aisel.aisnet.org/icis2019/human_computer_interact/human_ computer_interact/5. Burns, C.G., Oliveira, L., Thomas, P., Iyer, S. and Birrell, S. (2019), “Pedestrian decision-making responses to external human-machine interface designs for autonomous vehicles”, IEEE Intelligent Vehicles Symposium, Proceedings, Vol. 2019-June, pp. 70–75. Chen, X. and Xiao, B. (2019), “Analysis of Automotive Smart Touch Interaction

Scheme Based on Driver Usage Scenario”, No. Iset, pp. 77–80. Dou, J., Chen, S., Tang, Z., Xu, C. and Xue, C. (2021), “Evaluation of multimodal external human–machine interface for driverless vehicles in virtual reality”, Symmetry, MDPI AG, Vol. 13 No. 4, available at: https://doi.org/10.3390/sy m13040687. Hu, J., Bhowmick, P., Arvin, F., Lanzon, A. and Lennox, B. (2020), “Coo-

perative Control of Heterogeneous Connected Vehicle Platoons: An Ada- ptive Leader-Following Approach”, IEEE Robotics and Automation Let- ters, Institute of Electrical and Electronics Engineers Inc., Vol. 5 No. 2, pp. 977–984. John Clarkson, P. and Coleman, R. (2015), “History of inclusive design in the UK”, Applied Ergonomics, Elsevier Ltd, Vol. 46 No. PB, pp. 235–247.

Kettwich, C., Schrank, A. and Oehl, M. (2021), “Teleoperation of highly automated vehicles in public transport: User-centered design of a human-machine interface for remote-operation and its expert usability evaluation”, Multimo- dal Technologies and Interaction, MDPI AG, Vol. 5 No. 5, available at: https: //doi.org/10.3390/MTI5050026. Lim, H.S.M. and Taeihagh, A. (2018), “Autonomous vehicles for smart and sustai-

nable cities: An in-depth exploration of privacy and cybersecurity implications”, Energies, MDPI AG, Vol. 11 No. 5, available at: https://doi.org/10.3390/en 11051062. Mahdavian, A., Shojaei, A., McCormick, S., Papandreou, T., Eluru, N. and Oloufa, A.A. (2021), “Drivers and Barriers to Implementation of Connected, Automated, Shared, and Electric Vehicles: An Agenda for Future Research”, IEEE Access, Institute of Electrical and Electronics Engineers Inc., Vol. 9, pp. 22195–22213. Moore, D., Currano, R., Strack, G.E. and Sirkin, D. (2019), “The case for impli-

cit external human-machine interfaces for autonomous vehicles”, Proceedings - 11th International ACM Conference on Automotive User Interfaces and Inte- ractive Vehicular Applications, AutomotiveUI 2019, Association for Computing Machinery, Inc, pp. 295–307. Neßelrath, R. and Feld, M. (2013), Towards a Cognitive Load Ready Multimodal Dialogue System for In-Vehicle Human-Machine Interaction Situations-Adaptive Multimodale Interaktion Für Innovative Mobilitätskonzepte Der Zukunft View Project Towards a Cognitive Load Ready Multimodal Dialogue Sy. Nordhoff, S., Kyriakidis, M., van Arem, B. and Happee, R. (2019), “A multi-level

model on automated vehicle acceptance (MAVA): a review-based study”, The- oretical Issues in Ergonomics Science, Taylor and Francis Ltd., Vol. 20 No. 6, pp. 682–710. Pokam, R., Debernard, S., Chauvin, C. and Langlois, S. (2019), “Principles of transparency for autonomous vehicles: first results of an experiment with an augmented reality human–machine interface”, Cognition, Technology and Work, Springer London, Vol. 21 No. 4, pp. 643–656. Riener, A., Schlackl, D., Malsam, J., Huber, J., Homm, B., Kaczmar, M., Kleitsch,

I., et al. (2021), “Improving the ux for users of automated shuttle buses in public transport: Investigating aspects of exterior communication and interior design”, Multimodal Technologies and Interaction, MDPI, Vol. 5 No. 10, available at: https://doi.org/10.3390/mti5100061. Sun, S., Wong, Y.D., Liu, X. and Rau, A. (2020), “Exploration of an integrated automated public transportation system”, Transportation Research Interdisciplinary Perspectives, Elsevier Ltd, Vol. 8, available at: https://doi.org/10.1016/j.trip.2020. 100275. vom Brocke, J., Simons, A., Riemer, K., Niehaves, B., Plattfaut, R. and Cleven,

A. (2015), “Standing on the shoulders of giants: Challenges and recommenda- tions of literature search in information systems research”, Communications of the Association for Information Systems, Vol. 37 No. August, pp. 205–224. Wang, Z., Safdar, M., Zhong, S., Liu, J. and Xiao, F. (2021), “Public Preferences of Shared Autonomous Vehicles in Developing Countries: A Cross-National Study of Pakistan and China”, Journal of Advanced Transportation, Hindawi Limited, Vol. 2021, available at: https://doi.org/10.1155/2021/5141798. Yang, Y., Karakaya, B., Dominioni, G.C., Kawabe, K. and Bengler, K. (2018), “An

HMI Concept to Improve Driver’s Visual Behavior and Situation Awareness in Automated Vehicle”, IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC, Vol. 2018-November, Institute of Electrical and Electronics Engineers Inc., pp. 650–655.

Cover of the proceedings volume Human Factors in Transportation

Explore the complete digital volume

Human Factors in Transportation

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-36-0

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →