INTRODUCTION
The advancement of Human-Machine Interface (HMI) systems within the automotive sector marks a significant chapter in the interface design nar- rative, encapsulating a progression from rudimentary mechanical indicators to the contemporary touchscreen interfaces that define the modern vehicle dashboard. Tracing back to the origins, as outlined by Meixner et al. (2017), automotive HMIs have transitioned from simple, mechanical interfaces to sophisticated touchscreen systems, reflecting broader technological and soci- etal shifts. This evolution from basic instruments displaying essential vehicle metrics to integrated systems offering a blend of information, entertainment,
© 2024. Published by AHFE Open Access. All rights reserved. 373
374 Trijard-Caddy
and connectivity highlights the dynamic nature of automotive interface design (Meixner et al., 2017).
Today, the focus is increasingly on creating large-screen interfaces that are not merely functional but are also intuitive, enhancing the driving experi- ence by leveraging the latest advancements in touchscreen technology and user-centred design principles. This transition towards more adaptive and personalised HMI systems, inspired by the ubiquity and versatility of per- sonal digital devices, underscores the industry’s commitment to aligning automotive interfaces with the expectations and behaviours of the modern user (Meixner et al., 2017). Mayer et al. (2018), Ahmad et al. (2015), and Liu et al. (2022) further enrich our understanding, examining the nuances of touchscreen usability and the impact of interface design on driver safety and interaction quality. The challenges and opportunities presented by touch- screen in-vehicle HMIs not only contribute to the ongoing discourse on automotive interface evolution but also pave the way for future innovations aimed at enhancing user experience and safety in an new automotive era.
METHODOLOGY
A targeted keyword search was performed on the Web of Science database, focusing on touchscreen automotive HMI design. The keywords used for this search were “HMI,” “Touchscreen HMI,” “Touchscreen,” “Human Machine Interfaces,” “Full-touch HMI,” “Touchscreen User Interface,” “User-Interface,” “In-vehicle Touchscreen HMI,”“Vehicle HMI,”and “Auto- motive HMI.” This approach aimed to capture a broad spectrum of research pertinent to touchscreen interface technology in automotive applications. From the initial findings, 40 papers were selected based on relevance and their publication’s impact factor, ensuring a focus on significant contributions to the field of touchscreen HMI. The final step in the selection process refined this list to 8 papers that were specifically relevant to the core themes of touch- screen interface design, usability, adaptability, and the potential increased risk on driver safety. This focused selection ensures that the review compre- hensively covers studies that provide critical insights into how touchscreen interfaces in vehicles affect user interaction quality and safety, highlighting the importance of these factors in the development and evaluation of future automotive HMI systems.
Touchscreen In-Vehicle Interfaces Mayer et al. (2018) delve into the nuanced challenges faced by in-vehicle touchscreen interaction systems, particularly under the varying conditions of road perturbations. Mayer et al. demonstrate that despite advanced vehicle suspension systems designed to mitigate road inconsistencies, road pertur- bations significantly impair touch interaction, with speed bumps specifically identified as reducing touch accuracy by up to 19%. To counteract these perturbations, Mayer et al. (2018) introduce an innovative solution through the development of a Random Forest model. This model is not only adept at improving touch accuracy significantly, showcasing a 32% improvement on the test set and a 22.5% improvement on the validation set. Despite its
Enhancing Automotive Interface Design: Insights From Touchscreen Usability 375
focused exploration on the impacts of vertical accelerations caused by road bumps the study acknowledges the need for further research into other per- turbation types, such as lateral accelerations and vibrations from uneven road surfaces, Mayer et al. (2018). Moreover, it suggests the exploration of deep learning models for touch interaction improvement, using more com- prehensive datasets collected in real-world driving scenarios to address a broader range of driving conditions and vehicle movements. Mayer et al. (2018) highlight the imperative for adaptive touch interaction systems that can dynamically compensate for a variety of environmental and vehicular movements, thereby enhancing the safety and efficiency of touchscreen usage in the evolving landscape of in-vehicle HMI technology.
Ahmad et al. (2015) examine how road conditions influence the usability of in-vehicle touchscreens in vehicles. Central to their findings is the reve- lation that as road conditions worsen, the rate of failed selection attempts increases with instances of error rates reaching 75% under severe pertur- bations, Ahmad et al. (2015). This underscores a significant challenge in the design of in-vehicle HMI’s, especially under conditions of varied pertur- bations. Notably, the study highlights the marginal errors made by users, suggesting that slight adjustments to GUI design could mitigate many errors, thereby pointing to the feasibility of enhancing touchscreen usability through thoughtful design considerations. However, the research also identifies gaps, such as the need for broader participant representation in future studies and the exploration of a wider range of driving scenarios to gain more compre- hensive insights. Furthermore, the study opens avenues for future research into intent-aware displays and other technologies as potential methods to improve usability under challenging conditions, Ahmad et al. (2015). These findings are pivotal, suggesting that despite the susceptibility of current in- vehicle touchscreen systems to errors induced by road conditions, strategic HMI design and innovative interaction techniques could greatly improve performance and safety, Ahmad et al. (2015).
Liu et al. (2022) delve into the consequences of the growing sizes of in- vehicle touchscreens on user interaction and vehicular safety. The research critically examines how the dimensions and placements of touchscreen con- trols influence user engagement and overall safety, highlighting the critical nature of the precision of click and slide gestures for driver performance, Liu et al. (2022). Through a series of two experiments, the investigation seeks to pinpoint areas of the touchscreen that either support or impede user interac- tion, with the goal of providing direction for the enhancement of in-vehicle HMI design.
Results from the initial experiment reveal a strong correlation between the size of the touch controls and click offsets, indicating that larger control sizes might detract from interaction accuracy. Importantly, zones obscured by the driver’s arm or situated far from the driver were pinpointed as areas where precise inputs are challenging, Liu et al. (2022). The second exper- iment broadens the analysis to slide gestures, sorting them into accessible, normal, and challenging categories based on user performance, thus spot- lighting regions for optimal and suboptimal gesture interactions, Liu et al. (2022). The study recognises the demand for additional inquiry, advocating
376 Trijard-Caddy
for the incorporation of physiological and eye movement indicators to deepen the comprehension of optimal operation areas. The contributions made by this research emphasise the need to factor in control size and the positioning of screens in the design of In-Vehicle Information Systems (IVIS) to enhance driving safety and operational efficiency, offering essential advice for automo- tive designers on improving touchscreen interface design. It is recommended that designers position buttons that are frequently used closer to drivers. Data necessitating sliding gestures ought to be displayed at the centre of the screen, deemed the prime spot for such interactions. Additionally, the relationship between hot zone size and click accuracy could act as a guide when selecting an appropriate hot zone size to elevate the click accuracy for the majority of users, Liu et al. (2022).
Mayer et al. (2018), Ahmad et al. (2015), and Liu et al. (2022) col- lectively address how road conditions and design affect touchscreen use in vehicles, underscoring the importance of adaptive and ergonomic interface design. Mayer et al. (2018) reveal that road bumps significantly decrease touch accuracy, a challenge they counter with a Random Forest model, illus- trating the potential of machine learning to enhance touch interactions amid environmental perturbations. Ahmad et al. (2015) uncover that deteriorating road conditions lead to increased touchscreen errors, advocating for smarter HMI designs to reduce these errors. Liu et al. (2022) examine the ergonomic implications of increasing touchscreen sizes and placements, suggesting prac- tical design adjustments for improved usability and safety. Crucially, Liu et al. (2022) also emphasises the strategic placement of buttons for high- frequency use and the central display of information requiring slide gestures, adding another layer to the design considerations for in-vehicle touchscreens. Together, these studies highlight the evolving need for in-vehicle touchscreens to be both adaptable to external conditions and intuitively designed, paving the way for innovations that promise to make in-vehicle touchscreens more intuitive and safer for drivers.
Driver Gaze and Distraction Ma et al. (2020) explore the intricate relationship between HMI displays and driver distractions, particularly emphasising the potential of the Random For- est data mining technique to model critical driving distraction indicators like speed deviation and lane departure standard deviation. Their study, which utilises data from a driving simulator involving 24 drivers across various car models, not only highlights the effectiveness of advanced data mining in capturing nuances of driving distractions but also identifies key HMI design factors such as target speed, horizontal angle, and on-screen finger move- ment that significantly influence driver distraction. The study underscores the necessity of optimising HMI design to mitigate distractions by advocat- ing for simpler tasks, reduced on-screen finger movement, and appropriate interaction angles. Ma et al. (2020) call for further research incorporating a broader set of variables related to UI design and exploring more complex driving scenarios to refine design recommendations.
Enhancing Automotive Interface Design: Insights From Touchscreen Usability 377
Gonçalves et al. (2022) delve into the effects of dash-based HMI designs on drivers’ gaze behaviours and decision-making during the transition from automated to manual driving. They examine how vehicle automation, HMI informational content, and the proximity of vehicles in the offside lane influ- ence drivers’ Decision-Making Time (DMT). Their findings suggest that the vehicular environment plays a significant role in modulating DMT, pointing to a nuanced interaction between environmental cues and HMI information in shaping lane-change manoeuvres. The study emphasises the need for future HMI design research to focus on strategies that enhance driver support sys- tems while minimising distractions, advocating for a balanced approach that supports safe and informed decision-making.
Ulahannan et al. (2020) investigate driver expectations and information preferences in relation to in-vehicle HMI, employing a driving simulator to distinguish between High Information Preference (HIP) and Low Information Preference (LIP) drivers based on their desires for system status information. The study highlights the critical need for adaptive HMI designs that cater to diverse driver expectations to ensure the safe use of partially automated driving technologies. By advocating for interface designs that are adapt- able and minimise cognitive overload, Ulahannan et al. (2020) contribute to the discourse on optimising HMI designs in the burgeoning field of vehi- cle automation, stressing the alignment of HMI design with user expectations for a safer, more intuitive driving experience.
Together, these studies highlight the significance of enhancing HMI design for safety and efficiency, illustrating the utility of data mining in informing intuitive HMI solutions and pinpointing a demand for adaptive, user-centred HMI designs. Future research is encouraged to encompass broader demo- graphic studies, engage with more intricate driving scenarios, and achieve a deeper understanding of UI design principles. This collective body of work points towards a comprehensive approach in future HMI development aimed at augmenting road safety, efficiency, and improved user experience.
Interaction Speed and Accuracy In the evolving landscape of automotive user interfaces, the shift from tra- ditional physical controls to touchscreen systems presents new challenges and opportunities for interaction design. Ng and Brewster (2017) delve into this shift, evaluating the usability and effectiveness of various input methods for list-based scrolling tasks on in-vehicle touchscreens. The study compares three distinct input techniques: direct scrolling, pressure-based scrolling, and on-screen buttons. Each method is evaluated for its accuracy and speed, revealing a nuanced understanding of how different input modalities can affect user interaction with in-vehicle systems. A notable finding is the inher- ent trade-off between speed and accuracy in these input methods, with direct scrolling emerging as the quickest but least accurate, especially for smaller tar- get sizes (4mm). This insight underscores the complexity of designing touch interfaces that can accommodate the diverse needs of drivers, particularly in terms of target size and the precision required for safe interaction, Ng and Brewster (2017).
378 Trijard-Caddy
Ng and Brewster (2017) explore the potential of haptic (vibrotactile) feed- back to enhance user performance and satisfaction. While the addition of haptic feedback did not statistically improve performance metrics such as accuracy or selection time, it was subjectively preferred by participants, Ng and Brewster (2017). This preference suggests that haptic feedback could play a significant role in improving the user experience, even if its direct impact on performance is limited. The subjective preference for haptic feedback high- lights the importance of considering user perceptions and comfort in the design of in-car touchscreens, beyond mere functional performance metrics. Ng and Brewster (2017) identify several gaps and directions for future investi- gation, including the long-term effects of using these input methods, potential user fatigue, especially with pressure inputs, and the performance of these methods in real driving conditions. Another critical area for future research is the development of adaptive interfaces that can dynamically adjust to user needs and preferences, potentially enhancing both safety and efficiency. Ng and Brewster (2017) offer valuable insights into the design of in-car touch- screen interfaces, emphasising the intricate balance between speed, accuracy, and user satisfaction. By highlighting the nuanced trade-offs inherent in dif- ferent input techniques and the potential benefits of haptic feedback, this research contributes to a deeper understanding of how to optimise touch- screen interfaces for the unique demands of the automotive environment. The study sets the stage for further exploration of adaptive, user-centred design approaches that prioritise safety, efficiency, and user satisfaction, ultimately guiding the future of interaction design in automotive contexts.
In the context of the automotive industry’s ongoing transformation towards integrating larger, more interactive touch-screens within vehicle inte- riors, as mentioned in Liu et al. (2022), Breitschaft et al. (2022) presents a pivotal examination of electrostatic friction modulation (EFM) as an innova- tive haptic feedback technology. Breitschaft et al. (2022) made comprehensive evaluation of EFM within an automotive setting, leveraging a dual-task framework that incorporates both a primary driving task and a secondary target-selection task. This methodological approach was designed to simu- late the multifaceted interactions drivers engage with, offering insights into how EFM can impact user performance and experience in real-world sce- narios. Through this lens, the study aims to address the tactile deficiency observed in current touchscreen interfaces, which often lack the physical feedback intrinsic to traditional control systems, Breitschaft et al. (2022).
Breitschaft et al. (2022) reveal intriguing aspects of user interaction with EFM-enabled touchscreens. Notably, the research identified no significant difference in user performance metrics—such as response time, errors, and primary task performance—across the various feedback modalities examined (haptic, audio, and visual). This outcome suggests that the introduction of EFM does not detrimentally affect the fundamental aspects of user perfor- mance within the tested automotive multitask environment. However, the study uncovered a pronounced preference among users for multi-sensory feedback, with a combination of visual, audio, and haptic cues being favoured over singular feedback modalities, Breitschaft et al. (2022). Interestingly, while haptic feedback through EFM was anticipated to significantly enhance
Enhancing Automotive Interface Design: Insights From Touchscreen Usability 379
the user experience by restoring tactile engagement lost in the transition to touchscreens, audio feedback emerged as more impactful in elevating user experience, attributed possibly to its higher perceived salience and user famil- iarity. Despite this, audio cues were also reported to be more annoying than their haptic counterparts, indicating a delicate balance between enhancing user experience and avoiding sensory overload or irritation, Breitschaft et al. (2022).
Breitschaft et al. (2022) open up several avenues for future inquiry, including the need for stronger and more familiar haptic feedback, the exploration of EFM’s applicability in genuine driving contexts, and the devel- opment of adaptive, context-sensitive haptic designs that can dynamically cater to the user’s needs and environmental conditions. These gaps point to the broader challenge of integrating advanced haptic technologies into automotive interfaces in a manner that is both functionally effective and user-centric.
Ng and Brewster (2017) and Breitschaft et al. (2022) mark significant steps towards enhancing in-car touchscreen interfaces with advanced haptic feed- back. This move towards more interactive touchscreens and responds to the need for safer, more intuitive user interfaces in vehicles, highlighting a shift from solely touchscreen controls to touch-based inputs enriched with tac- tile feedback. Both sets of research underscore a clear user preference for interfaces that combine visual, auditory, and tactile feedback, even though EFM’s initial reception noted it as weaker than expected. Further research should include the need for stronger haptic feedback, more realistic test- ing conditions that simulate actual driving, and the development of adaptive interfaces that respond dynamically to user needs and environmental factors. Future research directions are aimed at refining EFM technology to make it more effective and familiar to users, thereby improving safety, efficiency, and satisfaction in vehicle interactions.
Future Research Directions Mayer et al. (2018), Ahmad et al. (2015), Liu et al. (2022), Ma et al. (2020), Gonçalves et al. (2022), Ulahannan et al. (2020), Ng and Brew- ster (2017), and Breitschaft et al. (2022) have significantly advanced the understanding of the challenges faced in-vehicle HMI design. Despite the current advancements, notable research gaps and future directions emerge from their findings. A recurring theme across these studies is the need for a deeper exploration into the effectiveness and user perception of haptic feed- back mechanisms, with particular emphasis on achieving a stronger and more intuitive tactile response (Breitschaft et al., 2022). Additionally, the impact of environmental conditions such as road perturbations on touch interac- tion accuracy underscores the urgency for developing adaptive HMI systems that maintain performance across diverse driving scenarios (Mayer et al., 2018). The literature also highlights a gap in understanding the long-term effects of using advanced HMIs, suggesting future studies should simulate real-driving conditions more extensively to validate the practical applicabil- ity of these technologies (Ng and Brewster, 2017). Moreover, there is a call
380 Trijard-Caddy
for broader demographic studies and the incorporation of a wider range of driving scenarios to ensure that forthcoming HMI designs are inclusive and robust across varied user groups and operational environments (Ahmad et al., 2015). Addressing these gaps requires an interdisciplinary approach, inte- grating data mining, machine learning, and user-centred design principles to create HMIs that are not only technologically advanced but also adaptive, intuitive, and safe for all users.
CONCLUSION
The literature explored underscores the evolving challenges and opportuni- ties in automotive interface design, particularly highlighting the critical role of haptic feedback in enhancing touchscreen interfaces. Despite advance- ments, the research collectively reveals several gaps, such as the need for interfaces that can adapt to environmental changes, user preferences, and the importance of considering the long-term implications of HMI usage on driver behaviour and safety. Key findings across these studies indicate a preference for multi-sensory feedback mechanisms, suggesting that the integration of tactile, audio, and visual cues can improve user interaction and safety. However, challenges related to the effectiveness of haptic feed- back, such as those explored in EFM technology, point to the necessity for stronger and more intuitive tactile responses. Additionally, the impact of external conditions, like road perturbations on touch interaction accuracy, calls for the development of adaptive HMI systems that are robust across diverse driving scenarios. This review not only sheds light on the current state of automotive HMI development but also charts a course for future research. It emphasises the imperative of fostering adaptive, intuitive, and user-centred automotive interfaces. Through continued interdisciplinary col- laboration and innovation, future research has the potential to significantly advance the field of automotive HMI design, ultimately leading to safer and more engaging driving experiences.
ACKNOWLEDGMENT
The author thanks Shanghai Jiao Tong University for the scholarship that supported this work, marking the beginning of an ongoing research journey. Appreciation is extended to Professor Gu Zheng Yu and Liu Pinkuan for their guidance and support at this stage and into the future.
REFERENCES
Ahmad, Bashar I., Langdon, Patrick M., Godsill, Simon J., Hardy, Robert., Skrypchuk, Lee., and Donkor, Richard. 2015. Touchscreen usability and input performance in vehicles under different road conditions: An evalu- ative study. In Proceedings of the 7th International Conference on Auto- motive User Interfaces and Interactive Vehicular Applications (AutomotiveUI ‘15). Association for Computing Machinery, New York, NY, USA, 47–54. https://doi.org/10.1145/2799250.2799284
Enhancing Automotive Interface Design: Insights From Touchscreen Usability 381
Breitschaft, S. J., Pastukhov, A., and Carbon, C.-C. 2022. Where’s My Button?
Evaluating the User Experience of Surface Haptics in Featureless Automotive User Interfaces. IEEE Transactions on Haptics, vol. 15, no. 2, pp. 292–303. https://doi.org/10.1109/TOH.2021.3131058 Gonçalves, Rafael C., Louw, Tyron L., Madigan, Ruth., Quaresma, Manuela., Romano, Richard., and Merat, Natasha. 2022. The effect of information from dash-based human-machine interfaces on drivers’ gaze patterns and lane-change manoeuvres after conditionally automated driving. Accident Analysis & Preven- tion, 174, 106726. https://doi.org/10.1016/j.aap.2022.106726 Liu, Xinze, Sun, Hongli, Gao, Yan, Zhang, Weibin, Ge, Yan, Qu, Weina.
2022. Exploring the performance of click and slide gestures on large in-vehicle touchscreens. Applied Ergonomics, Volume 99, 103613, ISSN 0003-6870. https://doi.org/10.1016/j.apergo.2021.103613 Ma, Jun, Gong, Zaiyan, Tan, Jianjie, Zhang, Qianwen, Zuo, Yuanyang. 2020.
Assessing the driving distraction effect of vehicle HMI displays using data mining techniques. Transportation Research Part F: Traffic Psychology and Behaviour, 69, 235–250. https://doi.org/10.1016/j.trf.2020.01.016 Mayer, Sven, Le, Huy Viet, Nesti, Alessandro, Henze, Niels, Bülthoff, Heinrich H., Chuang, Lewis L. 2018. The Effect of Road Bumps on Touch Interac- tion in Cars. In Proceedings of the 10th International Conference on Auto- motive User Interfaces and Interactive Vehicular Applications (AutomotiveUI ‘18). Association for Computing Machinery, New York, NY, USA, 85–93. https://doi.org/10.1145/3239060.3239071 Meixner, G. et al. 2017. Retrospective and Future Automotive Infotainment
Systems—100 Years of User Interface Evolution. In: Meixner, G., Müller, C. (eds) Automotive User Interfaces. Human–Computer Interaction Series. Springer, Cham. https://doi.org/10.1007/978-3-319-49448-7_1 Ng, Alexander and Brewster, Stephen. 2017. An Evaluation of Touch and Pressure- Based Scrolling and Haptic Feedback for In-Car Touchscreens. In Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI ‘17). Association for Computing Machin- ery, New York, NY, USA, 11–20. https://doi.org/10.1145/3122986.3122997 Ulahannan, Arun, Cain, Rebecca, Thompson, Simon, Skrypchuk, Lee, Mouzakitis, Alex, Jennings, Paul, Birrell, Stewart. 2020. User expecta- tions of partial driving automation capabilities and their effect on informa- tion design preferences in the vehicle. Applied Ergonomics, 82, 102969. https://doi.org/10.1016/j.apergo.2019.102969